TryHackme: Content Discovery

Discover hidden web content using manual techniques, OSINT, and Gobuster enumeration.

Task 1: Introduction

Learning Objectives

By the end of this room, you’ll be able to:

• Manually discover hidden content using robots.txt, sitemap.xml, favicons, HTTP headers, and framework analysis
• Use OSINT tools, including Google dorking, Wappalyzer, the Wayback Machine, GitHub, and S3 bucket enumeration
• Use Gobuster to brute-force directories, subdomains, and virtual hosts
• Apply a structured content discovery methodology in a penetration test

Task 2: Manual Discovery — common Files

Q1: What is the directory in robots.txt that isn’t allowed to be viewed by web crawlers?

Ans: /staff-portal

Q2: What is the path of the secret area found in sitemap.xml?

Ans: /s3cr3t-area

Task 3: Manual Discovery — Headers & Framework Stack

Q1: What is the flag value from the X-FLAG header?

Ans: THM{HEADER_FLAG}

Q2: What is the flag from the framework’s administration portal?

Get Md. MONTASHIR MOBINE RIFAT’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Ans: THM{CHANGE_DEFAULT_CREDENTIALS}

Task 4: OSINT- Search Engines & Web Tools

Q1: What Google dork operator limits results to a specific site?

Ans: site:

Q2: What online tool and browser extension identifies what technologies a website is running?

Ans: Wappalyzer

Task 5: OSINT — Repositories & Archives]

Q1: What is the website address for the Wayback Machine?

Ans: https://web.archive.org/

Q2: What URL format do Amazon S3 buckets end in? (Answer starts with a .)

Ans: .s3.amazonaws.com

Task 6: Automated Discovery- Gobuster Fundamentals

Q1: What is the name of the directory beginning with /mo that was discovered?

Ans: /monthly

Q2: What is the name of the log file that was discovered?

Ans: /development.log

Task 7: Automated Discovery- Subdomains & Virtual Hosts

Q1: Apart from dns and -w, which shorthand flag is required for dns mode?

Ans: -d

Q2: How many virtual hosts on acmeitsupport.thm respond with status code 200?

Ans: 3

Task 8: Conclusion

Content discovery is one of the most important phases of web application reconnaissance. The techniques in this room work together: manual checks surface quick wins, OSINT finds information the target has already shared publicly, and automated tools cover the breadth that neither approach can do alone.

Here’s a quick recap of what was covered: