Every professional penetration test, whether against a network or a web application, starts with a shared phase: reconnaissance. Before scanning a port or trying an injection, you need to know exactly which domains, subdomains, email addresses, and servers the target organization actually uses. OSINT (Open Source Intelligence) means gathering exactly this information, purely from public sources available to everyone — without sending a single packet to the target system. theHarvester is one of the best-known free tools for automating exactly this.
Note: even though OSINT isn't literally an "attack" (since no traffic is sent directly to the target system), in a formal penetration test, the reconnaissance phase still needs to fall within the written scope of the engagement. Gathering information about an organization without coordination — even from public sources — isn't advisable outside the framework of a formal agreement.
What exactly does theHarvester find?
By searching sources like Google, Bing, certificate transparency logs (crt.sh), Shodan, and social networks, this tool compiles a list of the following for a domain: email addresses belonging to the organization's staff, known subdomains, associated IP addresses, and sometimes employee names from public profiles.
Step 1: Install
Already installed on Kali Linux. On other systems:
pip install theHarvester
Step 2: Run a basic scan with one source
theHarvester -d example.com -b google
The -d flag specifies the target domain, and -b specifies the search source (Google, here).
Step 3: Use multiple sources at once
theHarvester -d example.com -b all
The all flag tells theHarvester to use every supported source simultaneously (including Bing, DuckDuckGo, crt.sh, and — with an API key — Shodan and Hunter.io) — giving a more complete picture of the organization's public footprint.
Step 4: Limit results and save output
theHarvester -d example.com -b all -l 500 -f result
The -l flag limits the number of results to check, and -f saves the output as both HTML and JSON in a file named result — handy for attaching to a final pentest report.
Step 5: Analyze the output
The list of discovered subdomains usually reveals the organization's real attack surface — secondary servers, forgotten admin panels, or staging environments that sometimes have weaker security than the main server. Discovered email addresses are also useful in later phases (e.g., staff security-awareness assessments, within the scope of a formal engagement).
Summary
A good penetration test always starts with information gathering, not with running the first exploit. The more complete your picture of the target organization's public infrastructure, the more focused and effective the active scanning and reconnaissance phase becomes (with tools like Nmap, covered elsewhere in this series) — exactly the logical sequence we work through, step by step, in my penetration-testing courses.
Blog