Search engines index whatever a misconfigured server hands them, and anyone can query that index for your exposed files, login panels and secrets without ever touching your network. "Google dorking" is the practice of combining advanced search operators to find that material, and attackers use it as cheap, silent reconnaissance. This guide shows defenders how to run the same searches against their own domains on a schedule, what to look for, how to get exposed content removed properly, and why robots.txt is not the fix. It is written for security engineers, attack surface and vulnerability management teams, web and platform owners, and anyone who runs an authorised self-assessment.
Key takeaways
- Dorking finds what is already public. The search engine is not the vulnerability: the exposure comes from servers, storage and documents that were published by mistake, which is why the Google Hacking Database calls the result a "Googledork".
- Attackers see you first unless you look. MITRE ATT&CK lists search engine reconnaissance as T1593.002, and it leaves nothing in your logs, so your only defence is to search yourself before someone else does.
- A small set of operators does most of the work. site:, filetype:, inurl:, intitle:, quotes and the minus sign, scoped to domains you own, cover the high-value checks.
- robots.txt does not hide anything. Google states plainly that robots.txt is not a mechanism for keeping a page out of search, and a disallowed URL can still be indexed if other sites link to it.
- Fix the source, then clean the index. Delete or protect the content and rotate anything secret first; removal tools and noindex only deal with the search listing.
What dorking is and why defenders should do it
Every major search engine supports operators that narrow results by domain, file type, URL text or page title. Used together, they turn a general web search into a targeted query such as "spreadsheets on our domain that contain the word password" or "pages on our subdomains titled with a database admin tool's name". The OWASP Web Security Testing Guide treats this as the first information gathering test, WSTG-INFO-01, with the objective of identifying sensitive design and configuration information exposed directly on the organisation's website or indirectly through third-party services.
Three properties make it worth a defender's time:
- Silent for the attacker: the queries go to the search engine, not to you, so web server logs, WAF and IDS see nothing. ATT&CK groups this under T1593 Search Open Websites/Domains, alongside T1594 Search Victim-Owned Websites and T1596 Search Open Technical Databases.
- Persistent: an index can hold a document long after a developer thinks they removed it, and archives such as the Internet Archive Wayback Machine and Common Crawl keep copies independently.
- Catalogued: the Google Hacking Database (GHDB), now maintained by OffSec as part of Exploit Database after Johnny Long handed it over in November 2010, publishes thousands of queries in categories such as Files Containing Passwords, Sensitive Directories and Pages Containing Login Portals. Your adversaries already have the playbook.
Dorking sits inside the wider reconnaissance picture. See the OSINT and Reconnaissance Guide for the full programme and the Attack Surface Management Guide for continuous discovery of internet-facing assets.
The operators that matter
Google documents its operators in two places: the Search Help page for everyday operators and the Search Central page for debugging operators. No space goes between the operator and the term. Other engines support similar syntax, but behaviour differs, so test each one you rely on.
| Operator | What it does | Defender use | Caveat |
|---|---|---|---|
| site: | Limits results to a domain, URL or URL prefix; site:example.com includes subdomains | Scope every query to domains you own; find forgotten subdomains | Google says the list is not exhaustive and should not be used to count indexed URLs |
| filetype: | Limits results to a file type, based on the Content-Type header or extension | Find exposed .sql, .env, .log, .xlsx, .bak, .conf files | Only finds files the engine chose to index |
| inurl: | Matches text in the URL | Find admin paths, /backup/, /uploads/, phpinfo pages | Not in Google's current documentation; listed by OWASP WSTG |
| intitle: | Matches text in the page title | Find "Index of /" directory listings and named admin consoles | Same as above |
| intext: / inbody: | Matches text in the page body | Find error messages, stack traces, internal hostnames | Engine support varies |
| "exact phrase" | Exact match | Search for internal project codenames, unique error strings | Very specific strings may return nothing even when exposed |
| - (minus) | Excludes a term or site | Remove your marketing site from results to surface stray hosts | None |
| before: / after: | Restricts to documents last updated before or after a date | Review only what appeared since the last sweep | Depends on the engine's view of the update date |
The WSTG v4.2 text also lists cache: for viewing previously indexed content. Google's current operator documentation no longer includes it, so use the Wayback Machine or Common Crawl to see historical copies instead.
What to look for on your own domains
OWASP's list of what this test should uncover is a sound starting point: network diagrams and configurations, archived posts and emails from administrators, logon procedures and username formats, usernames, passwords and private keys, third-party or cloud service configuration files, revealing error messages, and development, test, UAT and staging versions of sites. The table turns that into query patterns, always scoped with site: to your own domains.
| Exposure class | Example query pattern (own domain only) | Why it matters |
|---|---|---|
| Open directory listings | site:example.com intitle:"index of" | A listing hands visitors a map of every file in the folder. See the Directory Listing and Exposed Backup Files Guide. |
| Database dumps and backups | site:example.com filetype:sql OR filetype:bak | Full customer data and password hashes in one download |
| Configuration and environment files | site:example.com filetype:env OR filetype:conf | Cloud keys, database strings and API tokens (ATT&CK T1552.001 Credentials In Files) |
| Office documents | site:example.com filetype:xlsx "confidential" | Staff lists, pricing, network diagrams, username formats for spraying |
| Log and debug output | site:example.com filetype:log | Session IDs, internal paths, tokens in URLs |
| Login and admin panels | site:example.com inurl:admin OR intitle:login | Unexpected management interfaces exposed to the internet |
| Error messages and stack traces | site:example.com "stack trace" OR "SQL syntax" | Framework versions, query structure, file paths |
| Non-production sites | site:example.com inurl:staging OR inurl:dev OR inurl:test | Weaker controls, real data copied from production |
| Third-party exposure | "example.com" site:pastebin.com, site:trello.com, site:docs.google.com | Your data published on someone else's platform |
The third-party row matters more than many teams expect. The JSONFormatter and CodeBeautify leak showed how saved, shareable pastes from online developer tools exposed cloud keys, CI secrets and Active Directory credentials belonging to organisations that never published anything themselves.
Where identity fits
The highest-impact dork results are usually non-human credentials: AWS access keys in a .env file, a database connection string in a backup, an API token in a log, a service account password in a spreadsheet. These keys rarely have MFA and often carry broad permissions, so a single indexed file can become a cloud compromise. The 230 Million AWS Cloud Environments Campaign 2024 fed on IAM keys harvested from exposed .env files, and the Indian Government Breach 2021 involved credentials recovered from exposed .git and .env files. Treat every credential a dork finds as compromised: rotate it, review its activity, then remove the file. See the OWASP NHI Top 10 NHI2 Secret Leakage Guide and Leaked secret breaches.
Running a defensive dorking programme
- Get written scope. List the domains and brands you are authorised to assess, and agree with legal that the programme only searches public indexes and never downloads or opens third-party data beyond what is needed to confirm ownership. Searching is passive; downloading another organisation's leaked file is not your call to make.
- Build the domain list. Pull registered domains, certificate transparency subdomains and DNS records from your attack surface inventory, so site: queries cover more than the main website.
- Build a query library. Start from the table above and the GHDB categories that match your stack (Files Containing Passwords, Sensitive Directories, Error Messages, Pages Containing Login Portals). Version the library and record why each query exists.
- Run it against more than one engine. Google, Bing and DuckDuckGo index differently, and OWASP also lists Baidu, Startpage, Common Crawl and the Wayback Machine. Device search engines such as Shodan answer a different question (exposed services rather than documents) and belong in your attack surface programme.
- Automate within the rules. Use the search engine's official API or a licensed provider rather than scraping results pages, which breaches most engines' terms. Schedule weekly runs for high-value queries and monthly for the long tail; use after: to show only new results.
- Triage every hit. Confirm ownership, classify the data, and open a ticket with the asset owner. Anything containing credentials or personal data goes straight to incident handling.
- Fix the source, then clean the index (next section), and re-run the query to confirm.
- Feed back into prevention. Every finding should change something upstream: a web server default, a pipeline check, a storage policy or a training point.
Removing exposed content properly
Getting a result out of the index is the last step, not the first. If a file containing a secret was public, assume it was copied.
| Action | What it achieves | What it does not do |
|---|---|---|
| Rotate or revoke exposed credentials | Makes the leaked secret useless | Does not remove the file |
| Delete or update the content on the server | Google calls this the most secure option; the URL stops serving the data | Does not remove copies in archives or attacker hands |
| Password-protect the location | Keeps crawlers and visitors out; Google recommends it for private content | Does not help if credentials were already exposed |
| noindex meta tag or X-Robots-Tag header | Keeps the page out of Google results | Visitors and engines that ignore noindex can still reach it |
| Google Search Console Removals tool | Hides a URL from results within about a day, for roughly six months | Temporary; the content must also be fixed at source |
| robots.txt Disallow | Asks compliant crawlers not to fetch the URL | Does not keep the URL out of the index and publicly advertises the path you want hidden |
Google's removal guidance also warns that the same page may be reachable at several URL variants (case and query-string differences), and each one needs blocking. Request removal from every engine that indexed the content, and check the Wayback Machine, which has its own exclusion process.
Choosing the approach for your organisation
- Small organisations: a quarterly manual sweep of 20 to 30 queries against your main domains, done by one person with a checklist, catches most accidental publication. Add Search Console and Bing Webmaster Tools alerts for your properties.
- Mid-sized organisations: script the query library through a search API, store results, and diff each run so only new URLs reach a human. Fold findings into vulnerability management tickets.
- Large or highly targeted organisations: use an external attack surface management or digital risk protection service that already runs search engine, paste site and code repository monitoring, and keep an internal query library for anything specific to your naming and stack. See the Attack Surface Management Buyer's Guide.
- Build versus buy trade-off: in-house queries are cheap and precise but miss sources you do not think of; services cover breadth but produce noise and need tuning. Most mature teams run both.
Detection and measurement
You cannot see an attacker's searches, but you can see what happens next. Monitor web logs for direct requests to file types that should never be requested (.sql, .bak, .env, .git/config) and for requests whose referrer is a search results page landing on unexpected paths. Track how quickly findings close. Useful metrics:
- Exposure count per sweep: number of confirmed sensitive results, trending down.
- Time to remediate: median hours from discovery to source fix, with a tighter target for credential findings.
- Credential findings rotated: percentage of exposed secrets rotated within 24 hours (an NHI Mgmt Group recommendation for a target).
- Recurrence rate: findings on the same host or pipeline within 90 days, which shows whether the upstream fix worked.
- Coverage: share of owned domains and subdomains included in the query scope.
Common failure patterns
- Using robots.txt as a hiding place. Listing /backup/ or /admin/ in robots.txt tells every reader where to look, and Google confirms a disallowed URL can still appear in results.
- Removing the search result but not the file. The Removals tool expires after about six months and other engines and archives keep their copies.
- Deleting the file but not rotating the secret. The 230 Million AWS Cloud Environments Campaign 2024 shows that exposed .env keys are harvested at scale; once indexed, the key must be treated as stolen.
- Only checking the main website. Staging hosts, marketing microsites and old acquisitions are where most results appear; scope site: queries to every domain you own.
- Ignoring third-party platforms. Paste sites, public boards and online developer tools publish your data without touching your infrastructure.
- Scraping results in breach of terms. Automated scraping gets blocked and can breach the engine's terms; use official APIs or a provider.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| MITRE ATT&CK | T1593.002 Search Engines; T1594 Search Victim-Owned Websites; T1596 Search Open Technical Databases | The adversary behaviour this programme anticipates |
| MITRE ATT&CK | T1552.001 Credentials In Files | What attackers do with indexed config files |
| OWASP WSTG | WSTG-INFO-01 Conduct Search Engine Discovery Reconnaissance for Information Leakage | The test procedure |
| NIST SP 800-53 Rev 5 | AC-22 Publicly Accessible Content; RA-5 Vulnerability Monitoring and Scanning; CA-8 Penetration Testing | Review of public content, and discovery as part of scanning and testing |
| NIST CSF 2.0 | ID.RA-01; PR.DS-01 | Vulnerabilities identified and recorded; data at rest protected |
| CIS Controls v8.1 | Control 3 Data Protection; Control 16 Application Software Security | Data handling and secure web application configuration |
Practitioner checklist
- Get written authorisation listing the domains and brands in scope.
- Build the domain list from your attack surface inventory, including subdomains and acquired brands.
- Maintain a versioned query library covering listings, dumps, config files, logs, documents, panels and errors.
- Run queries on at least two search engines plus the Wayback Machine, through official APIs.
- Route credential findings to incident handling and rotate the secret before removing the file.
- Remove at source, then use noindex, password protection and the Removals tool as needed.
- Take sensitive paths out of robots.txt and protect them with authentication instead.
- Alert on web requests for .sql, .bak, .env and .git paths.
- Report time to remediate and recurrence monthly, and fix the upstream cause of each finding.
Standards and references
- OWASP: WSTG-INFO-01 Conduct Search Engine Discovery Reconnaissance for Information Leakage
- MITRE ATT&CK: T1593.002 Search Engines
- Google: Refine Google searches
- Google Search Central: Overview of Google search operators
- Google Search Central: Introduction to robots.txt
- Google Search Central: Remove a page hosted on your site from Google
- OffSec Exploit Database: Google Hacking Database
Related NHI Mgmt Group resources: OSINT and Reconnaissance Guide · Attack Surface Management Guide · Directory Listing and Exposed Backup Files Guide · Exposed .git Directory Response