Generic API Key Detection Guide: Entropy, Validity Checks and False Positives

Most secrets in code do not carry a vendor prefix, so scanners fall back on generic rules that combine a keyword, an assignment and an entropy score, and those rules produce most of the noise and most of the misses. An internal service key, a database password or a partner API key looks like any other random string, and the tools that catch ghp_ or AKIA reliably have far less to go on. This guide explains how generic detection works in Gitleaks, detect-secrets, TruffleHog and GitHub secret scanning, what Shannon entropy can and cannot tell you, how validity checking turns a finding into a priority, and how to tune false positives without opening blind spots. It is for AppSec engineers, platform teams and anyone running a secret scanning programme. Start with the Git Secret Scanning Guide for programme basics; this page goes deeper on the generic case.

Key takeaways

• Generic rules are keyword plus entropy. Gitleaks' generic-api-key rule looks for words such as key, secret, token or password next to an assignment, then requires a Shannon entropy of at least 3.5 in the captured value.
• Entropy is a weak signal on its own. UUIDs, hashes and build IDs score as high as real keys, and short or human-chosen secrets score low; entropy filters noise but cannot confirm a secret.
• Validity checks set priority, not scope. A verified-live result goes first, but "unknown" is not "safe": GitHub does not validity-check generic patterns at all, and HMAC-style secrets often cannot be tested.
• Tune with allowlists and baselines, never by raising thresholds globally. Path, stopword and fingerprint suppressions remove known noise; a higher entropy threshold silently drops real secrets.
• Fix the source of generic secrets. Give internal tokens a distinctive prefix and checksum so they become provider-style patterns that scanners can match precisely.

How identity and access work for generic API keys

A generic API key is a bearer credential with no self-describing format. Detection has to infer it from context. The table shows the main credential families and why they end up in generic rules.

Every unmatched generic secret is a non-human identity without an owner on record. Part of triage is working out which system issued it and who owns it, which is why an NHI inventory makes scanning far more effective. See the API Key Management Guide.

Admin roles and privileged access

A secret scanning programme concentrates sensitive data, so the scanner and its results need access control of their own.

• Who sees findings: results contain live secrets. Restrict alert views to repository admins and the security team, and use Gitleaks' --redact option (full or partial, as a percentage) so CI logs and reports do not reprint the secret.
• Who can dismiss: dismissing a finding as a false positive or adding a gitleaks:allow comment is a security decision. Require review for suppressions in shared configuration and track who dismissed GitHub alerts.
• Who can verify: validity checking means presenting a found credential to the issuing service. Limit it to the security team or the scanner's own service identity, and only for credentials your organisation owns or is authorised to test.
• Scanner credentials: organisation-wide scans need tokens with read access to every repository; store them in a vault and scope them read-only.

Integrations, API tokens and service accounts

How the main tools detect generic secrets

Tool behaviour as documented on 27 October 2026. Run a provider-pattern scanner and a generic scanner together; they fail in different ways. For the full comparison of rule sets see the Secret Scanner Rule Coverage Comparison Guide.

Entropy in practice

Shannon entropy measures how unpredictable the characters in a string are, in bits per character. A string drawn from hexadecimal can reach at most 4.0 and one drawn from the base64 alphabet at most 6.0, and short strings rarely approach the maximum. Some worked values, computed for this guide:

The practical rule: use entropy to discard obviously non-random values, rely on keyword context and structure for precision, and accept that low-entropy human passwords need their own detection.

Writing a precise custom rule

Once triage shows that a particular internal system issues the real secrets, replace guesswork with a rule built for that format. In Gitleaks a rule combines these parts, and the same thinking applies to custom patterns in other tools:

• id and description: name the issuing system so findings route to the right owner.
• keywords: short literal strings that must appear for the regex to run at all; a distinctive prefix is the ideal keyword and makes scanning fast.
• regex with a capture group: match the exact length and alphabet of the secret, and capture only the secret itself, because entropy and stopwords apply to the captured group.
• entropy: set it from measured values of real keys of that format, not from the generic default.
• allowlists: add per-rule exceptions for known example values and test paths, and use targetRules in a shared global allowlist when several rules need the same exception.

Test every new rule against a set of revoked real keys and a corpus of known false positives before enforcing it, and keep those samples in a protected test repository rather than in the rule file.

Where generic secrets hide beyond source code

Repository scanning finds only part of the problem. Generic credentials also sit in CI variables printed to logs, container image layers, infrastructure state files, wiki pages, ticket attachments and chat exports. Point the same tuned rules at those sources where your tools support them, since TruffleHog, for example, can scan container images and cloud storage buckets as well as Git. See the Terraform and IaC Pipeline Credential Security Guide.

Logging and detection

Validity checks: turning findings into priorities

• Verified or active: the issuing service accepted the credential. Revoke and rotate first, then investigate use. GitHub shows these as active; TruffleHog marks them verified.
• Unknown: the tool could not test it, either because the type is unsupported (all generic patterns on GitHub) or the check failed. Treat as live until the owner confirms otherwise. GitHub labels these "Possibly active secret".
• Inactive: the service rejected it. Still confirm no misuse happened while it was live, and remove it from history if policy requires.

Validity matters at scale. When GitGuardian and researcher Tom Forbes scanned every package on PyPI, they found 3,938 unique secrets and 768 that still worked; see the PyPI Secrets Exposure 2023. Without validity data, every one of those findings would have needed the same urgent handling.

Verification has costs. It sends the secret to a third party, generates a login in the provider's logs that may itself trigger the owner's alerts, and can count against rate limits. Only verify credentials you own or have authority to test, through the scanner's controlled integration, and record that you did.

Metrics

• Precision: share of generic findings confirmed as real secrets in triage, tracked per rule.
• Live rate: share of findings verified active, and time from detection to revocation for those.
• Suppression health: number of allowlist entries and dismissed alerts added per month, with reviewer.
• Prefix coverage: share of internally issued credentials that carry a scannable prefix.
• Unknown backlog: generic findings with no owner identified after 7 days.

Hardening baseline

Tuning without blind spots

1. Baseline existing history: run the generic rules across all repositories and store the report as a baseline (gitleaks git --baseline-path or a .secrets.baseline file), so new findings stand out while old ones are triaged separately.
2. Triage a sample: label at least a few hundred generic findings as real, test data or noise, and group the noise by cause (UUIDs, hashes, lockfiles, test fixtures, documentation examples).
3. Suppress by cause: use path allowlists for lockfiles and vendored code, stopwords for known placeholder values, and fingerprints in .gitleaksignore for individually reviewed findings. Avoid global regexes allowlists that match broad patterns.
4. Keep the threshold: do not raise the global entropy threshold to cut noise; if a rule is noisy in one context, add a targeted allowlist or a narrower custom rule.
5. Add custom rules for your own formats: once triage shows which internal systems issue the real secrets, write precise rules for them and keep the generic rule as a backstop.
6. Enforce at commit: run the tuned configuration in pre-commit hooks and CI, failing builds on new findings, and turn on GitHub push protection for provider patterns.
7. Review suppressions quarterly: remove allowlist entries for paths or values that no longer exist.

Design secrets that scanners can find

The durable fix for generic noise is to stop issuing generic secrets. Give every internally generated credential a distinctive, documented prefix and a fixed length, ideally with a checksum, so it can be matched with near-zero false positives and validated offline before any API call. Then register the pattern with your scanners. This is the same approach large providers use, and it turns your internal keys from generic findings into precise ones. See the Secrets Management Guide.

Small and large programmes

A small team can run Gitleaks defaults in CI, review findings weekly and keep a short .gitleaksignore. A large programme needs per-rule precision metrics, a central configuration with reviewed allowlists shared across repositories, automated owner lookup from code ownership data and a ticketing workflow that tracks every live finding to revocation.

Framework mapping

In the OWASP Non-Human Identities Top 10, this work addresses NHI2 Secret Leakage. See the OWASP NHI Top 10 NHI2 Secret Leakage Guide.

Common misconfigurations

• Turning generic rules off: teams disable generic-api-key after one noisy week and lose all coverage for internal credentials.
• Raising entropy globally: a threshold of 4.5 for every rule removes UUID noise but also every short or alphanumeric secret.
• Allowlisting whole directories: excluding test/ or config/ hides the places real secrets are most often committed.
• Treating unknown as inactive: generic findings cannot be validity-checked on GitHub, so "unknown" is the normal state, not a sign the key is dead.
• Verifying without authority: testing a third party's leaked key can breach terms or law; report it to the owner instead.
• Printing secrets in scan logs: running a scanner in CI without redaction copies every finding into build logs.

Practitioner checklist

• Run a generic rule set alongside provider-pattern scanning across code, history and CI configuration.
• Create a baseline and triage existing generic findings separately from new ones.
• Measure per-rule precision and tune with path, stopword and fingerprint suppressions, not global thresholds.
• Add password and connection string detection for low-entropy secrets.
• Prioritise findings by validity state and treat unknown as live.
• Restrict who can view, dismiss and verify findings, and redact secrets in scan output.
• Write custom rules for internally issued credential formats.
• Give new internal credentials a distinctive prefix, fixed length and checksum.
• Review allowlists and dismissed alerts quarterly.

Standards and references

• Gitleaks: Default rule configuration
• Gitleaks: Documentation
• Yelp: detect-secrets
• Truffle Security: TruffleHog
• GitHub Docs: Supported secret scanning patterns
• GitHub Docs: Evaluating alerts from secret scanning
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: Git Secret Scanning Guide · Secrets Scanning Buyer's Guide · API Key Management Guide · Secret Scanner Rule Coverage Comparison Guide