LLM-generated passwords are predictable and now detectable in the wild

At a glance

What this is: GitGuardian's analysis shows that LLM-generated passwords are statistically biased, detectable, and already appearing in public code at measurable scale.

Why it matters: IAM and NHI teams need to assume AI-generated secrets can be identified, predicted, and abused, which changes how password generation, code review, and agent guardrails should work.

By the numbers:

• The most predicted providers, Anthropic, Qwen, and Google, represent 63% of all occurrences.

👉 Read GitGuardian's analysis of LLM-generated passwords and wild exposure patterns

Context

LLM-generated passwords are a governance problem because they look random to users while still carrying statistical fingerprints that make them easier to classify and potentially easier to crack. For IAM and NHI programs, the issue is not just weak secrets, but the decision to let autonomous or semi-autonomous systems create credentials outside controlled generation and storage workflows.

The article extends earlier work by testing whether those fingerprints can be detected across a broader model set and then found in real-world code. That matters because AI-assisted development now intersects with secrets management, code review, and non-human identity controls in the same pipeline. The use of AI agents to generate and hardcode passwords is not the typical enterprise starting position, but it is becoming a realistic one.

Key questions

Q: How should security teams prevent AI tools from generating weak passwords?

A: Security teams should prohibit LLMs from creating production credentials and require a vault or password manager to generate every secret. The control should sit in the workflow, not in policy text alone. If a model can emit a password that later reaches code, logs, or deployment files, the team has already lost control of the secret lifecycle.

Q: Why are AI-generated passwords risky even when they look complex?

A: They are risky because complexity is not the same as entropy. LLMs often produce passwords with repeatable structures, shared substrings, and model-specific bias, which can make them statistically identifiable and easier to guess than truly random values. A secret that looks strong to humans can still be a weak NHI control if it was generated by a model.

Q: What is the difference between vault-generated secrets and LLM-generated secrets?

A: Vault-generated secrets are produced by controlled random processes and can be tracked, rotated, and audited. LLM-generated secrets are text outputs shaped by probability, which makes them harder to trust as credentials. For IAM teams, the difference is governance, not just appearance: one supports lifecycle control, the other introduces hidden predictability.

Q: How can organisations detect AI-generated passwords in source code?

A: They should combine secret scanning with pattern analysis that looks for repeated substrings, common character sequences, and other model-like structures in repositories and configuration files. Detection should be used to triage review and remediate exposed secrets quickly, not as a substitute for secure generation and rotation controls.

Technical breakdown

Why LLM-generated passwords leave statistical fingerprints

LLMs are optimized to produce probable sequences, not high-entropy secrets. That creates repeatable structures such as common character orderings, shared substrings, and similar starting positions across model families. A password may still look complex to a human, yet remain statistically distinguishable from a truly random value because its construction reflects language-model bias rather than cryptographic randomness. In NHI terms, this is a generation problem, not just a storage problem: if the secret is predictable at birth, downstream controls inherit that weakness. Practical implication: password creation must stay inside a vault or password manager, not inside a model prompt.

Practical implication: Keep secret generation out of LLM workflows and require cryptographic randomness for every credential.

How Markov chains can classify model-generated passwords

A Markov chain models the probability of one character following another, which makes it useful for detecting recurring structures in generated passwords. When trained on LLM-produced samples, the chain can score new passwords by how closely they match those learned patterns. That is why the technique can identify likely model output and, in some cases, infer the provider or model family. The same statistical profile also creates an offensive opportunity, because pattern-aware guessing is more efficient than blind brute force. Practical implication: defenders should assume pattern leakage can support both attribution and cracking attempts.

Practical implication: Use pattern detection as a signal for review, not as proof that a secret is safe because it appears complex.

Why AI agents can hardcode secrets even without human intent

The article shows that AI agents can independently generate and commit passwords into Terraform, JSON, and configuration files. That creates a non-human identity risk because the agent is not just recommending a secret, it is participating in secret creation and persistence. Once that happens, the secret may traverse logs, prompts, developer endpoints, and source control before a human ever reviews it. This is a lifecycle failure across creation, storage, and exposure. Practical implication: agent workflows need policy gates that inspect hook events and block secrets before they reach code or commit history.

Practical implication: Treat agent-generated credentials as governed outputs and scan agent hooks for secret creation events.

Threat narrative

Attacker objective: The attacker aims to recover predictable AI-generated secrets and use them to access systems, configuration files, or service accounts without triggering high-friction controls.

1. Entry occurs when an AI agent or developer asks an LLM to generate a password and the secret transits prompts, logs, or endpoint tooling before it is stored.
2. Escalation follows when the same password is hardcoded into source, Terraform, or environment files and becomes reusable across systems or deployments.
3. Impact arrives when predictable passwords enable online guessing, secret reuse, or unauthorized access to application and database services.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• AI LLM hijack breach — attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI-generated passwords are an NHI governance failure, not a convenience shortcut. The core problem is that LLMs optimize for probable text, while password generation requires unpredictable output. Once teams allow a model to create credentials, they introduce statistical bias into an identity control that should be entropy-driven. The practical conclusion is simple: treat AI-generated secrets as disallowed unless they are generated and stored by controlled cryptographic tooling.

Detectability changes the risk profile because weak secret generation is now measurable at scale. GitGuardian's findings show that LLM-generated passwords can be identified in leaked code and configuration data, which means the problem is not hidden. That creates both a defensive opportunity and an offensive one. Security teams should assume predictable generation patterns can help adversaries prioritize cracking attempts and should remove the model from the secret-generation path entirely.

Ephemeral AI workflows still create persistent secret debt. Even when an agent only touches a prompt or hook event once, the password it generates can survive in source control, CI logs, environment files, and downstream deployment artifacts. That is the identity blast radius problem: a short-lived action can create long-lived access exposure. The right response is to govern secret creation at the workflow boundary, not after the secret has already been committed.

Agent-generated credentials need explicit policy controls, not informal developer judgment. The article shows that some LLM passwords were created in connection strings and code commits, including by AI agents. That means the control plane must cover generation, review, and commit-time inspection. Teams that do not distinguish between human-authored and agent-authored secret creation will miss a growing source of NHI sprawl.

Pattern-aware detection should complement, not replace, vaulting and rotation. Statistical classifiers can help identify suspect passwords in repos and monitoring pipelines, but they do not make weak secrets acceptable. Governance should still require vault-based generation, rotation, and secret scanning at commit time. The practitioner takeaway is that detection is a backstop, while secure generation remains the primary control.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
• Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
• That gap points to the same governance problem this post raises, so teams should align secret handling with OWASP NHI Top 10 guidance on agent misuse and identity risk.

What this signals

LLM-generated credentials should be treated as an identity creation flaw, not as a low-severity code hygiene issue. As AI-assisted development moves deeper into CI/CD, the practical control point shifts to generation time and commit time. Teams that already use OWASP Agentic AI Top 10 language for tool misuse should extend that thinking to secrets created by agents, because identity risk now starts before deployment.

Secret lifecycle controls now need to cover agent output as well as human input. When a model can create a password that later appears in Terraform or .env files, the boundary between development assistance and identity exposure disappears. That means your programme should review hook-based scanning, vault enforcement, and secret creation policy together, not as separate controls.

The persistence of these passwords in public code and configuration also means that discovery and remediation must be faster than the average attacker workflow. If an exposed secret can be acted on within minutes, the operational question is whether your monitoring and rotation process can respond at the same speed.

For practitioners

• Ban LLMs from generating production secrets Require password managers or vaults to generate all human and machine credentials, and block prompts that ask models to produce reusable passwords.
• Scan agent hook events for secret creation Inspect AI agent hook events, commit hooks, and CI logs for passwords, API keys, and certificates before they reach source control.
• Flag predictable password patterns in code review Add secret scanning rules for repeated substrings, common character orders, and weakly random-looking credentials in Terraform, JSON, and .env files.
• Separate agent output from identity provisioning Do not let autonomous coding agents create credentials that can be reused across environments, especially in database strings and infrastructure code.
• Map LLM-generated secrets to NHI controls Treat model-created passwords as non-human identity artifacts and review them under secret lifecycle, access review, and rotation policies.

Key takeaways

• LLM-generated passwords are not secure by default because model output follows probability, not cryptographic randomness.
• GitGuardian's analysis shows that these passwords are detectable in real repositories and configuration files at measurable scale.
• The right response is to move secret generation into vaults, scan agent outputs, and treat AI-created credentials as governed NHIs.

Key terms

• LLM-generated password: A password created by a large language model rather than by a cryptographic random generator. These secrets can look complex while still carrying statistical patterns from model output, which makes them weaker than they appear and harder to trust in identity and access workflows.
• Secret lifecycle: The full path of a credential from creation to storage, use, rotation, and retirement. In NHI programmes, lifecycle control matters because a secret can become exposed or reusable at any stage, especially when generated by tools that are not designed for entropy or auditability.
• Identity blast radius: The range of systems, logs, repositories, and services that can be affected when a credential is created or exposed in one workflow. For AI-assisted development, the blast radius often extends beyond the original prompt to code, deployment artifacts, and downstream access paths.

What's in the full report

GitGuardian's full analysis covers the operational detail this post intentionally leaves for the source:

• The model-by-model breakdown of password bias, including provider-level uniqueness and repeated substrings.
• The Markov-chain scoring method used to classify likely LLM-generated passwords in public GitHub data.
• The repository and file-type breakdown showing where AI-generated secrets most often appear in real codebases.
• The detection workflow for distinguishing likely AI-generated passwords from ordinary weak human passwords.

👉 The full GitGuardian post covers classification thresholds, provider counts, and the detection method in more detail.

Deepen your knowledge

LLM-generated passwords and secret lifecycle governance are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If your teams are beginning to govern AI-assisted development and agent output, it is a practical place to start.