LLM-generated passwords are predictable and now detectable in the wild

At a glance

What this is: This analysis shows that LLM-generated passwords leave statistical fingerprints that make them detectable in the wild and therefore unsuitable as a trustable secret source.

Why it matters: IAM, PAM, and NHI teams need to treat AI-generated passwords as a governance failure mode because predictability, leakage pathways, and agent-generated secrets all widen exposure.

By the numbers:

• Anthropic, Qwen, and Google represented 63% of all predicted LLM-generated password occurrences.
• LLM-generated passwords were committed at an average rate of 1,500 per week during the study timeframe.

👉 Read GitGuardian's analysis of LLM-generated passwords and secret predictability

Context

LLM-generated passwords are not random in practice, even when they look complex. Because language models are optimised to predict likely sequences, they tend to reproduce recurring character patterns, repeated substrings, and model-specific fingerprints that can be detected at scale.

For identity security teams, that makes password generation by AI more than a convenience issue. It creates a governance gap across NHI, secrets management, and AI-assisted development, because the problem is not only weak passwords but also leaked, traceable, and machine-produced secrets entering code and configuration workflows.

Key questions

Q: What breaks when teams let LLMs generate passwords?

A: Password generation breaks when teams use an LLM as the source of a secret, because the output is shaped by probability rather than entropy. That makes the password more predictable, easier to classify if leaked, and more likely to be reused or hardcoded in code and configuration files.

Q: Why are AI-generated passwords risky even when they look complex?

A: They are risky because complexity is not the same as entropy. LLMs often produce passwords with repeatable structures, shared substrings, and model-specific bias, which can make them statistically identifiable and easier to guess than truly random values. A secret that looks strong to humans can still be a weak NHI control if it was generated by a model.

Q: How should security teams handle secrets in AI-generated code?

A: Security teams should treat AI-generated code as another source of credential exposure, not as a special case. The right response is broad secrets discovery, fast remediation, and ownership mapping across repositories, pipelines, chat systems, and endpoints. If the organisation cannot inventory where a secret exists, it cannot safely rotate or revoke it.

Q: Should organisations allow AI agents to use production credentials?

A: Only if those credentials are task-scoped, closely monitored, and revocable without affecting unrelated systems. In most cases, production credentials create unnecessary blast radius. A safer pattern is short-lived access, explicit approvals for non-read actions, and strong separation between agent identity and human privilege.

Technical breakdown

Why LLM-generated passwords are statistically predictable

LLMs do not generate secrets from entropy in the way a proper password generator does. They produce likely text sequences, which means the output is shaped by training data and token probabilities rather than uniform randomness. In this article, that shows up as repeated character ordering such as upper, digit, symbol, lower, along with recurring substrings across models and providers. Those fingerprints make classification possible, and they also make cracking more efficient than brute force because an attacker can prioritise the same statistical structure the model tends to emit.

Practical implication: do not treat model-generated passwords as equivalent to entropy-driven secrets.

How Markov chains detect model fingerprints in leaked passwords

A Markov chain is a probabilistic model that predicts the next character from the previous state. Here, it is used in reverse as a classifier by training on known LLM-generated passwords and then scoring candidate passwords in leaked datasets. That approach worked well enough to identify the likely model or provider in a majority of cases, which means the password itself becomes an attribution signal. In other words, the output is not just weak, it is recognisable as machine-generated, which makes it easier to hunt at scale in exposed repositories and secret stores.

Practical implication: assume leaked passwords may be machine-attributable and searchable, not opaque.

Why AI agents turn password generation into an identity governance issue

The article’s more important governance point is that AI agents can independently generate and hardcode passwords into code, Terraform, and commits. That shifts the problem from one-off user behaviour to machine-produced secret creation inside development workflows. Once an agent can create the secret, place it in code, and persist it in version control or configuration files, traditional review processes see only the artifact, not the decision context. That is an NHI governance problem because the secret now exists inside an identity lifecycle the organisation did not consciously design.

Practical implication: govern secret issuance inside agent workflows, not only after secrets appear in code.

Threat narrative

Attacker objective: The attacker’s objective is to recover or guess predictable passwords and use them to access systems, credentials, or connected services before the secret is rotated or detected.

1. Entry occurs when a user or AI agent generates a password through an LLM instead of a proper secret generator, or when an agent hardcodes the password directly into code or configuration.
2. Credential exposure follows when that password transits provider infrastructure, logs, commits, .env files, or source repositories, creating multiple reuse and discovery paths.
3. Escalation happens when predictable structure lets defenders or attackers classify the password as LLM-generated and use pattern-based guessing or cracking rather than brute force.
4. Impact is unauthorized access to services, databases, or third-party systems that relied on the predictable password as a valid credential.

Breaches seen in the wild

• LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.
• McKinsey AI platform breach: McKinsey AI platform hack exposed 46M chats and sensitive data.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 150+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Predictable secret generation is a governance failure, not a convenience feature. A password produced by an LLM is not simply user-chosen with AI assistance, it is a machine-generated artifact shaped by probability. That makes it fundamentally different from entropy-based secret issuance and far more exposed to pattern-based identification. Practitioners should treat AI-generated secrets as a controlled identity lifecycle event, not as a productivity shortcut.

LLM-generated passwords create an identity blast radius because the secret often enters multiple systems at once. Once a password is generated in a chat, copied into a repo, or inserted into an environment file, it may transit the provider, the endpoint, the CI pipeline, and the source control history. That multiplies exposure windows and breaks the assumption that password creation is a private, local act. The implication is that secret handling controls must follow the generation point, not just the final storage point.

AI agents make the governance gap sharper because they can originate secrets without human review at the moment of creation. This is not the same as a developer pasting a weak password by hand. When an agent generates and hardcodes a secret, the organisation loses the decision trail that access reviews and code review culture normally depend on. Practitioners should interpret agent-authored secrets as evidence that lifecycle governance has moved upstream into runtime behaviour.

LLM-generated passwords should be classified as predictable secret entropy collapse: the output looks like a credential but does not carry the unpredictability that credential controls assume. That assumption collapse matters because many secret policies focus on length or complexity rather than true unpredictability and provenance. The implication for identity governance is that validation must address secret source, generation method, and placement in workflow, not only static password rules.

Secret governance must now span NHI and agentic AI together. The same control family that governs service-account keys, API tokens, and certificates now has to account for AI-created passwords that show up in code and configuration. That does not mean every LLM output is a breach, but it does mean the control boundary has shifted from the vault to the workflow. Practitioners should align their NHI and AI governance models around issuance provenance and not just storage location.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• According to Forrester Research, a single password reset can cost around $70.
• Read next: LLM Provider API Key Security and LLMjacking Guide

What this signals

Predictable secret entropy collapse: when a model produces passwords, the output can look compliant while still carrying detectable structure that attackers can exploit. That means secret generation must be treated as a provenance problem, not just a formatting problem.

The operational signal is not the presence of AI in the workflow but the appearance of AI-made secrets in repos, .env files, and agent-authored commits. Teams that can see those artifacts need a governance model that validates how the secret was created and where it was allowed to land.

According to the State of Secrets Sprawl 2026, AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers. That trend reinforces the need to govern AI secret creation as part of the broader NHI estate.

For practitioners

• Disable LLMs as password generators Require vaults or dedicated password managers to generate all production secrets, and block AI-generated passwords from being accepted as source-of-truth credentials.
• Scan agent hooks for secret creation Inspect AI agent hooks, prompts, and commit outputs for passwords, API keys, and other secrets before they reach repositories or deployment files.
• Treat predictable passwords as compromised-by-design Flag any password created by an LLM for immediate replacement, because recognisable structure makes it easier to classify and attack later.
• Govern agent-authored secrets in code review Add review checks for Terraform, .env files, and configuration commits where an agent may have inserted a credential without human intent review.

Key takeaways

• LLM-generated passwords are structurally different from entropy-based secrets because their patterns can be detected, classified, and attacked.
• The article’s scan of 34 million GitHub passwords found 28,000 likely LLM-generated examples, which shows the issue is real even if not yet dominant.
• The practical control point is secret issuance inside AI and developer workflows, where vault generation, review, and blocking of AI-made credentials can stop exposure earlier.

Key terms

• LLM-generated password: A password created by a large language model rather than by a cryptographic random generator. These secrets can look complex while still carrying statistical patterns from model output, which makes them weaker than they appear and harder to trust in identity and access workflows.
• Predictable Secret Entropy Collapse: A condition where a secret looks random to a human reviewer but does not provide true unpredictability. In practice, this means the credential can be identified through pattern analysis or statistical bias, so the secret fails the basic security expectation that passwords resist guessing.
• Access Provenance: Access provenance is the record of how an identity was created, approved, used, and withdrawn. In NHI governance, it is the evidence trail that lets teams prove an account is legitimate, explainable, and still within its intended access boundary.
• Agent-Authored Secret: A credential created, inserted, or hardcoded by an AI agent during a workflow. This is an identity governance concern because the agent may originate the secret without human review at the moment of creation, which breaks normal approval and traceability assumptions.

What's in the full report

GitGuardian's full article covers the statistical fingerprinting and wild-password classification detail this post intentionally leaves for the source:

• The 8,000-password test dataset built from 40 LLM models and 11 providers
• Model-by-model uniqueness and substring analysis across Anthropic, Qwen, Llama, Gemma, and GPT families
• Markov-chain classification thresholds and scoring logic used to identify likely LLM-generated passwords
• Examples of how AI-generated passwords appear in Terraform files, .env files, and source code

👉 GitGuardian's full post covers the Markov-chain methodology, provider-level patterns, and wild-scan details

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.