AI security maturity models are grading the wrong evidence

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Cycode's blog post: AI security maturity models are useful, but most fail because they score claims that teams cannot prove with artifacts.

Why it matters: IAM, NHI, and AI security teams should care because maturity scoring built on self-report cannot reliably govern shadow AI, agent activity, or the evidence needed to enforce policy.

Context

AI security maturity models are supposed to help organisations compare current practice against a defined target state, but they break down when the score is based on memory, opinion, or survey answers rather than proof. In AI programmes, that is especially risky because visibility into models, assistants, agents, and code generation is often incomplete from the start.

The identity and governance angle is direct: if teams cannot inventory what AI systems exist, they cannot credibly govern their access, permissions, or lifecycle. That makes the problem less about choosing the right maturity framework and more about whether the framework is scored from retrievable evidence or from assumptions.

Cycode’s article argues that the common failure is not the existence of maturity models themselves. It is the tendency to treat self-assessment as an adequate signal in a domain where shadow usage and fast-changing toolchains make self-report especially weak.

Key questions

Q: Why do AI maturity models fail when they rely on self-assessment?

A: Self-assessment fails because the first thing being measured is visibility, and AI environments are often only partially visible. Teams can claim governance, policy coverage, or inventory completeness without being able to prove it. Maturity scoring should therefore be treated as a validation exercise, not a trust exercise, especially when AI systems use secrets or non-human identities.

Q: Why does AI visibility need to come before maturity scoring?

A: Because visibility is the first condition being graded. If an organisation cannot identify its assistants, models, agents, and related tooling, it cannot reliably say whether policy, access, or monitoring is working. Discovery and inventory are therefore prerequisites to meaningful scoring, not optional preparation work.

Q: How should teams prove that an AI maturity claim is real?

A: Use an artifact-first standard. Every scoreable statement should map to something a named person can produce quickly, such as an inventory export, attribution log, or enforcement record. If the evidence does not exist, the level should remain blank rather than being filled from assumption.

Q: How should organisations govern AI agents that can modify repository code?

A: They should treat the agent as a delegated non-human identity with tightly bounded authority. Separate instruction input from execution rights, require human approval for write actions, and log every agent-initiated change with enough context to support review. If the agent can edit code without that separation, prompt injection becomes a privilege escalation path.

Technical breakdown

Why self-report fails as evidence in AI maturity scoring

AI maturity models often inherit a Capability Maturity Model style of scoring, where organisations claim a level and then map that claim to a rubric. That works only when the underlying environment is stable and visible. In AI programmes, the inventory changes quickly across assistants, models, MCP servers, and code-generation workflows, so a verbal claim is not evidence. Mature scoring needs an artifact that can be produced, reviewed, and challenged. Without that, the maturity level measures confidence, not control.

Practical implication: replace self-scored cells with evidence requirements that can be verified within one business day.

Why AI visibility is the first control being graded

The article’s central technical point is that visibility is both the first dimension being scored and the hardest one to trust. If teams do not know where AI is used, they cannot reliably score policy coverage, enforcement, or risk. In governance terms, this is an inventory problem before it is a policy problem. For identity teams, the same logic applies to non-human identities: you cannot govern access for what you have not discovered, named, and bound to ownership.

Practical implication: build discovery and inventory before maturity scoring, then use those outputs as the scoring baseline.

Why code review is a weak proxy for AI risk

The article also shows why review-based maturity signals are degrading. AI-generated code can look clean, read well, and pass human review while still carrying vulnerable dependencies or unsafe patterns. That means review coverage alone no longer proves secure development. The important technical shift is from trusting inspection at the end of the pipeline to proving control at generation, attribution, and enforcement points. This is a broader software assurance issue, but it intersects identity where agents act with delegated repository or build permissions.

Practical implication: move assurance left into generation-time controls and repository evidence, not just human review gates.

Threat narrative

Attacker objective: The practical objective is not just code compromise but governance blind spots that let risky AI usage persist while appearing controlled.

1. Entry occurs through unmanaged AI tooling, assistants, or agentic development paths that were never fully inventoried.
2. Credential or authority exposure follows when those tools operate with access to code, repositories, or connected services that teams have not clearly baselined.
3. Impact is a maturity programme that reports control coverage the organisation cannot substantiate, masking shadow AI and underestimating real exposure.

Breaches seen in the wild

• 12,000 secrets in LLM training data: Truffle Security found 11,908 live API keys and passwords hard-coded in web pages captured by Common Crawl, a dataset used to train LLMs.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI maturity has become an evidence problem, not a framework problem: the article is right to separate model quality from scoring quality. The stronger frameworks fail when teams fill them from memory instead of artifacts, because a rubric cannot compensate for missing proof. For governance, the real question is whether each claim can be tied to something retrievable, reviewable, and attributable. That is the standard practitioners should apply before trusting any maturity score.

Visibility debt is the named failure mode here: organisations are grading AI security before they have a trustworthy inventory of what AI is actually in use. That is not merely incomplete housekeeping, it is a governance condition that makes every later score fragile. The same pattern already appears in non-human identity programmes when service accounts, agents, and tool credentials are known only partially. Practitioners should treat discovery as the control that precedes maturity, not the output maturity is meant to summarise.

AI security maturity needs to shift from assertions to artifacts: the article’s strongest idea is that every level statement should map to a named evidence object. That is aligned with how serious assurance works across identity, cloud, and software supply chain controls. In practice, the organisation that can produce an AIBOM, an attribution baseline, and a blocked-policy log has something measurable; the organisation that cannot is still describing intent. Practitioners should make the artifact the unit of maturity, not the checkbox.

Agentic development widens the identity problem inside software delivery: once AI systems write code, choose dependencies, and open pull requests, they are operating with delegated authority that has to be governed like a non-human identity. The article correctly points out that many maturity models still centre deployed AI systems more than the software factory that produces them. That leaves a governance gap around agent permissions, repository scope, and build-path evidence. Practitioners should score the full agentic workflow, not just the model at the edge.

What this signals

Visibility debt is the practical reason AI maturity programmes drift away from reality. If the inventory is partial, every downstream claim about policy, enforcement, or control coverage is built on a shaky baseline, which means the score can look mature while the operating model is still blind.

For identity and governance teams, that shifts the work from assessing frameworks to proving existence. Agents, assistants, and AI-connected services should be inventoried with the same discipline used for service accounts and other non-human identities, because undiscovered systems cannot be governed with any credibility.

For practitioners

• Bind each maturity claim to a named artifact Require an evidence object for every scoreable statement, such as an inventory export, attribution log, or blocked-policy record. If no one can produce it within one business day, leave the control unscored.
• Discover shadow AI before scoring maturity Run automated discovery across repositories, models, assistants, and agentic tooling before any board or audit assessment. A maturity score built before inventory is just a confidence statement.
• Measure the AI software factory, not only deployed AI Track agent attribution, commit volume, policy violations, and repository-level exposure so the score reflects how AI is actually changing delivery. This is where identity, privilege, and development governance meet.
• Use enforcement logs as maturity evidence Keep records of what was blocked at the IDE, CLI, or repository boundary and tie those records to the control level being claimed. Enforcement history is more defensible than intent statements.
• Refresh the baseline on a 90-day cycle Separate annual reporting from continuous evidence refresh. The inventory and enforcement baseline should move with the environment, while the score simply summarises the latest proof.

Key takeaways

• AI security maturity programmes fail when they turn self-report into a score, because the first thing they are grading is visibility.
• The more defensible approach is to require a retrievable artifact for every maturity claim and leave the control blank when no proof exists.
• Discovery, baselining, and enforcement need to happen before scoring, otherwise the maturity model becomes a description of confidence rather than control.

Key terms

• AI Security Maturity Model: An AI Security Maturity Model is a structured way to assess how well an organization protects AI systems across their lifecycle. It typically measures governance, data protection, model access, monitoring, incident response, and control automation, helping teams move from ad hoc safeguards to repeatable, risk-based security practices.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Evidence-Based Scoring: A maturity assessment method that requires each control claim to be backed by a retrievable artifact such as an inventory export, log, or policy record. It reduces self-report bias and makes the score defensible to auditors, boards, and regulators.
• Agentic Development Lifecycle: The Agentic Development Lifecycle is the control pattern for building, approving, deploying, and reviewing AI agents before they reach production. It extends software change discipline into identity governance by requiring traceability for creation, access grants, business purpose, and ongoing oversight.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and agentic AI identity. It helps practitioners connect evidence-based control design to real identity and access decisions across modern programmes.