AI Security Maturity Model: Levels, Assessment, Roadmap

Most organisations now run AI in production, from copilots and chatbots to retrieval pipelines and autonomous agents, yet few can say how mature their AI security is or which gap to close first. AI adds new assets (models, datasets, prompts, vector stores, agent tools) and new attacks (prompt injection, data poisoning, model theft) on top of ordinary cloud and identity risk. This five-level model, using the same Ad hoc to Optimised scale as the NHI Mgmt Group NHI Governance Maturity Model, scores AI security across ten dimensions and maps them to the NIST AI RMF, OWASP, Google SAIF and MITRE ATLAS. It is for CISOs, AI platform and ML engineering leads, and AI governance teams who need an honest baseline and a roadmap.

Key takeaways

Score ten dimensions separately: inventory and governance, risk assessment, secure development (MLSecOps), data protection, model and supply chain security, runtime guardrails, agent identity and access, monitoring and response, red teaming, and compliance.
Inventory comes first. You cannot assess, test or govern models, agents and AI API keys you do not know exist, and shadow AI is where most early incidents start.
Level 3 (Defined) is the right first target: every AI system registered with an owner and risk tier, approved model sources, guardrails on every user-facing application and agents on their own least-privilege identities.
Guardrails are not a security boundary on their own. Prompt injection can be reduced but not eliminated, so the worst an AI system can do is set by the access it holds, not by its filters.
Maturity must show in numbers: AI systems in the registry, models from verified sources, agents on dedicated identities, red-team findings closed and time to disable a misbehaving AI system.

The five levels

Level	Name	Description
1	Ad hoc	Teams adopt AI services and open models on their own. No inventory, AI API keys sit in code and notebooks, prompts and outputs are not logged, and AI risk is handled through general acceptable use wording.
2	Developing	An AI policy and approval process exist, a partial list of sanctioned AI systems is kept, and some applications have content filters. Testing is a one-off before launch and agents often run on shared or human credentials.
3	Defined	Every AI system is registered with an owner and risk tier, model and dataset sources are approved, guardrails and output handling are standard, agents have dedicated least-privilege identities, AI activity is logged and AI incidents have a playbook.
4	Managed	Security is built into the ML pipeline: models are scanned and signed, AI-BOMs are produced, agent access is task-scoped and approved per action for high-impact tools, AI-specific detections run in the SOC and red teaming happens before each major release.
5	Optimised	AI security is continuous. Discovery feeds the registry automatically, adversarial tests run in the pipeline, guardrail and detection tuning uses incident data, containment is automated and compliance evidence is generated from the platform.

The ten dimensions

Levels 2 and 4 sit between the descriptions shown; the roadmap below lists what moves you from each level to the next.

Dimension	Level 1 Ad hoc	Level 3 Defined	Level 5 Optimised
AI inventory and governance	No list of models, AI services or agents	Registry of all sanctioned AI systems with owner, purpose, model, data sources, risk tier and approval status; AI policy approved	Continuous discovery of AI use across SaaS, cloud, code and endpoints feeds the registry; unregistered systems cannot reach production data
AI risk assessment	Risk considered informally, if at all	Risk and impact assessment before deployment, using a threat model covering prompt injection, poisoning, data leakage and excessive agency	Assessments re-run automatically when model, data, tools or autonomy change; residual AI risk reported against appetite
Secure AI development (MLSecOps)	Notebooks to production with no review	Secure development standard for AI features; code, prompts and pipeline configuration under version control and review; secrets kept out of notebooks	Adversarial and safety tests run in CI; pipeline changes signed and traceable; training and deployment environments separated
Data protection for AI	Sensitive data pasted into public tools; training data provenance unknown	Data classification applied to training, fine-tuning and retrieval sources; retrieval respects source permissions; data loss controls on AI tools	Dataset provenance recorded; sensitive data detected and redacted before training and in outputs; poisoning checks on ingested data
Model and supply chain security	Models and packages pulled from any public hub	Approved model sources; models scanned before use (for example for unsafe serialisation formats); AI libraries in software composition analysis	Models and datasets signed and verified at deployment; AI-BOMs for every system; third-party models and AI vendors reassessed continuously
Runtime protection and guardrails	Raw model output shown to users or passed to systems	Input and output filtering on every user-facing application; model output treated as untrusted; rate and cost limits	Context-aware guardrails tuned from incidents and red-team results; high-risk outputs blocked or routed to review automatically
Agent identity and access	Agents use developers' tokens, shared service accounts or static keys	Each agent has its own identity, owner and least-privilege access; short-lived credentials; human approval for irreversible actions	Task-scoped, just-in-time access decided per action by policy; delegation records both user and agent; credentials revoked automatically on anomaly
Monitoring and incident response for AI	No logs of prompts, outputs or tool calls	Prompts, outputs, tool calls and identities logged centrally with privacy controls; AI incident playbook with a way to disable a system	AI-specific detections in the SOC; automated containment; incidents feed back into guardrails, tests and risk assessments
Red teaming and testing	No testing beyond functional QA	Adversarial testing before launch of high-risk systems, covering prompt injection, jailbreaks, data leakage and tool misuse	Continuous automated adversarial testing plus periodic expert red teaming; findings tracked to closure and turned into regression tests
Compliance (EU AI Act, ISO/IEC 42001)	Obligations unknown	Systems classified against the EU AI Act risk categories; AI management system scoped against ISO/IEC 42001; records kept per system	Compliance evidence generated from the registry and pipeline; certification or independent assurance where required

How this model relates to other frameworks

The OWASP AI Maturity Assessment (AIMA) version 1.0, released on 11 August 2025, covers five domains (Strategy, Design, Implementation, Operations and Governance) and integrates principles from frameworks such as OWASP SAMM. It covers responsible AI use as well as security, so use it alongside this model. The NIST AI RMF 1.0 (January 2023) is voluntary and organised into four functions: Govern, Map, Measure and Manage. Google's Secure AI Framework (SAIF) publishes a map of 15 AI risks, and MITRE ATLAS is a public knowledge base of adversary techniques against AI systems. The table shows where each dimension lands.

Dimension	NIST AI RMF	OWASP (LLM 2025 / Agentic 2026)	Google SAIF risk	MITRE ATLAS
AI inventory and governance	Govern	Supports all entries	Supports all risks	Not technique-specific
AI risk assessment	Map	LLM06 Excessive Agency	Rogue Actions	Used for threat modelling
Secure AI development	Map, Manage	ASI05 Unexpected Code Execution	Model Source Tampering	AML.T0018 Manipulate AI Model
Data protection for AI	Measure, Manage	LLM02 Sensitive Information Disclosure, LLM04 Data and Model Poisoning, LLM08 Vector and Embedding Weaknesses	Data Poisoning, Sensitive Data Disclosure, Excessive Data Handling	AML.T0020 Poison Training Data
Model and supply chain security	Govern, Manage	LLM03 Supply Chain, ASI04 Agentic Supply Chain Vulnerabilities	Model Exfiltration, Model Deployment Tampering	AML.T0010 AI Supply Chain Compromise
Runtime protection and guardrails	Manage	LLM01 Prompt Injection, LLM05 Improper Output Handling, LLM10 Unbounded Consumption	Prompt Injection, Insecure Model Output, Denial of ML Service	AML.T0051 LLM Prompt Injection, AML.T0054 LLM Jailbreak
Agent identity and access	Govern, Manage	ASI03 Identity and Privilege Abuse, ASI02 Tool Misuse and Exploitation	Rogue Actions, Insecure Integrated Component	AML.T0053 AI Agent Tool Invocation
Monitoring and incident response	Manage	ASI10 Rogue Agents, ASI08 Cascading Failures	Rogue Actions	AML.T0024 Exfiltration via AI Inference API
Red teaming and testing	Measure	Tests every entry	Model Evasion, Model Reverse Engineering	Used to plan test cases
Compliance	Govern	Not covered	Not covered	Not covered

For compliance, EU AI Act Article 15 requires high-risk systems to be "resilient against attempts by unauthorised third parties to alter their use, outputs" and names data poisoning, model poisoning, adversarial examples or model evasion, and confidentiality attacks as threats to address. ISO/IEC 42001:2023 specifies requirements for an AI management system for organisations "developing, providing, or using AI-based products or services". See the Agentic AI Compliance Guide.

Where identity fits

AI systems are non-human identities with unusual reach. A model gateway holds provider API keys, a retrieval pipeline holds read access to document stores, and an agent holds tokens for every tool it calls. When prompt injection succeeds, the attacker inherits that access, which is why OWASP lists ASI03 Identity and Privilege Abuse among its agentic risks. In the PocketOS database deletion, an AI coding agent used an over-privileged platform token to wipe production. Stolen AI API keys are also resold for free model access, as the LLMjacking Guide explains. Score this dimension with the Agentic AI Identity Maturity Model for agent-heavy estates, and see the AI Agent Authorisation Guide.

Self-assessment questions

Score each dimension from 1 to 5 using evidence, not intent. Your overall level is the lowest score among the dimensions that protect your highest-risk AI systems, not the average. Record the evidence behind each score and baseline the six metrics below at the same time.

Inventory: could you list every model, AI service, agent and AI API key in use today, including those added through SaaS features and browser extensions?
Risk assessment: which deployed AI systems went live without a documented threat model, and who accepted that risk?
Development: are system prompts, tool definitions and pipeline configuration version-controlled and reviewed like code?
Data: can your retrieval system return a document to a user who could not open it in the source system?
Supply chain: where did your last downloaded model come from, was it scanned before loading, and would you know if it changed?
Guardrails: is model output ever passed to a shell, browser, database query or email without validation?
Agent access: if an agent were fully hijacked by prompt injection, what is the worst it could do with its current credentials?
Monitoring: could you reconstruct which prompt led to which tool call and under which identity, and how long would it take to switch the agent off?
Red teaming: when was each high-risk AI system last tested adversarially, and are its findings closed?
Compliance: which of your AI systems fall into an EU AI Act risk category, and who owns that classification?

Moving up a level

From Ad hoc to Developing

Discover AI use across SaaS, cloud accounts, code repositories and expense data, and start a registry. See the Shadow AI Discovery Guide.
Publish an AI acceptable use policy and an approval route for new AI systems. See the AI Governance Guide.
Move AI API keys out of code and notebooks into a secrets manager and turn on secret scanning.
Add basic input and output filtering to user-facing applications.

From Developing to Defined

Require a threat model and risk tier before deployment, using the OWASP LLM and Agentic Top 10 as the minimum checklist. See the LLM Security Guide.
Approve model sources, scan models before loading and add AI libraries to software composition analysis. See the AI Supply Chain and AI-BOM Guide.
Standardise guardrails and output handling, starting in monitor-only mode to measure false positives, then enforce by application, keeping the previous configuration as the rollback. See the AI Guardrails and AI Firewalls Guide.
Give every agent its own identity and remove human and shared credentials. See the Agentic AI Identity Guide.
Log prompts, outputs and tool calls centrally, and write an AI incident playbook that includes how to disable a system and revoke its credentials.

From Defined to Managed

Produce an AI-BOM per system (CycloneDX ML-BOM covers models, datasets and their dependencies) and verify model signatures at deployment.
Make agent access task-scoped and just in time, with per-action policy and approval for high-impact tools.
Build AI-specific detections: unusual tool sequences, spikes in token use, access to data outside an agent's normal scope. See the AI Agent Observability and Incident Response Guide.
Red team high-risk systems before each major release. See Red Teaming AI Agents for Identity Abuse.

From Managed to Optimised

Run adversarial test suites in CI so a model, prompt or tool change cannot ship if it regresses.
Automate containment for high-confidence detections, with a reviewed release process.
Generate EU AI Act and ISO/IEC 42001 evidence from the registry and pipeline rather than by hand.

Metrics that show maturity

Registry coverage: AI systems and agents discovered that are registered with an owner and risk tier (target 100% for production).
Verified models: percentage of deployed models from approved sources with a scan or signature check.
Dedicated agent identities: percentage of agents not using human, shared or static credentials.
Guardrail coverage: user-facing AI applications with enforced input and output controls.
Red-team closure: high-severity adversarial findings closed within their target date.
Time to disable: minutes from decision to a misbehaving AI system being stopped and its credentials revoked, measured in exercises.

Common failure patterns

Filters as the only control: relying on prompt filters while the application holds broad data access. The EchoLeak zero-click prompt injection showed how an assistant's own access becomes the exfiltration path. See the Prompt Injection Guide.
Trusting AI packages and models by popularity: the Ultralytics PyPI compromise put a cryptominer in a widely used AI library through its build pipeline.
Exposed AI infrastructure: training and serving clusters treated as research kit rather than production. In ShadowRay, exposed Ray clusters gave attackers cloud credentials and AI API tokens.
One-off testing: a launch red team that is never repeated after model, prompt or tool changes.

Practitioner checklist

Set Level 3 as the 12-month target, with Level 4 for agents and systems that handle sensitive data or take irreversible actions.
Build the AI registry before buying AI security tooling. See the AI Security Platform Buyer's Guide.
Treat every model output as untrusted input to the next system.
Load models only from approved sources, after a scan, and record them in an AI-BOM.
Give each agent its own least-privilege identity and test the kill switch.
Reassess every six months and whenever you adopt a new model provider, agent framework or protocol.

Standards and references

Related NHI Mgmt Group resources: AI Governance Guide · LLM Security Guide · AI Guardrails and AI Firewalls Guide · Agentic AI Identity Maturity Model

AI Security Maturity Model