Agent Controls & Monitoring Maturity Model: 5 Levels

Most organisations can say how many AI agents they run, but few can prove that each agent's actions are bounded, recorded, watched and stoppable, and that gap is what turns a manipulated agent into an incident. This model scores agent controls and monitoring across seven dimensions on the NHI Mgmt Group five-level scale, from Ad hoc to Optimised, with observable indicators, the evidence an assessor should ask for and a scoring method. It assumes the concepts in the Agent Controls and Monitoring Guide and does not re-explain them; use that guide for the control order and this page to measure where you are. It is written for AI platform owners, security architects, SOC leads and the risk and audit teams who need a defensible maturity rating for agent deployments.

Key takeaways

Score each dimension separately. An organisation with excellent logging and no way to stop an agent is not "Level 3 overall"; report the lowest critical dimension alongside the average.
Attribution caps everything. If agents act under shared service accounts or borrowed human credentials, logging, detection and containment cannot rise above Level 2, because no record can say which agent did what.
Level 3 is the minimum for production autonomy. At Defined, every tool call is policy-checked and logged with agent and principal identity, irreversible actions need bound approval, and containment is documented.
Evidence beats intent. Each level is proven by artefacts: inventory exports, policy files, sampled logs, drill results. A dimension with no evidence scores Level 1, whatever the design documents say.
Kill switches are measured, not declared. Higher levels require drilled containment with a measured time to stop, as EU AI Act Article 14 describes for high-risk systems: a "stop" button or similar procedure that halts the system in a safe state.

The five levels

Level	Name	Description
1	Ad hoc	Agents are deployed by individual teams with no central record. They run under developers' tokens or shared service accounts with broad scopes. Tool access is whatever the framework exposes. Logs, if any, are application debug output. Nobody owns stopping an agent.
2	Developing	A partial inventory exists and new production agents get their own identity. Tool lists are written down but not enforced at a control point. Prompts and responses are logged in an observability tool, not in the SIEM. Some high-impact actions have human approval in the user interface. Stopping an agent means revoking a credential by hand.
3	Defined	Every production agent is registered with owner, purpose, identity and permitted tools. Tool and MCP calls pass through an enforcement point that applies allowlists and argument policy and logs every call with agent ID and acting principal. Irreversible actions require approval bound to the exact action. Agent telemetry reaches the SOC. Containment levels and owners are documented.
4	Managed	Behavioural baselines exist per agent and per model and prompt version, and deviations raise detections. Kill switch and containment drills run on a schedule with a measured time to contain. Security tests run in the release pipeline on every model, prompt or tool change. Control metrics are reported to governance.
5	Optimised	Autonomy is risk-adaptive: agents are promoted to wider permissions on measured behaviour and demoted automatically on anomaly. Red teaming is continuous, findings feed policy updates, and detections are tuned from incident and drill results.

The seven dimensions

Each dimension below lists what distinguishes the levels. The detailed how-to for each sits in the linked guides.

Dimension	Level 1 to 2	Level 3	Level 4 to 5
1. Inventory and attribution	Spreadsheet or none; shared credentials	Registry with owner, purpose, identity, tools, data; one identity per production agent	Registry reconciled automatically with runtime and identity provider; unregistered agents detected and blocked
2. Action boundaries	Framework defaults; tool list in documentation	Allowlist and argument policy enforced at a gateway; read-only by default; budgets and rate limits	Policy as code with tests; scopes narrowed from observed use; just-in-time elevation for rare tools
3. Human oversight	Confirmation dialogs; rubber-stamp risk	Approval bound to the exact action and arguments for irreversible operations; approver identity logged	Approval volume and override rates measured; tiers re-cut to fight approval fatigue and automation bias
4. Logging and telemetry	Debug logs, prompts only, short retention	Structured record per tool call: agent ID, principal, tool, arguments hash, decision, result; in the SIEM; retention set	Traces joined end to end across agents, tools and target systems; integrity protection on logs
5. Detection	None specific to agents	Rules for new tools, new destinations, denied-call spikes, budget breaches	Baselines per agent and version; sequence and goal-drift detections; tuned from drills
6. Containment and kill switch	Manual credential revocation, no owner	Documented containment levels (throttle, approval-only, read-only, stop) with owners and runbooks	Drilled with measured time to contain; automatic demotion on anomaly; safe-state recovery tested
7. Assurance and testing	Ad hoc testing before launch	Pre-deployment security checklist and test cases for prompt injection and tool misuse	Tests in CI on every model, prompt and tool change; continuous red teaming; findings tracked to closure

Inventory and attribution

Attribution is the dependency for every other dimension. At Level 3, a registry entry exists before an agent gets credentials, and each production agent has its own workload or service identity rather than a developer's token. See the AI agent registry and the Agentic AI Identity Guide.

Action boundaries and human oversight

Boundaries decide what an agent can attempt; oversight decides what it can complete without a person. MITRE ATLAS mitigation AML.M0029 Human In-the-Loop for AI Agent Actions describes requiring a user or other human stakeholder to approve agent actions before they are taken. At Level 3 the approval must be bound to the exact action and arguments, so an agent cannot obtain approval for one operation and execute another. See Human-in-the-Loop for AI Agents Guide and Agent Sandboxing vs Tool Allowlisting.

Logging, telemetry and detection

The OpenTelemetry generative AI semantic conventions (status: Development, checked November 2026) define agent spans such as create_agent and invoke_agent, plus execute_tool for tool execution, with attributes including gen_ai.agent.id and gen_ai.agent.name. Adopting a shared schema early makes Level 4 trace joining far easier, but treat attribute names as subject to change while the conventions are in development. For high-risk AI systems, EU AI Act Article 12 requires automatic recording of events over the system's lifetime, with traceability that supports identifying risk situations, post-market monitoring and deployer monitoring. See the AI Agent Audit Log Schema Template and the AI Agent Behaviour Baselining and Anomaly Detection Guide.

Containment and assurance

Containment should have levels short of a full stop, because stopping a business-critical agent has its own cost. Level 4 requires that each level has been exercised and timed. See the AI Agent Kill Switch and Containment Guide. Testing should cover the agentic risks described in the OWASP Top 10 for Agentic Applications for 2026 (published December 2025), including rogue agent behaviour covered in the OWASP Agentic Top 10 ASI10 Guide.

Evidence an assessor should request

Dimension	Evidence for Level 3	Evidence for Level 4
Inventory and attribution	Registry export; sample of 10 agents traced to their own identities in the identity provider	Reconciliation report showing unregistered agents found and actioned
Action boundaries	Gateway policy files for sampled agents; test showing a non-allowlisted tool call is denied	Policy test suite results in CI; scope reduction history
Human oversight	Approval records showing action, arguments, approver and outcome	Approval volume, approval time and rejection rate trends
Logging and telemetry	SIEM query returning tool calls by agent ID and principal for a given day	An end-to-end trace from user request through agent, tools and target system
Detection	Rule list and a fired test alert	Baseline definitions per version; detection tuning records
Containment	Runbook with containment levels and named owners	Drill reports with measured time to contain
Assurance	Pre-deployment checklist signed for sampled agents	CI test logs for the last model or prompt change; red team findings tracker

Self-assessment questions

Answer each question for your highest-risk production agents, not your best-run ones. A "no" to a Level 3 question caps that dimension at Level 2.

Can you list every production agent, its owner and the identity it acts under, within one hour? (Inventory, Level 3)
Does any production agent use a developer's personal token or a credential shared with another workload? (Attribution: "yes" caps at Level 2)
Is every tool or MCP call checked against a policy at a point the agent cannot bypass? (Action boundaries, Level 3)
For delete, pay, send-externally and privilege-change actions, does the approval record the exact arguments approved? (Oversight, Level 3)
Can the SOC answer "what did agent X do between 10:00 and 11:00 yesterday, and on whose behalf?" from the SIEM alone? (Logging, Level 3)
Do you have at least one detection that would fire if an agent called a tool it has never used before? (Detection, Level 3)
Who can stop each agent, how, and when was that last tested and timed? (Containment, Level 3 to 4)
Does a model, prompt or tool change trigger security tests before release? (Assurance, Level 4)
Are baselines versioned with the model and prompt, so a new version does not inherit the old baseline? (Detection, Level 4)
Do agents lose permissions automatically when their behaviour deviates? (Containment, Level 5)

Scoring method

Per dimension: the highest level for which all indicators are met and evidenced.
Critical floor: report the minimum of Inventory and attribution, Logging and telemetry, and Containment as the headline, because a weakness in any of these defeats the others.
Average: report the mean of all seven as a secondary figure for trend tracking.
Scope: assess separately for internally built agents, agents embedded in SaaS products and coding assistants, since evidence and control points differ.

Where identity fits

Agent identity is the join key for every dimension: policy is written per identity, logs are attributed per identity, and containment usually means suspending or narrowing an identity. Agents that act on behalf of users also need the user recorded as the acting principal, so delegation can be audited. The identity side is scored in the Agentic AI Identity Maturity Model; use it alongside this model rather than scoring identity twice.

Moving up: roadmap

Level 1 to 2: start a registry, give new production agents their own identities, ship prompt and tool logs to a central store and name an owner for stopping each agent.
Level 2 to 3: put a gateway in the tool path in log-only mode, convert observed use to allowlists agent by agent, add bound approvals for irreversible actions, forward structured tool-call records to the SIEM and write containment runbooks. Keep a rollback flag that returns the gateway to log-only.
Level 3 to 4: build baselines per agent version, add security tests to the release pipeline, run quarterly containment drills and report metrics to the AI governance forum.
Level 4 to 5: automate promotion and demotion of autonomy from measured behaviour, run continuous red teaming and feed results into policy and detection updates.

Metrics that show maturity

Attributed agents: percentage of production agents with their own identity and registry entry.
Policy coverage: percentage of tool calls passing through an enforcement point.
Bound approvals: percentage of irreversible actions with an approval record matching executed arguments.
Time to contain: median minutes from decision to stop in the last drill.
Change test coverage: percentage of model, prompt and tool changes that ran security tests before release.

Framework mapping

Dimension	NIST SP 800-53 Rev 5	NIST CSF 2.0	Other
Inventory and attribution	AC-2 Account Management	ID.AM-02	OWASP Agentic Top 10 (2026)
Action boundaries	AC-6 Least Privilege	PR.AA-05	OWASP Agentic Top 10 (2026)
Human oversight	AC-5 Separation of Duties	PR.AA-05	MITRE ATLAS AML.M0029; EU AI Act Art. 14
Logging and telemetry	AU-2 Event Logging; AU-12 Audit Record Generation	PR.PS-04	EU AI Act Art. 12; OpenTelemetry GenAI conventions
Detection	SI-4 System Monitoring	DE.CM-09
Containment	IR-4 Incident Handling	RS.MI-01	EU AI Act Art. 14(4)
Assurance	CA-7 Continuous Monitoring	GV.OV-01	OWASP Agentic Top 10 (2026)

Common failure patterns

Rating the design, not the runtime: a gateway exists but agents call tools directly through SDK credentials that bypass it.
Approval fatigue: approvers see dozens of prompts a day and approve without reading, the automation bias risk Article 14 asks overseers to stay aware of.
Prompts logged, actions not: observability captures conversation text but not the tool call, arguments and result, so investigators cannot reconstruct impact.
Kill switch never pulled: containment is a credential revocation nobody has tried, and it takes hours because the credential is shared.
Embedded agents left unscored: SaaS-embedded agents and coding assistants are excluded because the organisation does not run them, though they act on its data.

Practitioner checklist

Assess the seven dimensions separately and report the critical floor.
Collect evidence for each rating; no evidence means Level 1.
Remove shared and personal credentials from production agents first.
Route every tool and MCP call through an enforcement point that logs agent and principal.
Bind approvals to exact actions for irreversible operations.
Send structured tool-call records to the SIEM with agent-specific detections.
Document containment levels and drill them with a measured time to contain.
Run security tests on every model, prompt and tool change.
Reassess quarterly and after any agent incident.

Agent Controls & Monitoring Maturity Model: Scoring Guardrails, Logging and Kill Switches