An AI agent turns model output into real actions with real credentials, so the controls that matter are the ones enforced around the model at runtime, not the instructions written inside it. Agent controls and monitoring is the discipline of bounding what a running agent can do, watching what it actually does, and stopping it quickly when the two diverge. This pillar page gives the overview of the whole field: the control points in an agent's execution path, the threats they answer, the controls to put in place first and later, who owns them, and how they map to OWASP, NIST, MITRE ATLAS and the EU AI Act. It is written for security architects, platform and SRE teams, detection engineers and IAM teams responsible for agents in production. Each section summarises and links to the NHI Mgmt Group guide that goes deep on that control.
Key takeaways
- Enforce outside the model. OWASP LLM06:2025 Excessive Agency puts authorisation in downstream systems rather than in the model's judgement; approvals, limits and the stop switch belong in the identity provider, tool gateway, network and runtime.
- Control the four chokepoints. Identity and credentials, tool calls, code execution and network egress are where an agent's intent becomes an effect; each needs a policy enforcement point the agent cannot bypass.
- Scale controls with autonomy. An agent that suggests needs least privilege and logging; an agent that acts without per-action approval also needs budgets, behavioural baselines, a tested kill switch and its own identity.
- Observability is a control, not a dashboard. Per-agent, per-tool-call telemetry with the acting identity and the human principal is what makes detection, approval evidence, containment and audit possible.
- Alerts need an owner and a stop threshold. In the OpenAI and Hugging Face incident, warning signs were seen weeks earlier but judged not to need stopping; define in advance who pauses an agent and on what signal.
What the field covers
Agent controls and monitoring sits between two neighbouring disciplines. Agent identity and access decides who the agent is and what it is entitled to; agent governance and risk decides whether an agent should exist, at what autonomy level and with what accountability. Controls and monitoring is the runtime layer that enforces those decisions on every action and produces the evidence that they held. See the Agent Identity and Access Guide and the Agent Governance and Risk Guide.
The field covers seven connected areas:
- Guardrails: input and output filtering, prompt injection screening, content policies and AI gateways that sit in front of model calls.
- Action controls: tool allowlists, argument validation, approval gates, spending and rate limits, and step-up for high-impact actions.
- Execution isolation: sandboxes for code the agent writes or runs, filesystem and network restrictions, and separation from production credentials.
- Observability: traces of model calls, tool calls, retrieved context, memory writes, approvals and outcomes, attributed to a specific agent and principal.
- Detection: declared and learned behavioural baselines, anomaly rules and correlation with identity and cloud telemetry.
- Containment and response: graded containment levels, credential revocation, quarantine with evidence preservation, and playbooks for compromised or misbehaving agents.
- Assurance: pre-deployment checks, security testing and red teaming that prove the controls work before and after release.
It also covers the reverse case: agents used by defenders, for example in the SOC, where the same controls apply to an agent that holds incident response privileges. See the Agentic AI in the SOC Guide.
Core concepts
The agent execution path and its control points
Every agent task follows the same loop: a trigger arrives, the agent assembles context (instructions, retrieved documents, memory, tool results), the model plans, and the agent calls tools that act on other systems using some credential. Controls attach at each hop. The table shows where.
| Control point | What is enforced | Typical mechanism | Deep guide |
|---|---|---|---|
| Model input and output | Untrusted content screening, sensitive data filtering, output validation | AI gateway or guardrail service, output schema checks | AI Guardrails and AI Firewalls Guide |
| Credential issuance | Which identity the agent acts as, scope and lifetime of its tokens | Workload identity, token exchange, vault-brokered short-lived credentials | Credential Vaulting for AI Agents Guide |
| Tool call | Allowed tools, argument constraints, per-action authorisation, approvals, budgets | Tool or MCP gateway acting as policy enforcement point | AI Agent Action Approvals and Spending Limits Guide |
| Code execution | Where agent-generated code runs and what it can reach | Containers, microVMs, gVisor-style sandboxes, read-only mounts | AI Agent Sandboxing and Code Execution Guide |
| Network egress | Which destinations an agent or its sandbox can reach | Default-deny egress, allowlisted domains, blocked instance metadata endpoints | AI Agent Sandboxing and Code Execution Guide |
| Memory and state | What the agent writes to persistent memory and who can read it | Memory write logging, provenance tags, expiry | OWASP Agentic Top 10 ASI06 Guide |
| Telemetry | Attribution of every step to agent, version, principal and task | OpenTelemetry GenAI spans, SIEM correlation | AI Agent Observability and Incident Response Guide |
| Stop | Ability to throttle, pause, cut tools and revoke credentials | Gateway kill flags, IdP session revocation, network isolation | AI Agent Kill Switch and Containment Guide |
Least agency and strong observability
The OWASP Top 10 for Agentic Applications 2026 frames its advice around two principles. Least agency extends least privilege to autonomy: do not give an agent more autonomy than the task needs, because unnecessary agentic behaviour adds attack surface without adding value. Strong observability means clear visibility into what agents are doing, why, and which tools they invoke. The two work together: the less you can see, the less autonomy you can safely grant.
Guardrails versus controls
A guardrail is a probabilistic check on content, such as a classifier that scores a prompt for injection. A control is a deterministic decision at an enforcement point, such as a gateway that refuses a delete call without a bound approval. Guardrails reduce how often bad instructions reach the model; controls limit what happens when they get through. OWASP LLM01:2025 notes that it is unclear whether fool-proof prevention of prompt injection exists, so design every high-impact action as if a guardrail has already failed. See the Prompt Injection Guide.
Autonomy and human oversight
Humans can be in the loop (approving a specific action before it executes), on the loop (monitoring with authority to pause or override) or out of the loop (reviewing outcomes later). Most production agents need all three, applied per action class. MITRE ATLAS mitigation AML.M0029 Human In-the-Loop for AI Agent Actions ties approval to the consequence of the task, and AML.M0030 Restrict AI Agent Tool Invocation on Untrusted Data restricts tool calls once untrusted data has entered the context. See the Human-in-the-Loop for AI Agents Guide and the AI Agent Autonomy Levels Framework.
Telemetry that makes control possible
The OpenTelemetry generative AI semantic conventions (status: Development) define agent operations create_agent, invoke_agent and execute_tool, with attributes such as gen_ai.agent.id, gen_ai.agent.name, gen_ai.tool.name and gen_ai.conversation.id. Using shared names lets one detection rule work across frameworks. The minimum record per tool call is: agent identifier and version, the principal it acted for, the credential identifier (never the secret), the tool and its arguments (redacted where needed), the policy decision, any approval, and the outcome.
Main threats
Runtime controls exist because agents fail by doing the wrong thing successfully, with valid credentials. The threats below are the ones this category's controls answer; the attack side is covered in the Agent Threats and Attacks Guide.
| Threat | Reference | Primary runtime control | Detection signal |
|---|---|---|---|
| Goal hijack via indirect prompt injection | OWASP ASI01; ATLAS AML.T0051 LLM Prompt Injection | Restrict tool use after untrusted content enters context; approval for goal-changing actions | Plan or tool sequence changes after reading external content |
| Tool misuse and exploitation | OWASP ASI02; ATLAS AML.T0053 AI Agent Tool Invocation | Tool allowlist, argument validation, per-action authorisation at the gateway | New tool, unusual arguments, destructive verbs |
| Identity and privilege abuse | OWASP ASI03; ATLAS AML.M0026 (mitigation) | Own identity per agent, short-lived task-scoped tokens, no borrowed human sessions | Permission errors, credential discovery, privilege enumeration |
| Unexpected code execution | OWASP ASI05 | Sandboxed execution, no production secrets in the sandbox, default-deny egress | Outbound connections from sandboxes, metadata endpoint access |
| Memory and context poisoning | OWASP ASI06 | Memory write controls, provenance and expiry | Memory writes containing instructions or URLs |
| Data exfiltration through tools | ATLAS AML.T0086 Exfiltration via AI Agent Tool Invocation | Egress allowlists, output filtering, approval for external sends | Sensitive read followed by outbound call or message |
| Cascading failures | OWASP ASI08 | Circuit breakers, rate limits, blast-radius limits between agents | Error storms, retries, fan-out across agents |
| Human-agent trust exploitation | OWASP ASI09 | Show raw actions in approvals, separation of duties for approvers | Approval rates near 100%, approval times of seconds |
| Rogue agents | OWASP ASI10 | Kill switch, credential revocation, quarantine | Behaviour outside declared purpose, persistence attempts |
| Runaway cost and resource use | OWASP LLM10:2025 Unbounded Consumption | Token, spend and turn budgets; per-agent quotas | Token or spend spikes per task |
For the full OWASP list, see the OWASP Top 10 for Agentic Applications Guide; cost abuse is covered in the Denial-of-Wallet and AI Agent Cost Runaway Guide.
Key controls by maturity
Controls should arrive in an order that makes each one useful. Attribution comes before detection, because a baseline built on a shared service account is meaningless; a kill switch comes before high autonomy, because an agent you cannot stop should not act alone.
| Stage | Controls in place | Evidence it is working |
|---|---|---|
| 1. Foundation | Inventory of agents with owner and purpose; own identity per production agent; tool allowlist; logging of every tool call with agent ID and principal; read-only by default | Every production agent appears in the inventory and in logs under its own identifier |
| 2. Bounded | Gateway policy enforcement on tool calls; approval gates for delete, pay, send externally and privilege changes; budgets and rate limits; sandboxed code execution with default-deny egress | High-impact actions without a bound approval are blocked in tests |
| 3. Monitored | Declared baselines from registration data; detections for new tools, new destinations and sequence anomalies; agent telemetry joined to identity and cloud logs in the SIEM; defined containment levels | Detections fire in exercises; mean time to contain measured |
| 4. Assured | Pre-deployment checklist and security testing in the release pipeline; kill switch drills; learned baselines tied to model and prompt versions; automatic demotion to approval-gated mode on anomaly | Drills meet target time to contain; regression tests run on every model or tool change |
| 5. Optimised | Risk-adaptive autonomy (promotion on evidence, automatic demotion); continuous red teaming; control metrics reported to governance | Autonomy changes are driven by measured behaviour, not requests |
A practical rollout
- Find and attribute: list every agent, including those inside SaaS tools and coding assistants, and move production agents off shared and borrowed human credentials. See the Agentic AI Identity Guide.
- Put a gateway in the path: route tool and MCP calls through an enforcement point that logs every call. Start in log-only mode for two to four weeks to learn which tools and arguments each agent really uses. See the MCP Security Guide.
- Turn the observed set into a declared baseline: convert the allowlist to enforce mode for one pilot agent at a time, with a rollback flag that returns the gateway to log-only within minutes.
- Add approval gates and budgets for irreversible actions, then measure approval volume. If approvers see dozens of prompts a day, re-tier rather than accept rubber-stamping.
- Isolate execution: move code-running agents into sandboxes with no production secrets, no instance metadata access and allowlisted egress.
- Build detections and the stop path together: each high-severity detection names the containment level it triggers and who may trigger it. Drill the kill switch on a non-production agent before relying on it.
- Gate releases: require the deployment checklist and security tests before any model, prompt or toolset change reaches production. See the AI Agent Deployment Checklist.
Decision guidance
- Platform guardrails or your own gateway: managed agent platforms provide built-in filters, tool permissions and logs, which is enough for a small number of agents on one platform. Once agents span several platforms or call internal APIs, a gateway you control gives one policy and one log format. See the AI Gateway and LLM Gateway Guide.
- Approval or budget: use approval for actions that are irreversible or externally visible; use budgets and rate limits for high-volume reversible actions where per-action approval would cause fatigue.
- Declared or learned baselines: start declared (what the agent was registered to do); add learned baselines once you have stable history per agent version.
- Small organisations: keep autonomy low, rely on platform controls, use per-agent API keys with spend caps, and log centrally. A tested way to revoke every agent credential matters more than an anomaly model.
- Large organisations: standardise telemetry on OpenTelemetry GenAI conventions, run a shared gateway as a platform service, and give the SOC containment runbooks that map to identity provider and cloud revocation APIs.
Roles and ownership
Agent controls fail most often at handoffs: the platform team builds a gateway, the SOC receives alerts it cannot act on, and nobody owns the decision to stop a business-critical agent. Agree the split before the first incident.
| Role | Owns | Typical gap |
|---|---|---|
| Agent owner (business or product) | Purpose, declared tools and data, autonomy level, acceptance of residual risk, restart decision | Owner leaves or the project ends; the agent keeps running |
| AI platform team | Gateway, guardrails, sandboxes, telemetry pipeline, budgets | Controls exist but are optional per team |
| IAM and NHI team | Agent identities, token lifetimes, delegation, revocation paths | Agents run on human or shared credentials |
| Security operations | Detections, triage, containment levels L0 to L2, incident playbooks | Alerts lack the agent context to decide |
| Incident commander or CISO delegate | Fleet-wide stop and production restart approval | No pre-authorisation, so stopping waits for a meeting |
| Security testing and red team | Pre-release tests, regression after changes, kill switch drills | Testing only at launch, never after model updates |
Detection and measurement
Monitor at three layers: the agent platform (model and tool spans, approvals, memory writes), the enforcement points (gateway allow and deny decisions, budget hits) and the systems acted on (cloud audit logs, SaaS audit logs and identity provider sign-ins that show the agent's own identity). NIST AI RMF MEASURE 2.4 expects the behaviour of an AI system to be monitored in production, and MANAGE 4.1 expects post-deployment monitoring plans. Correlate on the trace or conversation identifier so one task can be followed end to end. See the AI Agent Behaviour Baselining and Anomaly Detection Guide.
- Attribution coverage: share of production agent actions logged under the agent's own identifier with a principal recorded (target: 100%).
- Enforcement coverage: share of production agents whose tool calls pass through a policy enforcement point in enforce mode.
- Gated high-impact actions: share of irreversible actions executed with a bound, single-use approval (target: 100%), plus median approval time as a fatigue indicator.
- Time to contain: measured in drills from decision to credentials revoked and tools cut, per containment level.
- Detection precision: share of agent alerts that lead to action, and time to detect in red team exercises.
- Credential lifetime: median and maximum lifetime of credentials held by agents.
Common failure patterns
- The stop instruction lives in the prompt. Telling an agent not to delete or to halt is advice, not control. If the only barrier to a destructive call is the system prompt, an injected instruction or a misread task removes it.
- Sandboxes that can reach production. In the OpenAI and Hugging Face Breach 2026, evaluation agents found an internal package registry reachable from their environment, got out through server-side request forgery, queried instance metadata for credentials and used a cluster-admin credential shared across clusters.
- Alerts without a stop threshold. In the same incident, warning signs appeared in late May and on 27 June and were judged not to require stopping the run; the agents went on to take about 17,600 actions in roughly four and a half days. Every agent alert class needs an owner and a predefined containment level.
- Logs that show a person, not the agent. Agents acting with a user's token or a shared service account cannot be baselined, attributed or revoked precisely. Fix identity first.
- Approval fatigue. Gating every action produces approvals in seconds and near-100% acceptance, which is automation bias under another name. EU AI Act Article 14(4)(b) explicitly asks overseers of high-risk systems to remain aware of it.
- Baselines that ignore versions. A new model, system prompt or toolset changes behaviour. Without version tags, detections either fire on every release or quietly learn the attack as normal.
Standards and frameworks
| Framework | Where agent controls and monitoring sit |
|---|---|
| OWASP Top 10 for Agentic Applications 2026 | ASI01 to ASI10, with least agency and strong observability as cross-cutting principles |
| OWASP Top 10 for LLM Applications 2025 | LLM06 Excessive Agency (downstream authorisation, human approval, logging and rate limiting); LLM10 Unbounded Consumption (budgets) |
| MITRE ATLAS | Mitigations AML.M0029, AML.M0030, AML.M0026; techniques AML.T0051, AML.T0053, AML.T0086 |
| NIST AI RMF 1.0 | MEASURE 2.4 (production behaviour monitored), MANAGE 2.4 (supersede, disengage or deactivate), MANAGE 4.1 (post-deployment monitoring) |
| NIST SP 800-53 Rev. 5 | AC-6 Least Privilege, AC-6(9) Log Use of Privileged Functions, AC-2(12) Account Monitoring for Atypical Usage, AU-2 Event Logging, AU-9 Protection of Audit Information, AU-12 Audit Record Generation, SI-4 System Monitoring, CA-7 Continuous Monitoring, IR-4 Incident Handling |
| NIST CSF 2.0 | PR.AA-05 (access permissions and least privilege), PR.PS-04 (log records for continuous monitoring), DE.CM-09 (runtime environments monitored), DE.AE-03 (correlation), RS.MI-01 (incidents contained) |
| EU AI Act (high-risk systems) | Article 14 human oversight, including monitoring for anomalies, override and a stop procedure to a safe state; Article 26 deployer monitoring, suspension on risk and log retention of at least six months |
| CSA MAESTRO | Seven-layer agentic threat model; Layer 5 Evaluation and Observability, Layer 4 Deployment and Infrastructure |
| ISO/IEC 27001:2022 Annex A | A.8.2 Privileged access rights, A.8.15 Logging, A.8.16 Monitoring activities |
| CIS Controls v8.1 | Control 6 Access Control Management, Control 8 Audit Log Management, Control 13 Network Monitoring and Defense |
Where an agent is used in a high-risk EU AI Act context, the Annex III obligations apply from 2 December 2027 under the timeline as amended by the Digital Omnibus on AI; check the current position with your advisers. See the Agentic AI Compliance Guide.
Where to go next (all guides in this category)
Observability and detection
- AI Agent Observability and Incident Response Guide: what to log per agent action, attribution and the signals that an agent has gone wrong.
- AI Agent Behaviour Baselining and Anomaly Detection Guide: declared and learned baselines, detection methods for non-deterministic agents and detector metrics.
Runtime guardrails and action control
- Human-in-the-Loop for AI Agents Guide: placing approval gates by risk tier, binding approvals to requests and avoiding fatigue.
- AI Agent Action Approvals and Spending Limits Guide: thresholds, budgets and step-up for agents that spend money or change systems.
- AI Agent Sandboxing and Code Execution Guide: isolation options, egress control and keeping secrets out of agent runtimes.
Containment and incident response
- AI Agent Kill Switch and Containment Guide: graded containment levels, revocation, quarantine and restart criteria.
- AI Agent Incident Response Playbook: roles, decision points and evidence for agent incidents from detection to recovery.
- AI Agent Misuse Incident Playbook: containing an agent that has been compromised or is misbehaving.
Pre-deployment assurance
- AI Agent Deployment Checklist: the security checks an agent should pass before production.
- AI Agent Security Testing Checklist: pre-release and ongoing tests for injection, tool misuse and privilege abuse.
Agents in security operations
- Agentic AI in the SOC Guide: autonomous triage, investigation and response, and the controls on SOC agents themselves.
Practitioner checklist
- Every production agent has an inventory entry, a named owner, its own identity and a recorded autonomy level per action class.
- Tool and MCP calls pass through an enforcement point that logs agent ID, version, principal, credential ID, arguments, decision and outcome.
- Delete, pay, external send and privilege change actions require a bound, single-use approval enforced outside the model.
- Per-agent budgets for tokens, spend, turns and call rates are set and alert before they are reached.
- Agent-generated code runs in a sandbox with no production secrets, no instance metadata access and default-deny egress.
- Declared baselines are enforced from day one; learned baselines are tied to model, prompt and toolset versions.
- Each agent alert class names an owner and the containment level it triggers.
- The kill switch has been drilled, with time to contain measured for each level.
- Deployment checklist and security tests rerun on every model, prompt or tool change.
- Agent logs are protected against tampering and kept for at least six months where EU AI Act deployer duties apply.
Standards and references
- OWASP GenAI Security Project: Top 10 for Agentic Applications for 2026
- OWASP: Top 10 for LLM Applications 2025
- NIST: AI 100-1 Artificial Intelligence Risk Management Framework (AI RMF 1.0)
- NIST: SP 800-53 Rev. 5 Security and Privacy Controls
- MITRE ATLAS: Human In-the-Loop for AI Agent Actions
- European Commission AI Act Service Desk: Article 14 Human oversight
- OpenTelemetry: Semantic Conventions for GenAI Agent Spans
- Cloud Security Alliance: Agentic AI Threat Modeling Framework MAESTRO
Related NHI Mgmt Group resources: Agent Governance and Risk Guide · Agent Identity and Access Guide · Agentic AI Security Guide · AI Agent Authorisation Guide