AI agent detection is becoming central to NIST cyber AI governance

At a glance

What this is: Zenity’s recap of the NIST NCCoE working session argues that AI agent detection is becoming a core requirement for AI governance.

Why it matters: That matters because identity, monitoring, and accountability models built for static software do not adequately cover agents that choose actions at runtime across tools and environments.

By the numbers:

• 300 people joined the discussion on how a, n on how a cyber AI profile should overlay NIST CSF 2.0 for AI system components.

👉 Read Zenity's takeaways from the NIST NCCoE AI security working session

Context

AI agent detection is the governance problem this article puts at the centre of the discussion. The issue is not whether AI systems exist inside the enterprise, but whether security teams can observe what an agent is doing once it is running, selecting actions, and moving across tools in ways traditional application monitoring was never designed to track.

That makes the topic relevant to agentic AI identity and broader IAM governance at the same time. The working session reflects a shift in practitioner thinking: the hard part is no longer just defining policy for AI systems, but building detection, accountability, and organisational context into the controls that surround them.

Key questions

Q: How should security teams detect risky AI agent behaviour in production?

A: Security teams should detect risky AI agent behaviour by monitoring runtime decisions, tool selection, action sequences, and deviations from the approved use case. Authentication alone is not enough. The useful signal is whether the agent is still acting within the behavioural envelope defined by its purpose and ownership model, especially when it operates across multiple systems.

Q: Why does AI complicate NIST CSF 2.0 governance?

A: AI complicates NIST CSF 2.0 governance because the framework still works, but the interpretation of Govern and Detect changes when the system can act autonomously. Security leaders must define context, ownership, and expected behaviour first, or the framework produces policy without operational enforcement.

Q: What breaks when organisations monitor AI agents like normal applications?

A: What breaks is behavioural visibility. Normal application monitoring assumes stable process paths and predictable event patterns, but AI agents can chain actions, shift tools, and move across systems in ways that look normal at the log level while being unusual at the control level.

Q: Who is accountable when an AI agent takes an unexpected action?

A: Accountability should sit with the team that defined the agent’s purpose, access, and oversight model, not with the monitoring tool or the model itself. If ownership, acceptable use, and escalation paths are unclear, the organisation cannot explain or govern the action after the fact.

Technical breakdown

Why AI agent detection is different from application monitoring

AI agents do not behave like fixed-function applications. They can choose actions at runtime, chain tool calls, and continue working without a human checkpoint between steps. That means conventional monitoring, which looks for known process paths or static service behaviour, can miss the meaningful part of the event: the agent’s decision path. In an identity context, the thing to watch is not only the credential used, but the actions taken under that credential and whether those actions still match the approved purpose.

Practical implication: security teams need telemetry that captures agent decisions, tool use, and action sequences, not just login events or API call volume.

Cyber AI Profile as an overlay to NIST CSF 2.0

The working session framed the cyber AI profile as an overlay rather than a replacement for NIST CSF 2.0. That matters because CSF functions such as Govern, Identify, Protect, Detect, Respond, and Recover still apply, but AI introduces new interpretation pressure inside each one. For AI agents, Detect becomes more than alerting. It becomes continuous behavioural understanding, including when an agent’s activity diverges from its intended task or starts producing unexpected operational patterns.

Practical implication: map AI agent controls to existing CSF functions instead of creating a separate governance island.

Organisational context is a control, not a backdrop

The article argues that AI adoption depends on stakeholder alignment, mission clarity, and shared expectations before security strategy can hold. That is an important governance point: context is not a soft planning layer, it is the condition that makes every downstream control interpretable. If the board, operators, and security leaders do not agree on why the AI exists, what it may do, and who owns its outcomes, then monitoring and policy enforcement become theoretical exercises rather than enforceable controls.

Practical implication: define ownership, use cases, and expected outcomes before you treat AI monitoring data as meaningful.

Threat narrative

Attacker objective: The objective is to let the agent operate far enough inside the environment that its activity blends into normal business execution while bypassing timely detection.

1. Entry occurs when an AI agent is granted operational access into enterprise tools and networked systems to perform real work.
2. Escalation occurs when the agent begins chaining actions autonomously across systems, creating a behavioural web that is difficult to follow with conventional monitoring.
3. Impact occurs when teams cannot reliably detect, explain, or respond to what the agent did before its actions affect data, workflows, or trust boundaries.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• AI LLM hijack breach — attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Detection becomes the governance boundary when AI agents move from theory to operations: Once an agent can select actions at runtime, the old assumption that monitoring is mostly about known application paths no longer holds. Security teams now need to judge whether a runtime identity is acting within expected behavioural bounds, not just whether it authenticated correctly. The practitioner implication is that AI security programmes must treat detection as a primary control surface, not a secondary log review function.

Organisational context is the control that decides whether AI governance is actionable: The session’s emphasis on stakeholder alignment reflects a deeper reality that AI controls fail when mission, ownership, and acceptable use are undefined. That is true across human IAM, NHI governance, and autonomous systems, but AI makes the gap more visible because behaviour changes faster than policy cycles. The practitioner conclusion is that governance without agreed context produces reports, not control.

Runtime behaviour is the new identity signal for autonomous systems: Operational AI agents should be evaluated less like software licences and more like dynamically acting identities whose access pattern changes during execution. That shifts the governance question from static entitlement to observable purpose, sequence, and scope. The implication is that identity programmes must measure what the actor did, not only what it was allowed to do at provisioning time.

NIST CSF 2.0 remains relevant, but AI forces a stricter reading of its Detect and Govern functions: The article is not arguing for a new framework so much as a sharper operationalisation of the one already in use. Detection, oversight, and supply chain risk all become more demanding when the asset in question can self-direct across systems. The practitioner conclusion is to adapt existing control architecture before creating parallel AI-only governance structures.

AI agent governance will converge with broader identity governance faster than most programmes expect: The reason is simple: once an agent is operating as a runtime actor, its control problems resemble identity control problems more than application support problems. That brings AI oversight into the same conversation as lifecycle ownership, access scope, and behavioural monitoring. The practitioner conclusion is that teams should stop treating AI security as a niche capability and fold it into identity governance now.

From our research:

• 1 in 4 organisations are already investing in dedicated NHI security capabilities, with an additional 60% planning to do so within the next twelve months, according to The State of Non-Human Identity Security.
• Only 1.5 out of 10 organisations are highly confident in their ability to secure NHIs, compared to nearly 1 in 4 for securing human identities.
• The governance gap is broad enough that teams should use the NHI Lifecycle Management Guide to connect ownership, review, and offboarding before agent oversight becomes an afterthought.

What this signals

Runtime AI oversight is converging with identity governance, not replacing it: Teams that already manage privileged human and machine access have the right mental model for AI agents, but they need stronger behavioural telemetry. The control question is moving from who authenticated to what the runtime actor did after authentication, which is where identity and monitoring now intersect most sharply.

Organisational context will decide whether AI security programmes succeed: If the use case is unclear, detection data becomes noise and policy becomes theatre. Security leaders should treat mission definition, ownership, and acceptable behaviour as prerequisites for meaningful monitoring, especially where agents can operate beyond a single workflow.

With 1 in 4 organisations already investing in dedicated NHI security capabilities, the market signal is that operational oversight is no longer optional. The next step is to bind those investments to established identity controls and to Top 10 NHI Issues so AI governance does not fragment into isolated tooling.

For practitioners

• Define AI agent detection requirements Specify which runtime behaviours must be logged, correlated, and alertable, including tool choice, action sequence, and scope drift across systems.
• Map AI controls to CSF 2.0 Assign each AI security requirement to an existing NIST CSF 2.0 function so governance, detection, and response remain operationally coherent.
• Establish organisational context for AI use Document the mission, stakeholders, and approved outcomes for each AI use case before you rely on telemetry to judge acceptable behaviour.
• Treat agents as monitored identities Build oversight for AI agents the way you would for privileged identities, with ownership, reviewability, and response paths that match their operational reach.

Key takeaways

• AI agent detection is becoming a core control because runtime behaviour matters more than static authentication in operational environments.
• The NIST CSF 2.0 overlay discussion shows that AI governance works best when it extends existing identity and monitoring disciplines rather than bypassing them.
• Security teams should define mission, ownership, and acceptable behaviour before expecting telemetry to deliver meaningful AI oversight.

Key terms

• AI agent detection: AI agent detection is the ability to observe, interpret, and alert on what an agent does after it starts running. It goes beyond login logs or API counts and focuses on behaviour, sequence, scope drift, and whether the agent is still acting inside its approved purpose.
• Cyber AI profile: A cyber AI profile is an overlay that adapts existing cybersecurity frameworks to AI-specific behaviour and risk. It preserves the underlying framework structure while adding expectations for runtime control, behavioural monitoring, governance, and response when the system can act more dynamically than traditional software.
• Organisational context: Organisational context is the set of mission goals, stakeholder expectations, dependencies, and legal or contractual constraints that shape security decisions. In AI governance, it determines whether detection data and policy controls have a real operational meaning or simply describe an abstract intent.
• Runtime actor: A runtime actor is a system that makes operational decisions while it is executing, rather than only following a fixed script or scheduled workflow. For AI agents, that means access control and monitoring must account for live decision-making, not just pre-approved configuration.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or governance in your organisation, it is worth exploring.

This post draws on content published by Zenity: Why Detection? Why Now? Key Takeaways from the NIST NCCoE Public COI Working Session. Read the original.