Policy-invisible violations show why LLM agent controls miss state

At a glance

What this is: This analysis shows why policy-invisible violations in LLM agents can pass content checks yet still break organisational rules when the agent lacks the state needed to enforce policy correctly.

Why it matters: IAM and security teams need to treat agent decisions as state-dependent authorisation events, not just model outputs, because missing context can turn a valid action into an unlawful one.

Context

LLM agent controls fail when the enforcement layer only sees the request and not the governing state behind it. In this pattern, the action can look valid in isolation, but the policy decision depends on recipient identity, document audience, or prior session history that is outside the model's immediate field of view.

That makes the problem an access-governance issue as much as a model-safety issue. For human IAM, NHI, and agentic workflows alike, the control boundary has to move from content inspection to context-aware authorisation at the moment an action would change the organisation's state.

Highflame frames this as policy-invisible violations: cases where the agent appears to comply while still producing an unauthorised outcome. The distinction matters because the failure is not that the model is confused, but that the enforcement logic is blind to the facts it needs.

Key questions

Q: What breaks when LLM agents only inspect content and not state?

A: Controls miss violations that depend on recipient role, document audience, or prior session history. The action can look valid in isolation while still creating an unauthorised outcome. To govern these agents properly, teams need enforcement that evaluates the state the action will create, not just the text the model produced.

Q: Why do agent policy checks fail when session history is missing?

A: Because many violations are cumulative. A read in one step may be harmless, but the later share or send becomes risky only when the earlier context is carried forward. If the system forgets what the agent has already seen, it cannot detect cross-turn leakage or audience mismatch.

Q: How do security teams know if coding agent monitoring is actually working?

A: Monitoring is working when it can correlate session behavior, file access, tool use, and network attempts into a single investigative trail. Useful signals include taint tracking after sensitive file access, anomaly detection after external content ingestion, and alerts on writes to restricted paths. If those patterns are visible, the program can distinguish intended work from manipulated behavior.

Q: What should organisations do when an LLM agent is about to share data externally?

A: Treat the share as an authorisation event, not a text-generation event. Check the recipient's role, the document's audience restriction, and any prior session data that may make the outbound action unlawful. If the system cannot evaluate those factors, it should block or require clarification before the action completes.

Technical breakdown

Why content checks miss policy-invisible violations

Content-based controls evaluate what the agent says or sends, but not the hidden facts that make the action permissible. In the article's example, the share action is structurally valid, yet the recipient's role and the file's audience restriction determine whether the action is allowed. That means the same tool call can be safe in one context and a violation in another. This is why prompt text and output filters only partially help. They can influence the model's behaviour, but they cannot guarantee access decisions when the relevant policy state is not present at evaluation time.

Practical implication: move enforcement from message inspection to state-aware authorisation.

Session state is the missing control primitive

The paper shows that many violations emerge only after multiple steps, when earlier reads shape later sends. That means the dangerous state is cumulative, not local to a single action. A session-level ledger or taint record is therefore more useful than evaluating each step independently, because the control must remember what the agent has already observed or inferred. This is especially important where a seemingly harmless read becomes relevant only when combined with a later outbound action. The core design lesson is that the policy engine needs session memory, not just a request parser.

Practical implication: retain session-level context for data flow, not just individual tool calls.

Policy decisions need the world state, not just the prompt

The article's architectural argument is that enforcement should happen against the resulting organisational state, not against the model's text. Whether that state is represented as a graph mutation engine or as declarative context fields, the control question is the same: what change will this action create, and is that change permitted? That is a different problem from asking whether the model followed instructions. For agent governance, this shifts the design centre from reasoning quality to authoritative state representation, with the policy layer holding the facts the model cannot reliably know.

Practical implication: evaluate agent actions against organisational state transitions before they are committed.

Threat narrative

Attacker objective: The objective is to cause unauthorised disclosure or policy violation through an agent action that appears valid in isolation.

1. Entry occurs through a legitimate, user-sanctioned LLM agent request that looks structurally correct and passes basic tool validation.
2. Credential or policy access is not stolen, but the agent lacks the recipient-role, document-audience, or session-history facts required to judge the action correctly.
3. Escalation happens when the agent combines individually acceptable reads and writes into a cross-context data flow that violates organisational policy.
4. Impact is an unauthorised disclosure or policy breach that appears normal at the action layer because the violation only exists in the hidden state relationship.

Breaches seen in the wild

• EchoLeak (Microsoft 365 Copilot) 2025: A crafted email could make Microsoft 365 Copilot leak data from its context with no click, a zero-click prompt injection fixed as CVE-2025-32711.
• ForcedLeak (Salesforce Agentforce) 2025: A critical Agentforce flaw let attackers steer an AI agent through a Web-to-Lead form to leak CRM data via an expired allowlisted domain.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Policy-invisible violations are a state-governance problem, not a prompt-engineering problem. The article shows that structurally correct agent actions can still violate policy when recipient identity, document audience, or session history is invisible at decision time. That means the core failure sits in the authorisation boundary, not in model reasoning quality. Practitioners should treat hidden state as part of the access decision, not as an optional enrichment layer.

Content-only DLP is the wrong control plane for multi-step agent behaviour. DLP can detect obvious leakage in the message itself, but it cannot reason over accumulated context across turns. Once an agent has read something in one step and shared something in another, the control problem becomes relationship-aware rather than string-aware. Practitioners need governance that evaluates state transitions, because isolated message review cannot catch cross-turn policy violations.

State-aware enforcement is the new baseline for LLM agent governance. The important control assumption is that the policy engine knows who the recipient is, what the object is classified as, and what the agent has already seen. When any of those facts are missing, the decision can be locally correct and globally wrong. The implication is that agent governance must be built around authoritative context, not model inference alone.

World-state visibility is the named concept that separates safe agent actions from policy-invisible ones. The article's strongest contribution is to show that the decisive variable is not whether the action looks legitimate, but whether the system can see the state needed to authorise it. That reframes LLM agent security as a visibility problem across identity, object classification, and session memory. Practitioners should use that lens when deciding where enforcement belongs.

Agent control design now spans human IAM logic and autonomous workflow logic. The same weakness that allows a human-approved action to cross a policy boundary also appears when an LLM agent composes valid steps into an unlawful outcome. That makes this an identity-governance issue, not just an AI safety issue. The practical conclusion is that authorisation models must cover stateful delegation chains, not just single prompts or single users.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: Agentic AI Security Guide

What this signals

Policy-invisible violations: This is the category teams need to name when a model is obedient but the outcome is still wrong. The control failure is not better text filtering, but richer state representation for authorisation decisions. Agent programmes that keep enforcement at the message layer will continue to miss multi-step leakage.

Identity and access teams should expect this pattern to expand as agents move from single actions to chained workflows. The practical question is no longer whether the model can follow instructions, but whether the system can preserve enough state to decide if each step is allowed. That is where governance moves from advisory to determinative.

For practitioners

• Implement state-aware policy evaluation Enforce agent actions against recipient role, document classification, and session context before the action is committed to the world state.
• Track session-level data flow Maintain a session ledger for what the agent has read, inferred, or touched so later sends can be checked against accumulated context.
• Separate content inspection from authorisation Use content filters as a detection layer, but base the allow or block decision on authoritative state rather than the message body alone.
• Model cross-turn violations explicitly Define rules for accumulated session leakage, audience mismatch, and cross-context forwarding because these cases only appear after multiple steps.
• Validate agent actions at irreversible boundaries Check the policy outcome when the agent is about to share, write, delete, or forward data, because that is where state becomes consequential.

Key takeaways

• LLM agents can violate policy while still appearing structurally correct, which means the control gap is visibility into state rather than failure of model output quality.
• The decisive evidence in the article is that recipient role, document audience, and session history are all required to judge an action correctly, and content-only controls cannot see them.
• The practical response is to enforce at the state boundary, carry session context forward, and decide allow or block before the action changes organisational data.

Key terms

• Policy-Invisible Violation: A policy-invisible violation is an action that appears valid when viewed in isolation but is still unauthorized once the full context is known. In agent systems, the failure usually comes from missing recipient, classification, or session-state facts at decision time.
• State-Aware Enforcement: A control model that evaluates the effect of an action against authoritative organisational state before it is committed. Instead of judging only the prompt or output, it checks whether the resulting change is permitted by identity, classification, and session context.
• Session-Level Taint: A record of what an agent has already read, inferred, or touched during a session so later actions can be judged in context. It is the governance equivalent of accumulated memory, and it is essential when a harmless read becomes risky only after a later send or share.
• World-State Boundary: The world-state boundary is the point at which an agent's proposed action would change organisational reality, such as sharing, sending, deleting, or writing data. Good enforcement checks the consequences of the action at that boundary rather than only reviewing the request text.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.