AI agent trust boundaries are the new security problem at Black Hat

At a glance

What this is: This is an analysis of how AI agents change the trust model inside enterprise environments, with a focus on autonomous action, exposed credentials, and boundary crossing.

Why it matters: It matters because identity teams now have to govern systems that can decide, act, and escalate faster than human-centric IAM, PAM, and review processes can reliably contain.

👉 Read Noma Security's analysis of AI agent trust boundaries at Black Hat USA 2026

Context

AI agent identity risk is no longer a theoretical edge case. When a goal-seeking system can move from evaluation into real systems, the identity programme has to account for what the actor can decide to do, not just what it was authorised to start with.

The governance gap is that most IAM and access review models still assume a stable, human-paced subject behind the account. Once autonomous behaviour enters the trust chain, the question changes from who approved access to what the system can do before anyone notices.

This article frames Black Hat as the trigger, not the subject. The real issue is whether agentic systems are being treated as governed identities with scoped authority, or as software that inherited trust by accident.

Key questions

Q: How should security teams govern AI agents that move across multiple trust boundaries?

A: They need runtime controls that follow the agent rather than staying attached to one platform. The practical test is whether enforcement, telemetry, and inventory remain consistent as the agent moves from IDEs to MCP servers to downstream SaaS actions. If the control breaks at the boundary, governance is incomplete.

Q: Why do agentic AI systems change IAM risk?

A: They change risk because they can move from observation to execution. That means the control problem shifts from whether a system can detect a problem to whether it can safely act on one. IAM teams must now manage privilege scope, action boundaries, and revocation for autonomous workflows.

Q: What breaks when AI workflow inputs can influence execution?

A: Traditional data validation is not enough when a prompt, model file, loader, or dataset can change what runs next. In that case, the ingestion path becomes part of the security boundary, and teams need runtime checks that detect misuse after validation has passed.

Q: Who is accountable when an AI agent causes a security incident?

A: Accountability should sit with the business owner, the system owner, and the security function together, because agent behaviour crosses operational boundaries. Organisations need a defined owner for approval, monitoring, and retirement, plus audit evidence that shows what the agent accessed and why.

Technical breakdown

Executable data and agent trust boundaries

AI pipelines often treat datasets, prompts, model files, loaders, plugins, and tool manifests as ordinary inputs, but many of those artefacts can influence execution. That makes the ingestion path part of the security boundary. If untrusted content can trigger code paths, alter configuration, or shape downstream action selection, the pipeline is no longer just processing data. It is participating in execution. The core technical issue is not only validation at ingest, but whether the runtime can still enforce containment after the initial check has passed.

Practical implication: classify AI pipeline inputs that can affect execution as security-sensitive artefacts and test runtime containment, not just pre-ingest validation.

Agent authority, tool use, and credential scope

An AI agent is not just a user interface for a model. When it can call tools, access data, and make decisions at runtime, it begins to behave like a control plane for the workflow it sits in. That changes the security model from authenticated request handling to delegated authority management. The critical issue is that the same identity may be used for human actions, scripted actions, and agent actions unless the organisation separates them. Without distinct scope, logging, and revocation paths, accountability collapses into a single opaque identity trail.

Practical implication: separate agent authority from human and scripted access paths, and require distinct identity, audit, and revocation handling for each.

Containment, observability, and blast radius for AI systems

Once an agent or automated workflow is compromised, the important question becomes how far its access can spread. That depends on least privilege, workload identity, secret rotation, egress control, segmentation, and runtime logging working together. In AI environments, the failure is often not a missing control but an inherited access pattern that was never narrowed for non-human use. If the same credentials can reach code, data, clusters, and external tools, the blast radius grows quickly and incident response becomes a search for all the places that identity was trusted.

Practical implication: map each agent identity to a bounded blast radius and verify that compromise of one workflow cannot reach unrelated systems.

Threat narrative

Attacker objective: The attacker aims to turn AI workflow trust into broader internal access that can be used for lateral movement, credential theft, and control over downstream systems.

1. Entry begins when a malicious dataset or compromised evaluation path exploits code-execution opportunities inside an AI processing pipeline.
2. Escalation occurs as the system gains node-level access, exposes cloud and cluster credentials, and uses legitimate tooling to move laterally.
3. Impact follows when the agent or intruder crosses trust boundaries, expands access across internal clusters, and reaches systems beyond the original workflow.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• Meta AI Instagram Account Takeover — 20,225 Instagram accounts hijacked via compromised Meta AI support chatbot with overprivileged access.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI agent trust boundaries are now an identity problem, not just a model safety problem. The article shows that the most consequential failures happen when autonomous systems cross boundaries that were designed for human-paced or script-driven action. Once an agent can choose actions and continue execution without a human in the loop, the trust model has to be judged as identity governance, not simply AI safety. Practitioners need to treat agent authority as a governed access surface.

Access review processes assume privilege persists long enough to be reviewed, and that assumption fails under autonomous behaviour. A review cadence designed around stable entitlements cannot explain or constrain a system that can acquire, chain, and use access within a single goal-directed run. The issue is not that reviews are absent, but that the underlying identity premise no longer fits the actor. The implication is that governance models must be reconsidered for time-bounded, runtime-driven privilege rather than static entitlement lists.

Executable data creates a runtime governance gap that traditional IAM did not have to solve. Datasets, prompts, loaders, and tool manifests now sit on the boundary between information and instruction. That means a control model built for passive data handling no longer covers the full attack surface. Practitioners should recognise that the security boundary has moved upstream into the data-processing path, where trust decisions can influence execution itself.

Authority without actor separation creates accountability collapse across humans, scripts, and agents. If the same identity is used for human operations, automation, and agentic action, post-incident attribution becomes blurred and revocation becomes imprecise. That weakens both technical response and legal defensibility. The practical conclusion is that identity programmes need explicit separation between who requested work, what executed it, and which non-human system acted on it.

Containment, not just detection, is the decisive control for agentic environments. The article repeatedly points to systems that can move from one trust boundary to another using legitimate access once they are inside. That is a blast-radius problem as much as an intrusion problem. NHI and AI governance should therefore be judged by how narrowly an agent can fail, not just by whether the failure is visible.

From our research:

• 85% of organisations lack full visibility into third-party vendors connected via OAuth apps, according to The State of Non-Human Identity Security.
• Only 1.5 out of 10 organisations are highly confident in their ability to secure NHIs, compared with nearly 1 in 4 for securing human identities.
• For a broader governance baseline, see Ultimate Guide to NHIs , Key Challenges and Risks for the visibility, sprawl, and over-privilege patterns that still dominate NHI programmes.

What this signals

Agentic AI changes the review cadence problem into a runtime containment problem. If a system can decide, execute, and escalate inside one session, governance based on quarterly reviews or after-the-fact certification will always trail the event. Identity teams should be preparing for controls that prove scope during execution, not only after the fact.

Executable data is the concept practitioners should carry forward from this article. When data artefacts can influence execution, the boundary between information and code disappears in practice. That should push security architects to reassess pipeline trust, runtime inspection, and the separation between data ingestion and privilege-bearing action.

The wider implication is that AI governance cannot be bolted onto existing IAM as an afterthought. The programme has to distinguish human intent, scripted automation, and autonomous action or it will continue to miss the fastest-moving identity paths in the environment.

For practitioners

• Define agent-specific trust boundaries Map every AI workflow to the exact inputs, tools, data sources, and execution paths it is allowed to reach. If a dataset, prompt, or connector can influence execution, treat it as part of the security boundary and test it for runtime containment.
• Separate agent identity from human access paths Do not let agent actions disappear into shared user, service, or pipeline identities. Give autonomous workflows distinct credentials, distinct logging, and distinct revocation handling so accountability survives incident response.
• Rework access reviews for runtime behaviour Assess whether your recertification model can meaningfully govern access that is created, used, and discarded inside one execution window. If not, move to controls that inspect runtime behaviour and scope rather than only periodic entitlement lists.
• Bound the blast radius of every agent Apply least privilege, egress control, segmentation, secret rotation, and workload identity so compromise of one workflow does not become broad internal reach. The test is simple: if one agent identity is stolen, how much of the environment is exposed?

Key takeaways

• AI agents are now part of the identity threat surface, because they can cross trust boundaries and act faster than human governance loops can respond.
• The evidence points to a shift from static access management to runtime containment, where the key question is how far a compromised workflow can move.
• Practitioners should separate agent authority, tighten blast radius, and test whether current IAM and review processes still make sense for autonomous behaviour.

Key terms

• Agent Authority: The permission an AI agent receives to act on behalf of a verified person. In this model, authority is inherited rather than original, so governance must trace the agent back to the human intent, device context, and current trust state that authorised it.
• Executable data: Information that can influence behaviour or trigger action inside an AI system rather than remaining passive content. This is a useful way to describe why AI changes security planning, because the same data now affects both what the system knows and what it may do next.
• Blast Radius: The potential scope of damage if a specific credential or identity is compromised. Identities with broad permissions have a larger blast radius and represent a higher priority for least-privilege enforcement and security controls.

What's in the full article

Noma Security's full article covers the operational detail this post intentionally leaves for the source:

• Specific Black Hat talking points and how the vendor frames each AI security signal for conference attendees
• Direct commentary on the Hugging Face and OpenAI incidents, including the exact sequence of trust-boundary crossings
• The vendor's full set of questions for evaluating AI demos, pipelines, and agent authority in practice
• Operational context on what CISOs should listen for when AI capabilities are presented as security controls

👉 The full Noma Security article covers the incident details, Black Hat questions, and the practical trust-boundary test.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.