AI BOMs miss the real risk in agentic AI applications

At a glance

What this is: This is an analysis of why AI BOMs are insufficient for agentic AI security and why deep repository scanning plus runtime guardrails expose the real attack surface.

Why it matters: It matters because IAM, PAM, and governance teams need to understand what AI systems can access and do, not just which models are present, especially as agentic and shadow AI expand across the enterprise.

👉 Read Pillar Security's analysis of why AI BOMs miss the real risk in agentic AI applications

Context

AI BOMs create a documentation layer, but they do not describe the effective privileges of an AI system. For agentic AI, the security question is not only which model is deployed. The primary keyword here is AI BOM, and the real issue is that inventory does not capture tools, prompts, data access, or the conditions under which an AI system can cause harm.

That gap matters across NHI governance, autonomous system oversight, and human IAM operations. Security teams can satisfy compliance with a list while still missing the combinations that create real exposure: untrusted input, private data, and external communication. In practice, that means the programme knows what exists, but not what it can reach, modify, or exfiltrate when behaviour changes at runtime.

Key questions

Q: What breaks when AI teams rely on an AI BOM for security?

A: An AI BOM breaks down when teams treat it as a security control instead of a record of what exists. It does not show tool permissions, data reach, or runtime actions, so it cannot explain blast radius or stop prompt injection from becoming privileged activity. Security teams need capability mapping and enforcement, not inventory alone.

Q: Why do agentic AI systems create a larger access risk than simple chatbots?

A: Agentic AI systems create a larger access risk because they can act through tools, not just generate text. Once the model can query databases, run code, or call external APIs, a malicious prompt can become a write or execute path. The risk is defined by connected capability, not by the model label.

Q: How should security teams reduce AI application blast radius?

A: Security teams should reduce blast radius by separating low-risk conversation workflows from high-impact actions such as code execution, database writes, and external communication. They should pair repository scanning with runtime guardrails so the system is assessed before deployment and constrained during operation. That combination gives both visibility and enforcement.

Q: How do you know if AI governance is working in practice?

A: AI governance is working when the team can show which tools an application can use, which data it can touch, and which actions are blocked at runtime. If the programme only produces an inventory, it measures documentation rather than control. Effective governance is visible in action boundaries, not in model counts.

Technical breakdown

AI BOMs vs effective privilege

An AI BOM is a disclosure artifact, not a control plane. It records model names, versions, and locations, but it does not describe the permissions attached to the application, the tools it can invoke, or the data it can touch. That makes it useful for audit inventory and weak for threat modelling. In agentic environments, risk is determined by effective privilege, which is the combination of model behaviour, connected tools, exposed data, and execution context. Two systems can share the same model and have entirely different blast radii depending on what the surrounding application allows. That is why inventory alone creates a false sense of visibility.

Practical implication: map tool access, data scope, and execution paths alongside model inventory before treating AI BOMs as governance evidence.

Prompt injection becomes a write and execute problem

Prompt injection is dangerous because it changes what an AI can do once the model has access to tools. Without tools, the attack is mostly about manipulating output. With tools such as database queries, code execution, email, or API calls, the same injection can become a write, persist, or execute event. That is the architectural shift the article highlights. The controlling factor is not the prompt itself, but the reachable actions behind the prompt. If the agent can call high-impact tools and there is no scoping boundary, an attacker can use ordinary conversation as a delivery mechanism for privileged actions.

Practical implication: classify every tool by impact level and block high-risk actions unless the agent’s task scope explicitly requires them.

Deep repository scanning and runtime guardrails

Deep repository scanning looks inside the codebase, configuration files, prompts, MCP settings, and model artifacts to reveal how the application actually behaves. It surfaces hidden controls and hidden risks, including poisoned model files, permissive system instructions, and tool configuration drift. Runtime guardrails then enforce policy during execution so malicious input cannot simply flow through to a sensitive action. The two controls are complementary. Scanning identifies the attack surface and the intended boundaries. Guardrails enforce those boundaries when the system is live. A programme that has one without the other either documents risk without reducing it or blocks traffic without understanding what to block.

Practical implication: pair repository scanning with in-path runtime policy enforcement for the same AI workload, not as separate programmes.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• AI LLM hijack breach — attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI BOMs are governance artefacts, not security controls. They answer compliance questions about what exists, who owns it, and where it runs, but they do not answer what the system can reach when its behaviour is altered. That distinction matters because real AI risk emerges from effective privilege, not from model inventory. Practitioners should treat AI BOMs as a starting point, not evidence of control maturity.

Blast radius in agentic AI is determined by connected capability, not model choice. The same model can sit behind a low-risk chatbot or a high-risk agent with database, code, and external communication access. The article’s core insight is that the privilege footprint lives in the surrounding architecture, including tools, prompts, and MCP-style connections. The implication is that inventory-led governance misses the actual path to harm.

Runtime guardrails matter because scanning alone cannot stop a live attack path. Deep repository scanning reveals hidden surfaces such as configuration backdoors, malicious artifacts, and unsafe instructions, but it does not block exploitation during execution. That is why the control model must extend from discovery to enforcement. Practitioners should rethink AI security as a layered identity and policy problem, not a documentation exercise.

Effective AI governance requires understanding the lethal trifecta, not just the model stack. Access to private data, exposure to untrusted content, and the ability to communicate externally create the condition for abuse. Once those three are combined, even a well-documented AI application can become an exfiltration path. Security teams should map that combination explicitly across their highest-risk workloads.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
• Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
• For the broader control picture, see OWASP NHI Top 10 for agentic application risk patterns.

What this signals

AI BOMs will not be enough as agentic adoption scales. With 92% of organisations agreeing that governing AI agents is critical, but only 44% having implemented any policies, the gap is already operational rather than theoretical. Teams should expect auditors to move from asking whether an AI exists to asking what it can do, what it can reach, and who can shut it down. The control conversation is shifting from documentation to enforceable privilege boundaries.

Capability mapping will become the real governance differentiator. The next maturity step is not another inventory field, but a defensible view of tools, data paths, and action permissions across each workload. That aligns closely with the control logic in the OWASP Agentic Applications Top 10, where tool misuse and privilege abuse are explicit risks. Practitioners who cannot answer those questions will struggle to classify risk consistently across NHI and autonomous systems.

For practitioners

• Inventory effective privilege, not just model names Document every tool, data source, system prompt, and external connection tied to each AI workload. Use that map to identify where an AI system can read, write, execute, or communicate beyond the intended business task.
• Classify AI tools by blast radius Treat database write, code execution, email sending, and external API access as separate control tiers. Require explicit business justification and tighter policy enforcement for high-impact actions exposed to agentic workflows.
• Scan repositories for hidden AI control inputs Inspect prompts, configuration files, MCP settings, model artifacts, and dependency packages for malicious instructions, unsafe defaults, and serialization risks before approving deployment or upgrades.
• Deploy runtime guardrails at the action boundary Enforce allowlists, content filters, and action-level policies in path so a compromised prompt cannot directly trigger destructive queries, exfiltration, or persistence events.
• Reassess shadow AI as an access problem Find unmanaged AI applications by tracing data access, tool permissions, and external communications, then route them through the same governance and review process used for other non-human identities.

Key takeaways

• AI BOMs help with compliance, but they do not reveal the effective privilege that determines real AI security risk.
• The evidence in this article points to a gap between inventory and control, where tools, prompts, and data access define the attack surface.
• Practitioners should pair deep repository scanning with runtime guardrails so they can see what AI can do and constrain it when it acts.

Key terms

• AI BOM: An AI Bill of Materials is a structured inventory of the components used to build and run an AI system. It is useful for disclosure and audit, but it does not explain effective privilege, runtime behaviour, or the attack surface created by connected tools and data.
• Effective Privilege: Effective privilege is the real set of actions an AI system can perform in context, including what it can read, write, execute, and send. For agentic systems, this is shaped by tools, prompts, permissions, and external connections, not just by the model itself.
• Runtime Guardrails: Runtime guardrails are in-path controls that inspect and constrain AI behaviour while the system is operating. They are designed to prevent unsafe prompts, tool misuse, data exfiltration, and destructive actions from reaching downstream systems.
• Shadow AI: Shadow AI refers to AI systems that exist in an environment without proper visibility, governance, or approval. These systems may use real credentials, access real data, and interact with production tools while remaining outside the organisation's standard inventory and control processes.

Deepen your knowledge

AI BOM limitations, AI application inventory, and runtime guardrails are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building governance for AI systems with tool access and hidden control inputs, it is worth exploring.

This post draws on content published by Pillar Security: Not All AI BOMs Are Created Equal. Read the original.