Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

AI BOMs and agentic risk: what IAM teams are missing


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 9079
Topic starter  

TL;DR: Security teams are being pushed to inventory AI systems, but model lists and vendor registers reveal little about actual blast radius. Pillar Security argues that the risk sits in tools, system prompts, data access, and runtime behaviour, not in the BOM itself.

NHIMG editorial — based on content published by Pillar Security: Not All AI BOMs Are Created Equal

Questions worth separating out

Q: What breaks when AI teams rely on an AI BOM for security?

A: An AI BOM breaks down when teams treat it as a security control instead of a record of what exists.

Q: Why do agentic AI systems create a larger access risk than simple chatbots?

A: Agentic AI systems create a larger access risk because they can act through tools, not just generate text.

Q: How should security teams reduce AI application blast radius?

A: Security teams should reduce blast radius by separating low-risk conversation workflows from high-impact actions such as code execution, database writes, and external communication.

Practitioner guidance

  • Inventory effective privilege, not just model names Document every tool, data source, system prompt, and external connection tied to each AI workload.
  • Classify AI tools by blast radius Treat database write, code execution, email sending, and external API access as separate control tiers.
  • Scan repositories for hidden AI control inputs Inspect prompts, configuration files, MCP settings, model artifacts, and dependency packages for malicious instructions, unsafe defaults, and serialization risks before approving deployment or upgrades.

What's in the full article

Pillar Security's full blog covers the operational detail this post intentionally leaves for the source:

  • Examples of repository artefacts that reveal unsafe AI behaviour, including prompts, MCP configurations, and serialized model files
  • The specific vulnerability patterns behind pickle-based model risk and configuration backdoors in coding assistants
  • How runtime guardrails are applied to block destructive tool use, data exfiltration, and unsafe external communications
  • The article's full argument for moving from inventory-led compliance to application-level AI security control

👉 Read Pillar Security's analysis of why AI BOMs miss the real risk in agentic AI applications →

AI BOMs and agentic risk: what IAM teams are missing?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 2 months ago
Posts: 8508
 

AI BOMs are governance artefacts, not security controls. They answer compliance questions about what exists, who owns it, and where it runs, but they do not answer what the system can reach when its behaviour is altered. That distinction matters because real AI risk emerges from effective privilege, not from model inventory. Practitioners should treat AI BOMs as a starting point, not evidence of control maturity.

A few things that frame the scale:

  • 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
  • Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.

A question worth separating out:

Q: How do you know if AI governance is working in practice?

A: AI governance is working when the team can show which tools an application can use, which data it can touch, and which actions are blocked at runtime. If the programme only produces an inventory, it measures documentation rather than control. Effective governance is visible in action boundaries, not in model counts.

👉 Read our full editorial: AI BOMs miss the real risk in agentic AI applications



   
ReplyQuote
Share: