Notifications
Clear all
Topic starter
22/06/2026 10:13 am
TL;DR: A model recall does not address the real security boundary in AI systems, because refusals are statistical and the decisive control sits in the harness and blast radius around the model, according to Pillar Security. The durable fix is to bound what an AI system can reach and do, not to rely on the model to police itself.
NHIMG editorial — based on content published by Pillar Security: The Fable Recall Puts the Spotlight in the Wrong Place
Questions worth separating out
Q: How should security teams limit AI system damage when model refusals are unreliable?
A: Security teams should limit the AI system's reachable blast radius first, then enforce policy outside the model for any tool use, data access, or secret handling.
Q: Why do prompt injections remain dangerous even when a model is well aligned?
A: Prompt injections target the harness, not just the model.
Q: What do identity teams get wrong about AI access controls?
A: They often focus on whether the model behaves safely instead of whether the system's permissions are tightly scoped.
Practitioner guidance
- Define the model's reachable blast radius Inventory every API, file store, secret, and execution path the AI system can touch, then remove anything that is not required for the use case.
- Enforce action controls outside the model Place deterministic policy checks between the model and any privileged operation so the model cannot directly decide on sensitive access, data movement, or tool invocation.
- Separate untrusted content from trusted instructions Treat retrieved web pages, tool outputs, and user prompts as different trust classes, and block any path where untrusted text can be interpreted as governance instructions.
What's in the full article
Pillar Security's full blog post covers the operational detail this post intentionally leaves for the source:
- The article's full framing of how to distinguish jailbreaks from prompt injections in operational AI environments.
- The vendor's explanation of how to model blast radius around tools, files, and credentials before deployment.
- The source post's discussion of where to place controls between the model and sensitive systems so instructions cannot bypass policy.
- The article's examples of how recall decisions miss the actual access problem when the capability is already commoditised.
👉 Read Pillar Security's analysis of AI model refusals, prompt injection, and blast radius →
AI blast radius and model guardrails: what IAM teams miss?
Explore further
22/06/2026 11:02 am
Model refusals were never the control boundary, and treating them as one is a category error. Refusal behaviour is probabilistic, so it cannot function like an identity policy or an authorization gate. That assumption breaks as soon as the system is allowed to act on tools, files, or credentials. The implication is that AI governance must start with the reachable environment, not with the model's output layer.
A few things that frame the scale:
- 98% of companies plan to deploy even more AI agents within the next 12 months, despite documented rogue behaviour in 80% of current deployments, according to AI Agents: The New Attack Surface report.
- A separate finding in the same report shows that 33% of organisations say their AI agents have already accessed inappropriate or sensitive data beyond intended scope.
A question worth separating out:
Q: How do you know if an AI agent is overexposed?
A: An AI agent is overexposed when it can reach resources that are not strictly necessary for its job, especially secrets, execution tools, and internal data stores. A good test is simple: remove a permission and see whether the use case still works. If not, the permission may be justified; if yes, it is likely excess reach.
👉 Read our full editorial: AI model recall misses the real control point in agent security
