Topic starter
05/07/2026 6:52 pm
TL;DR: AI agent security is being tested at the backbone LLM level using nearly 200,000 human red-team attempts and ten threat snapshots to measure how models behave under prompt injection, tool misuse, and data exfiltration pressure, according to Lakera. The key shift is that security must be measured at the decision point, not inferred from safety labels or end-to-end agent complexity.
Editorial analysis by NHI Mgmt Group, based on content published by Lakera: “The Backbone Breaker Benchmark: Testing the Real Security of AI Agents”.
Key questions
Q: How should security teams test AI systems beyond output filtering?
A: Security teams should test the full execution path, including prompt handling, retrieval, memory, tool selection, parameter construction, and downstream API responses.
Q: Why do safe-looking models still fail in agentic environments?
A: Because safety and security are different failure modes.
Q: What are the signs that AI agent testing is too shallow?
A: If evaluation only checks refusal behaviour, overall benchmark scores, or end-to-end task success, it is probably missing the real attack surface.
Practitioner guidance
- Separate backbone and wrapper testing Validate the core model independently from orchestration, memory, and tool integrations so you can isolate whether failures originate in the LLM or the surrounding stack.
- Test against real attack paths Include prompt injection, malicious tool calls, and data exfiltration scenarios drawn from observed adversarial behaviour rather than only synthetic safety prompts.
- Treat AI action boundaries as governance controls Define which model decisions require human review, which tool calls are allowed, and which execution paths must terminate when context appears manipulated.
Bottom line: AI agent security cannot be judged reliably from safety labels or full-stack complexity alone, because the exploitable moment often sits inside the backbone model’s decision call.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
05/07/2026 7:12 pm
Backbone-first testing exposes the real control boundary for AI agents. The decisive security boundary is not the outer workflow, but the backbone LLM call where a malicious prompt becomes a tool action or an exfiltration decision. That means agent security cannot be assessed only through interface hardening or application-level policy. Practitioners need to treat the core model as the point where runtime trust is either preserved or lost.
A few things that frame the scale:
- Systems with least-privileged AI access had a 17% incident rate vs 76% for over-privileged systems. Organisations failing to scope AI access properly are 4.5x more likely to experience a security incident, according to the 2026 Infrastructure Identity Survey.
- 88% of organisations have embedded AI agents in their workflows, according to KPMG's 2026 report.
A question worth separating out:
Q: How should security teams govern AI agents that choose tools at runtime?
A: Security teams should treat runtime tool choice as a governed access event, not a normal application call. That means task-scoped credentials, explicit approval boundaries for sensitive actions, and logs that record both the tool selected and the identity used. If the agent can change its plan, the control model must be able to change with it.
👉 Read our full editorial: Backbone-first AI agent security exposes the limits of safety tests