Public benchmark scores expose agent egress control gaps

At a glance

What this is: PipeLab is publishing public benchmark results for tools that inspect traffic between AI agents and the network, showing that containment, detection, and evidence quality are separate control outcomes.

Why it matters: This matters because IAM and security teams evaluating AI agent controls need proof of blocked exfiltration paths, not just alerts, especially when tools sit on the runtime boundary between agents and external systems.

By the numbers:

• The benchmark has grown to 197 logical cases across 18 categories.
• The original published Pipelock v2.1.2 run reported 96.2% containment on applicable cases.
• The run covered 142 of 151 applicable cases.
• The corpus includes 149 block-expected attack cases, 47 allow-expected false-positive controls, and one warn-class drift case.

Context

The control problem here is runtime boundary governance for AI agents, not model quality or prompt writing. When an agent can move data through URLs, request bodies, headers, WebSockets, MCP calls, or nested tool arguments, the security question becomes whether the runtime layer can stop exfiltration before it leaves the environment.

Public scoring changes the discussion from feature claims to measurable control behaviour. A shared corpus makes it easier to compare tools on the same attack patterns, but it also exposes an uncomfortable truth: a tool can block traffic and still leave poor evidence, weak attribution, or blind spots in specific payload formats.

Key questions

Q: What breaks when AI agent egress controls only inspect obvious payloads?

A: They miss the cases that matter most in production, including encoded bodies, fragmented requests, and nested tool arguments. That leaves the organisation with a control that looks effective in demos but fails against realistic exfiltration paths. The right test is whether the boundary control can stop hidden data before it leaves the environment.

Q: Why do containment and detection need to be measured separately for AI agent controls?

A: Because a tool can block traffic without proving what it blocked or why. Containment tells you whether the exfiltration stopped. Detection and evidence tell you whether the event can be triaged, tuned, and audited. Without those extra signals, a control may be operationally opaque even when it appears to work.

Q: How should security teams set access boundaries for AI agents?

A: Security teams should define agent boundaries around data sources, tools, and allowed actions before any production rollout. The boundary must be explicit enough to block unintended combinations of access, because agent value often comes from chaining capabilities across systems. Treat the boundary as a runtime control, not just a policy document.

Q: When should organisations treat AI agent traffic inspection as a governance issue?

A: As soon as the agent can move secrets, tokens, or sensitive data through networked tools. At that point the question is no longer just malware-style blocking. It becomes who can send what, through which channel, under which policy, and whether the control can prove it acted correctly.

Technical breakdown

What containment measures at the agent-network boundary

Containment is the core runtime control in this benchmark because it measures whether the tool actually blocks an attack before data leaves the boundary. In practice, this is closer to an inline enforcement point than a passive detector. For AI agents, that boundary can include HTTP requests, tool calls, WebSocket frames, and nested payloads that hide secrets inside encoded or fragmented content. A high containment score is useful only when the tool can distinguish malicious exfiltration from normal agent traffic without collapsing into broad blocking. Read Zero Trust for AI Agents.

Practical implication: validate blocking behaviour on the exact transport and encoding paths your agents use, not just on simple URL-based leaks.

Why detection and evidence are not the same control

Detection answers whether the tool recognised the attack type, while evidence answers whether it produced machine-readable proof of what happened. Those are separate requirements. A block with no attribution may stop exfiltration, but it does not support incident triage, trend analysis, or control tuning. For agent environments, this distinction matters because the same control point may need to explain whether the trigger was prompt injection, credential exfiltration, SSRF bypass, or tool poisoning. Without structured evidence, operators cannot tell which rule worked or which one failed silently. See the AI Agent Observability, Audit and Incident Response Guide.

Practical implication: require structured findings, not just pass or fail, so blocked agent activity remains auditable.

Why payload decoding depth determines real coverage

The corpus highlights a common failure mode: scanners often inspect only the first layer of encoding. If a credential is base64-wrapped, embedded in multipart form data, or split across multiple requests, single-pass inspection misses it. That is not a narrow implementation detail. It is the difference between intercepting obvious leaks and catching deliberate evasion. For AI agent traffic, attackers and accidental leaks both benefit from multi-layer encodings, chained tool calls, and protocol nesting, so the enforcement layer must understand enough structure to inspect beyond surface text. See the Agentic AI Security Guide.

Practical implication: test recursive decoding, multipart parsing, and protocol nesting before trusting an agent egress control in production.

Threat narrative

Attacker objective: The objective is to exfiltrate credentials or other sensitive data through AI agent traffic without triggering an effective inline block or producing usable evidence.

1. Entry occurs when the agent sends data through ordinary runtime channels such as requests, headers, WebSocket frames, or tool arguments.
2. Escalation occurs when the payload is hidden with multi-layer encoding, fragmentation, or protocol nesting that bypasses shallow inspection.
3. Impact occurs when credential material or other sensitive content leaves the environment without being blocked or attributed by the control layer.

Breaches seen in the wild

• CoPhish OAuth phishing via Copilot Studio: Datadog showed Copilot Studio agents on a Microsoft domain can front OAuth consent phishing and forward stolen tokens; no victims reported.
• AI agent retail card theft campaign 2026: AI agents breached 27+ retailers for about $25 each, used cloud keys and a Secrets Manager dump, and stole 600,000+ payment cards.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Runtime-boundary assurance is becoming the real identity control for AI agents: the article is not about whether a model can think well, but whether the enforcement layer can stop identity-bearing traffic before it exits the network. That shifts the control conversation from model safety to session-level boundary control, where containment is the primary security outcome and evidence is the audit layer. Practitioners should treat agent egress as an identity governance problem with runtime consequences, not just a DLP variant.

Containment without structured evidence is not operationally complete: a blocked leak that cannot be attributed still leaves the programme blind to what fired, what was missed, and which transport path needs tuning. That gap matters because agent traffic is heterogeneous and often protocol-rich, so a single inline decision does not equal observability. Security teams should measure the enforcement layer on both block quality and proof quality, or the control will be too opaque to tune.

Multi-layer decoding is now a governance assumption, not a niche parsing feature: the benchmark’s encoded and fragmented cases show that shallow inspection assumes payloads are legible at first glance. That assumption breaks when the actor is an agent that can assemble content across nested calls, tool outputs, and transformed payloads. The implication is that runtime policy must account for structure, not just strings, if it is to govern AI agent authorisation in a meaningful way.

Public scoreboards will pressure vendors to prove the full control stack, not just headline block rates: once benchmark results are shared, buyers can see whether a tool blocks traffic, explains decisions, and handles common evasion patterns. That changes evaluation from demo-driven confidence to corpus-driven scrutiny. Practitioners should expect more attention on coverage gaps, attribution quality, and applicability rules than on simple success percentages.

AI agent security is converging with zero-trust enforcement at the network edge: the benchmark reinforces that identity, request context, and policy need to be evaluated per action, not assumed from a trusted session. A control that cannot verify and explain each decision will struggle as agent traffic becomes more varied and more autonomous. Teams should plan for enforcement models that behave like continuous verification, not static allow or deny rules.

From our research library:

• 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
• Systems with least-privileged AI access had a 17% incident rate vs 76% for over-privileged systems. Organisations failing to scope AI access properly are 4.5x more likely to experience a security incident, according to the 2026 Infrastructure Identity Survey.
• Read next: Zero Trust for AI Agents

What this signals

Public benchmarking is pushing AI agent security away from vendor claims and toward repeatable proof. The real programme question is whether the boundary layer can stop exfiltration across the transports and encodings your agents actually use, not just whether it can catch a demo payload.

Runtime boundary governance: the next stage of AI agent security is less about static allowlists and more about whether policy can reason over request context, nested payloads, and protocol-specific exfiltration before data leaves the environment. That is where inline control and auditability have to converge.

The scale pressure is already visible in NHI governance data: 70% of organisations grant AI systems more access than they would give a human employee performing the exact same job. That gap makes egress control and delegated authority part of the same governance conversation, not separate ones.

For practitioners

• Validate inline containment on real agent transports Test the exact paths your agents use, including URLs, request bodies, headers, WebSockets, and nested tool calls, before trusting block results.
• Measure evidence quality separately from block rate Track whether the control produces machine-readable attribution for each blocked event, not just whether it stopped the traffic.
• Test recursive decoding and multipart parsing Run payloads that are base64-wrapped, double-encoded, or fragmented across multipart bodies to expose shallow inspection gaps.
• Map agent access to the egress boundary Treat outbound exfiltration paths as part of the agent authorisation model so runtime controls and identity decisions are aligned.

Key takeaways

• Public benchmark scores show that agent egress protection is measurable, but only if teams look beyond headline containment and test real payload paths.
• Detection quality matters as much as blocking, because a control that cannot explain what it stopped is hard to audit or improve.
• Encoded, fragmented, and nested payloads remain the pressure point for runtime enforcement, so shallow inspection is not a trustworthy production baseline.

Related NHI Mgmt Group resources: Zero Trust for AI Agents · AI Agent Observability, Audit and Incident Response Guide · AI Agent Authorisation Guide · Agentic AI Security Guide

Key terms

• Agent Egress Control: Agent egress control is the governance of where an AI agent can send data, which tools it can call, and what external destinations it may reach. It is essential when agents can transform internal information into outbound requests that may leak sensitive content or trigger unintended actions.
• Containment: The phase of incident response that stops an incident from spreading while preserving the evidence needed to investigate it. In cloud environments, containment often starts with identity revocation, isolation of workloads, and protection of logs before any system is terminated or cleaned up.
• Structured Evidence: Evidence stored in a consistent format that can be searched, filtered, and mapped to controls without rework. Structured evidence improves auditability because the same record can support review, reporting, and automation without being re-entered or reinterpreted in multiple systems.
• Runtime boundary: The point at which an identity's permissions, execution context, and observable behaviour are contained for policy purposes. In agentic systems, runtime boundaries matter because they determine whether the organisation can explain, audit, and limit what the agent did after execution begins.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.