Agent egress benchmarking: what the public scoreboard reveals

TL;DR: Its public agent-egress benchmark has grown to 197 logical cases across 18 categories, according to PipeLab, and the current Pipelock run reports 96.2% containment on applicable cases but exposes uneven performance in body and header scanning as well as missing structured evidence.

Editorial analysis by NHI Mgmt Group, based on content published by PipeLab: “I published my benchmark scores. Your turn.”.

By the numbers:

• The benchmark has grown to 197 logical cases across 18 categories.
• The original published Pipelock v2.1.2 run reported 96.2% containment on applicable cases.
• The run covered 142 of 151 applicable cases.

Key questions

Q: What breaks when AI agent egress controls only inspect obvious payloads?

A: They miss the cases that matter most in production, including encoded bodies, fragmented requests, and nested tool arguments.

Q: Why do containment and detection need to be measured separately for AI agent controls?

A: Because a tool can block traffic without proving what it blocked or why.

Q: How should security teams set access boundaries for AI agents?

A: Security teams should define agent boundaries around data sources, tools, and allowed actions before any production rollout.

Practitioner guidance

• Validate inline containment on real agent transports Test the exact paths your agents use, including URLs, request bodies, headers, WebSockets, and nested tool calls, before trusting block results.
• Measure evidence quality separately from block rate Track whether the control produces machine-readable attribution for each blocked event, not just whether it stopped the traffic.
• Test recursive decoding and multipart parsing Run payloads that are base64-wrapped, double-encoded, or fragmented across multipart bodies to expose shallow inspection gaps.

Bottom line: Public benchmark scores show that agent egress protection is measurable, but only if teams look beyond headline containment and test real payload paths.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →