Notifications
Clear all
Topic starter
14/09/2026 10:09 am
TL;DR: Inline LLM judges are accurate but too expensive and slow to inspect every prompt, retrieval, and tool call at agent scale, according to Lasso Security, while its CPU-based LEAP architecture claims GPU-class detection with far higher throughput and lower latency. The key issue is not model quality, but whether security guardrails can be affordably run on every AI action without becoming the bottleneck.
NHIMG editorial — based on content published by Lasso Security: Introducing LEAP: CPU-based AI Security Guardrails With GPU-Class Accuracy
By the numbers:
- Lasso Security says a single GPU-backed SageMaker ml.g7.2xlarge instance costs roughly $2,300 per month before storage, networking, and redundancy.
- Lasso Security says Sheriff achieved 1,444 documents per second with about 1 millisecond latency in its benchmark.
Questions worth separating out
Q: How should security teams reduce the cost of inline AI guardrails without losing detection quality?
A: Use a layered model. Put a low-cost first-line classifier in front of every prompt, retrieval, response, and tool call, then reserve heavier reasoning models for genuinely ambiguous cases. That keeps universal inspection affordable while preserving the option to escalate when context, intent, or tool use cannot be resolved cheaply.
Q: Why do guardrails fail to secure agentic AI workflows?
A: Guardrails fail because they are probabilistic and operate on model output, while the risk lives in the execution chain.
Q: What are the signs that an AI security control is not scaling well enough?
A: Watch for rising latency, growing inference spend, queueing, and a widening gap between the number of agent actions and the number of actions you can inspect.
Practitioner guidance
- Design for full-trace inspection Map every prompt, retrieval, response, and tool call that can trigger enforcement, then decide which events require universal inspection and which can be sampled.
- Separate first-line filtering from hard-case reasoning Use a cheap classifier or CPU-based guardrail for routine decisions, and escalate only genuinely ambiguous cases to a reasoning model.
- Measure cost per inspected decision Track the direct cost, latency, and infrastructure load for each security decision so the guardrail is evaluated as a control, not a feature.
What's in the full article
Lasso Security's full research covers the operational detail this post intentionally leaves for the source:
- Benchmark tables comparing Sheriff with 11 other guard models across detection, over-defense, throughput, and latency
- Implementation notes on the two-tier inspection path that escalates only uncertain cases to a reasoning model
- Production design detail on running inline enforcement inside a customer-managed VPC or air-gapped environment
- The full explanation of how LEAP handles long traces without a fixed context-window cap
👉 Read Lasso Security's analysis of CPU-based AI security guardrails and LEAP →
CPU-based AI guardrails: can they keep up with agentic traffic?
Explore further
14/09/2026 10:10 am
Cost-per-decision is becoming a core security constraint for AI programmes. The article shows that detection quality alone is no longer enough if the control cannot be run at production scale. In AI security, the effective control is the one that can inspect every request, every retrieved document, and every tool call without becoming financially or operationally unsustainable. Practitioners should treat unit cost per decision as a control metric, not just a procurement line item.
A question worth separating out:
Q: How do organisations decide when to keep a reasoning model in the guardrail path?
A: Keep it where the decision is genuinely ambiguous, especially when context spans multiple turns, retrieved content, and tool permissions. If a simpler classifier can reliably resolve the common cases, let the reasoning model handle only the exceptions that need deeper context.
👉 Read our full editorial: CPU-based AI guardrails may outscale GPU judges in production
