Topic starter
25/10/2026 11:08 am
TL;DR: Offloading TLS cryptography from Envoy to NVIDIA BlueField-3 roughly doubled single-core throughput on a 5 MB payload, from 6.6 Gbps to 13.6 Gbps, while keeping L7 policy unchanged, according to Tetrate. The result matters because encrypted AI traffic is shifting proxy bottlenecks from policy to per-byte crypto work.
Editorial analysis by NHI Mgmt Group, based on content published by Tetrate: “Doubling Envoy Throughput on NVIDIA BlueField”.
By the numbers:
- On a single core with AES-256 and a 5 MB payload, throughput went from 6.6 Gbps to 13.6 Gbps.
Key questions
Q: How should teams decide whether TLS offload belongs in the gateway path?
A: Use TLS offload when encryption is consuming the same CPU budget as routing and policy, especially on high-volume or large-payload traffic.
Q: Why does encrypted AI traffic change gateway sizing assumptions?
A: AI traffic often carries much larger payloads than legacy API calls, so the per-byte cost of symmetric cryptography becomes a dominant part of the proxy workload.
Q: What breaks when crypto and policy share the same CPU budget?
A: The first thing to suffer is control headroom.
Practitioner guidance
- Measure crypto cost separately from policy cost Profile TLS termination, routing, JWT validation, rate limiting, and body inspection as distinct contributors to proxy load before deciding where to optimise.
- Test hardware offload under real payload sizes Use large payloads that resemble model weights, embeddings, and long responses, because the benefit changes materially as encrypted bytes increase.
- Verify policy equivalence after acceleration Confirm that route rules, observability, and access controls behave identically when cipher work moves off host cores.
Bottom line: The article shows that TLS offload can materially improve gateway efficiency without removing Layer 7 policy enforcement.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →