Notifications
Clear all
Topic starter
19/08/2026 3:32 pm
TL;DR: Teams often misdiagnose Kubernetes as the bottleneck in AI agent sandboxes because they trace model calls but not sandbox creation, tool execution, filesystem I/O, permission failures, and eval latency, according to Arize. The operational question is whether runtime infrastructure, model latency, or harness design is actually slowing agent trajectories.
NHIMG editorial — based on content published by Arize: Trace before you migrate: Measuring Kubernetes bottlenecks in AI agent sandboxes
Questions worth separating out
Q: How should security teams trace AI agent sandboxes before changing runtime infrastructure?
A: Trace the full sandbox lifecycle, not only the model call.
Q: When does Kubernetes become the wrong runtime for AI agent execution?
A: Kubernetes becomes a poor fit when the workload depends on very fast startup, short-lived isolation, local state, and repeated dependency setup for sub-minute runs.
Q: What do teams get wrong when they treat sandbox failures as model failures?
A: They collapse environment problems into AI quality problems.
Practitioner guidance
- Trace sandbox lifecycle spans Instrument sandbox creation, readiness, tool execution, filesystem I/O, and teardown as separate spans so you can see where agent time is actually spent.
- Separate harness latency from model latency Track the time from agent request to first successful tool call, then compare it with token generation and eval write-back time.
- Audit sandbox permissions as runtime identity controls Review which credentials, files, and outbound network paths each sandbox can reach, and treat permission failures as governance signals rather than generic runtime noise.
What's in the full article
Arize's full article covers the operational detail this post intentionally leaves for the source:
- Span-by-span tracing guidance for sandbox creation, readiness, execution, and teardown.
- Concrete examples of how to distinguish model latency from runtime latency in real trajectories.
- How Arize AX and Phoenix map runtime events into trace attributes for evaluation workflows.
- Decision criteria for comparing Kubernetes against purpose-built sandbox schedulers.
👉 Read Arize's analysis of AI agent sandbox bottlenecks on Kubernetes →
AI agent sandboxes on Kubernetes: what bottlenecks should teams trace?
Explore further
19/08/2026 4:14 pm
Runtime observability is now an identity governance problem, not only a performance problem. When an AI agent can request files, launch shells, and retry commands across ephemeral environments, the execution layer becomes part of the control plane for access. That means the questions are not only about latency but also about who or what was allowed to execute, where credentials were exposed, and whether the environment was truly isolated. NHI governance has to account for the sandbox as a runtime identity boundary, not just a compute target.
A question worth separating out:
Q: How should organisations decide whether to keep Kubernetes or move to purpose-built sandboxes?
A: Base the decision on representative traces, not assumptions. If provisioning, filesystem latency, and retry overhead dominate the trajectory, a purpose-built sandbox may fit better. If model latency, tool selection, or eval design dominate, changing the runtime will not fix the problem and may only add migration risk.
👉 Read our full editorial: Tracing AI agent sandboxes before replacing Kubernetes
