Notifications
Clear all
Topic starter
20/08/2026 6:05 pm
TL;DR: Production AI agents fail less from raw model quality than from missing execution controls, exploding context, and weak evaluation discipline, according to Arize. The deeper lesson is that agent governance now depends on code-enforced constraints, out-of-band state, and testable behaviour, not prompt instructions alone.
NHIMG editorial — based on content published by Arize: AI Agent Debugging: Four Lessons from Shipping Alyx to Production
Questions worth separating out
Q: What breaks when AI agent access decisions are handled in prompts?
A: Prompt-based access control is fragile because it places security logic inside the same system attackers are trying to influence.
Q: Why do AI agents need out-of-band state management?
A: Agents need out-of-band state because conversational history is fragile under long tool sequences and large outputs.
Q: How do you know if an AI agent is still following the right task?
A: You measure task fidelity, not just answer quality.
Practitioner guidance
- Enforce agent task completion in code Require agents to pass through a validation gate before they can finish a workflow, and reject completion when required tasks remain incomplete.
- Store agent plans outside conversation history Keep the authoritative plan on the server side and regenerate a compact current-state message on every turn.
- Use stable handles for large structured outputs Return a preview plus a stable object handle when tool output is too large for context, then provide query tools that let the agent pull only the slices it needs.
What's in the full article
Arize's full post covers the implementation detail this analysis intentionally leaves at a higher level:
- The exact todo planning pattern, including task statuses, finish gating, and how the agent is forced back into the work loop
- The LargeJson preview and query pattern for keeping structured experiment data out of the active context window
- The testing approach for golden traces, decision-point assertions, and end-to-end trajectory evaluation
- The debugging workflow that combines traces, backend spans, and logs to isolate root cause across agent and infrastructure layers
👉 Read Arize's account of shipping Alyx to production and the debugging lessons it exposed →
AI agent debugging in production: what control gaps are teams missing?
Explore further
20/08/2026 6:40 pm
Code-enforced control, not prompt persuasion, is the real reliability boundary for agents. The article shows that once an AI agent can chain tool calls, prompt text alone cannot reliably govern behaviour. In identity terms, the agent is operating like a non-human identity with execution authority, so the control plane has to enforce scope, not just describe it. The practitioner conclusion is clear: treat agent instructions as advisory and tool policy as authoritative.
A question worth separating out:
Q: What should organisations do before giving agents broader tool access?
A: They should validate the control logic with evals that test unsupported claims, unsafe actions, and failed tool calls. If a policy cannot maintain output quality under failure conditions, broader tool access only increases blast radius. Scope should expand only after the escalation path is proven and observable.
👉 Read our full editorial: AI agent debugging exposes the control gaps production teams miss
