Topic starter
19/08/2026 3:28 pm
TL;DR: Production AI agents need different harnesses, eval plans, rollout controls, and failure management because customer-facing, internal enterprise, and developer platform patterns fail in different ways, according to Arize. Governance has to match the agent’s pattern, or bounded execution and observability will not hold under production load.
Editorial analysis by NHI Mgmt Group, based on content published by Arize: “3 production patterns for AI agents and how to evaluate each one”.
Key questions
Q: Why do AI agents create access risk even when the model is accurate most of the time?
A: Because the risk is not only incorrect reasoning, it is incorrect action.
Q: What are the signs that an AI agent harness is not working?
A: Look for rising token cost, tool calls that surprise the team, traces that are hard to interpret, and failures that only appear after rollout.
Q: How should security teams govern production AI agents by deployment pattern?
A: Start by classifying the agent as customer-facing, internal enterprise, or developer platform, then set permissions, traces, and rollout gates to match that pattern.
Practitioner guidance
- Define the production pattern first Classify each agent as customer-facing, internal enterprise, or developer platform before you write controls, because the rollout and failure model changes with the pattern.
- Constrain context to verified sources Limit each agent to account-scoped data, policy documents, and workflow state that can be inspected in traces, rather than letting it infer missing context.
- Treat permissions as runtime identity Give the agent only the tools and data paths needed for the task, and review those permissions as if they belonged to a non-human identity.
Bottom line: Production AI agents need different controls because a customer assistant, an internal workflow agent, and a developer-platform agent fail in different ways.
Glossary terms: Production Agent Pattern, Harness, Trace-Level Evaluation, Context Engineering
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 9 hours ago by NHI Mgmt Group
19/08/2026 4:12 pm
Production AI agents are becoming governance objects, not just software features. The article’s core insight is that the decisive unit is the production pattern, because customer-facing, internal enterprise, and developer-platform agents carry different trust assumptions. That means AI governance has to attach to context, permissions, and runtime boundaries rather than to the model class alone. Practitioners should treat the deployment pattern as a control boundary, not a product taxonomy.
A few things that frame the scale:
- 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
A question worth separating out:
Q: What is the difference between evals and observability in AI operations?
A: Evals test anticipated behaviour before release. Observability shows what the system actually did under real conditions after release. Teams need both because evals are bounded by what they expected, while observability reveals the failures, edge cases, and unintended effects that only appear in production.
👉 Read our full editorial: Production AI agent patterns expose new governance and eval gaps