Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

OpenInference for AI agents: what it means for governance teams


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 18004
Topic starter  

TL;DR: Microsoft’s open trust stack links ASSERT evaluation, Agent Control Specification runtime controls, and observability through OpenInference, Arize’s OpenTelemetry-based trace contract for AI applications. The architectural shift matters because agent governance now depends on shared evidence across test, policy, and production rather than isolated prompts or framework-specific wrappers.

NHIMG editorial — based on content published by Arize: Microsoft’s open trust stack runs on OpenInference

Questions worth separating out

Q: What breaks when AI agent access decisions are handled in prompts?

A: Prompt-based access control is fragile because it places security logic inside the same system attackers are trying to influence.

Q: Why do shared traces matter for AI agent security teams?

A: Shared traces matter because they connect behaviour, control decisions, and production evidence in one record.

Q: How should organisations govern external tools used by AI agents?

A: Organisations should review external tools as security inputs, not convenience features.

Practitioner guidance

  • Define a trace contract for every agent Require every agent framework in scope to emit the same minimum telemetry fields for tool calls, routing decisions, policy outcomes, and state transitions.
  • Map policy checkpoints to agent lifecycle stages Place controls at input, model call, state, tool execution, and output so each decision can be reviewed independently.
  • Tie evaluation failure to runtime control changes Use regression testing to identify where an agent violated a spec, then update the runtime control at the same checkpoint and re-test against the same traces.

What's in the full article

Arize's full article covers the operational detail this post intentionally leaves for the source:

  • How ASSERT turns behavioural specifications into executable tests for agent regression testing.
  • How ACS applies controls at five agent checkpoints, including input, state, tool execution, and output.
  • How OpenInference spans flow into Phoenix and Arize AX for production observability.
  • How the reference implementation wraps tools with policy enforcement and logs each decision.

👉 Read Arize’s analysis of Microsoft’s open trust stack and OpenInference →

OpenInference for AI agents: what it means for governance teams?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 3 months ago
Posts: 17593
 

Shared trace contracts are becoming the practical foundation of agent governance. OpenInference matters less as a developer convenience than as a governance primitive because it creates evidence continuity across evaluation, runtime controls, and observability. That continuity is what allows security teams to inspect agent behaviour without relying on framework-specific instrumentation. In identity terms, this is how delegated actions become auditable instead of opaque. Practitioners should treat trace portability as a control requirement, not a logging preference.

A question worth separating out:

Q: What is the difference between agent evaluation and runtime control?

A: Evaluation tests whether the agent behaved as intended under specified conditions, while runtime control decides what the agent is allowed to do in live operation. Evaluation finds defects. Runtime control limits impact. Mature programmes need both, and they should share the same telemetry so results can be compared.

👉 Read our full editorial: Microsoft’s open trust stack ties agent controls to shared traces



   
ReplyQuote
Share: