Topic starter
18/08/2026 2:43 pm
TL;DR: Booking.com shows how AI observability now has to cover both traditional ML and agentic workflows, with traces, prediction logs, privacy-aware redaction, and cohort analysis used to reconstruct behaviour and explain regressions, according to Arize. The governance lesson is that telemetry is no longer just monitoring; it is part of making AI systems auditable, debuggable, and trustworthy at scale.
Editorial analysis by NHI Mgmt Group, based on content published by Arize: “How Booking.com scales AI observability with Arize”.
Key questions
Q: How should teams govern agentic AI when the model can act across multiple tools and services?
A: Teams should govern the full execution path, not just the model endpoint.
Q: Why do agentic workflows require stronger observability than traditional ML models?
A: Agentic systems make decisions through chains of actions, so failure can come from any step in the sequence.
Q: What signs show that AI observability is not capturing the right evidence?
A: Common signs include unexplained regressions, inconsistent outputs across cohorts, repeated debugging without root cause, and alerts that show a problem but not the workflow step that caused it.
Practitioner guidance
- Instrument end-to-end agent traces Capture prompts, retrieval calls, tool use, guardrails, latency, and fallback decisions so each AI interaction can be reconstructed without ambiguity.
- Separate agent telemetry from raw production logs Use structured traces for agentic workflows and prediction logs for traditional ML so each system is monitored with the right evidence model.
- Apply redaction before observability export Filter sensitive attributes and personal data in the telemetry pipeline before traces or logs are routed to downstream analytics or monitoring systems.
Bottom line: AI observability is becoming a runtime control because teams now need to explain how agents and models reached specific outcomes, not just whether they responded successfully.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
18/08/2026 3:27 pm
AI observability is now part of the control plane, not a reporting afterthought. Once agents can plan, retrieve, and call tools, the organisation needs evidence of what happened inside the workflow, not just whether the request succeeded. That shifts observability from metrics collection to governance over runtime decisions, especially where humans may still review the final action.
A few things that frame the scale:
- 7% of security leaders admit they do not know how often their AI systems are making autonomous changes to infrastructure, according to the 2026 Infrastructure Identity Survey.
A question worth separating out:
Q: How should security teams govern AI observability in enterprise environments?
A: Security teams should treat AI observability as a governance control, not a monitoring add-on. Focus on identity attribution, data lineage, output quality, and policy evidence so every meaningful AI action can be traced back to an owner, a model version, and an access decision. That makes investigations, reviews, and accountability possible.
👉 Read our full editorial: AI observability for agentic workflows is becoming an operational control