Topic starter
18/08/2026 2:43 pm
TL;DR: AI observability at Booking.com is being used to reconstruct agent and model behaviour because traces, prediction logs, privacy-aware redaction, and cohort analysis each serve a different governance function, Arize says. Telemetry has to support explanation, not just status checks, if runtime decisions are going to be auditable.
Editorial analysis by NHI Mgmt Group, based on content published by Arize: “How Booking.com scales AI observability with Arize”.
Key questions
Q: Why do agentic workflows require stronger observability than traditional ML models?
A: Agentic systems make decisions through chains of actions, so failure can come from any step in the sequence.
Q: What signs show that AI observability is not capturing the right evidence?
A: Common signs include unexplained regressions, inconsistent outputs across cohorts, repeated debugging without root cause, and alerts that show a problem but not the workflow step that caused it.
Q: How should teams govern agentic AI when the model can act across multiple tools and services?
A: Teams should govern the full execution path, not just the model endpoint.
Practitioner guidance
- Instrument end-to-end agent traces Capture prompts, retrieval calls, tool use, guardrails, latency, and fallback decisions so each AI interaction can be reconstructed without ambiguity.
- Separate agent telemetry from raw production logs Use structured traces for agentic workflows and prediction logs for traditional ML so each system is monitored with the right evidence model.
- Apply redaction before observability export Filter sensitive attributes and personal data in the telemetry pipeline before traces or logs are routed to downstream analytics or monitoring systems.
Bottom line: AI observability is becoming a runtime control because teams now need to explain how agents and models reached specific outcomes, not just whether they responded successfully.
Glossary terms: AI observability, Agentic workflow, Prediction Log, Privacy-Aware Redaction
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 5 hours ago by NHI Mgmt Group
18/08/2026 3:27 pm
AI observability is now part of the control plane, not a reporting afterthought. Once agents can plan, retrieve, and call tools, the organisation needs evidence of what happened inside the workflow, not just whether the request succeeded. That shifts observability from metrics collection to governance over runtime decisions, especially where humans may still review the final action.
A few things that frame the scale:
- 7% of security leaders admit they do not know how often their AI systems are making autonomous changes to infrastructure, according to the 2026 Infrastructure Identity Survey.
A question worth separating out:
Q: How should security teams govern AI observability in enterprise environments?
A: Security teams should treat AI observability as a governance control, not a monitoring add-on. Focus on identity attribution, data lineage, output quality, and policy evidence so every meaningful AI action can be traced back to an owner, a model version, and an access decision. That makes investigations, reviews, and accountability possible.
👉 Read our full editorial: AI observability for agentic workflows is becoming an operational control
This post was modified 5 hours ago by NHI Mgmt Group