Notifications
Clear all
Topic starter
18/08/2026 2:43 pm
TL;DR: Booking.com shows how AI observability now has to cover both traditional ML and agentic workflows, with traces, prediction logs, privacy-aware redaction, and cohort analysis used to reconstruct behaviour and explain regressions, according to Arize. The governance lesson is that telemetry is no longer just monitoring; it is part of making AI systems auditable, debuggable, and trustworthy at scale.
NHIMG editorial — based on content published by Arize: How Booking.com scales AI observability with Arize
Questions worth separating out
Q: How should organisations govern agentic AI in observability workflows?
A: Treat AI as a bounded operator, not an all-purpose controller.
Q: Why do agentic AI systems need different monitoring from traditional ML models?
A: Agentic systems create a sequence of decisions, not just a single prediction, so aggregate metrics often hide the root cause of a failure.
Q: What do organisations get wrong about AI observability?
A: They often confuse technical telemetry with governance evidence.
Practitioner guidance
- Instrument agent workflows end to end Capture prompts, model calls, retrieval steps, tool invocations, fallback decisions, and evaluation outputs in a single trace path so investigations can reconstruct the full workflow.
- Separate agent telemetry from raw application logs Build a dedicated telemetry pipeline for AI systems that samples records, redacts personal data, and routes only approved fields into observability and analytics tools.
- Govern context growth in multi-turn agents Set thresholds for token accumulation, retained conversation history, and long reference content so prompt bloat does not quietly degrade quality or increase cost.
What's in the full article
Arize's full case study covers the operational detail this post intentionally leaves for the source:
- How Booking.com instruments OpenTelemetry and OpenInference across agent and ML workflows in production
- The telemetry collector design for sampling, PII-aware redaction, and routing into observability systems
- Examples of latency regression detection and configuration-change diagnosis in a live agent workflow
- The multi-turn context bloat case study that shows how evaluation scores degrade as conversation state grows
👉 Read Arize's case study on Booking.com's AI observability architecture →
AI observability for agents and ML: what practitioners need to know?
Explore further
18/08/2026 3:27 pm
AI observability is becoming a governance control, not a debugging luxury. Once agents can chain prompts, retrieve content, and call tools, the organisation needs evidence of what happened at each step. Aggregate dashboards can show latency or failure rates, but they cannot explain whether a system stayed within policy. Practitioners should treat observability as part of AI control design, not post-incident forensics.
A question worth separating out:
Q: Who is accountable when vendor telemetry exposure reveals AI user identity data?
A: The primary organisation remains accountable for choosing the vendor, defining the data shared, and enforcing outbound access rules. Privacy, security, and identity teams should jointly own that decision because telemetry exposure is a governance failure, not just a vendor incident.
👉 Read our full editorial: AI observability for agentic workflows is becoming an operational control
