Notifications
Clear all
Topic starter
25/06/2026 10:26 pm
TL;DR: AI observability must now track the data AI systems read, the outputs they produce, and the actions autonomous agents take in production, because model metrics alone miss downstream risk, according to Collibra. Access reviews assume privilege is stable enough to be observed, but agents can execute, chain tools, and change state faster than traditional governance cycles.
NHIMG editorial — based on content published by Collibra: AI Observability Explained: How to Monitor Models and Agents in Production From One Command Center
Questions worth separating out
Q: How should security teams govern AI agents that can take actions in production?
A: Security teams should govern action-capable AI agents like high-risk identities, not like passive models.
Q: Why do AI agents create more governance risk than model-only systems?
A: AI agents create more governance risk because they do things, not just predict things.
Q: What signals show that an AI system is drifting outside its approved use case?
A: The clearest signals are unexpected tool calls, access to data outside the approved scope, repeated retrieval of unneeded context, and actions that diverge from the documented workflow.
Practitioner guidance
- Define AI system ownership at registration Capture each model, agent, and use case with a named owner, risk tier, and approved purpose before it reaches production.
- Require decision traces for action-capable agents Log the tools used, retrieved context, decisions made, and the resulting action for any agent that can touch data or trigger workflows.
- Tie observability signals to policy posture Map runtime AI signals to the policies the system must satisfy, including data-access limits, delegated execution rules, and regulatory obligations.
What's in the full article
Collibra's full blog post covers the operational detail this post intentionally leaves for the source:
- How the AI Command Center maps signals to owners, policies, and use cases at registration time
- How code-first registration and traceability are implemented across production AI systems
- How the AI Trust Score is assembled and used for ongoing readiness review
- How operators can intervene, including pausing an agent when a signal breaks
👉 Read Collibra's analysis of AI observability for models and agents in production →
AI observability for agents and models: are your controls keeping up?
Explore further
25/06/2026 10:57 pm
AI observability is now an identity control problem, not just an engineering telemetry problem. The article correctly shows that agents emit actions, not only predictions, which changes what must be governed. Once tool use, data access, and delegated execution are part of the runtime, identity, entitlement, and ownership data become observability inputs. Practitioners should treat runtime AI evidence as part of access governance, not a separate monitoring layer.
A few things that frame the scale:
- 72% of organisations have experienced or suspect they have experienced a breach of non-human identities, with 46% confirmed and 26% suspected, according to The 2024 ESG Report: Managing Non-Human Identities.
- The average organisation believes more than 1 in 5 of their non-human identities are insufficiently secured, which shows how visibility gaps become governance gaps before an incident does.
A question worth separating out:
Q: How do observability and compliance fit together for AI systems?
A: Observability supports compliance by turning runtime behaviour into evidence. Regulators and auditors want to know what data the system touched, who owns it, what decision path it followed, and whether it stayed inside policy. Without that evidence chain, compliance becomes reconstruction after the fact rather than control during operation.
👉 Read our full editorial: AI observability for autonomous agents is now an IAM problem
