AI observability gaps: what security teams can actually govern

TL;DR: AI observability splits into five different questions, but most organisations only see partial token, cost, user, API, or agent activity data, according to DataBahn. Watching can support accounting and after-the-fact review, yet governing requires request-path control, not dashboard visibility.

NHIMG editorial — based on content published by DataBahn: The Real Challenge of AI Observability

Questions worth separating out

Q: How should security teams govern AI use when users, APIs, and agents all generate different telemetry?

A: Start by separating the governance problem into distinct control domains.

Q: Why does AI observability often fail to reduce risk even when telemetry is available?

A: Because observability shows activity after it has already occurred, while risk reduction requires control before the action completes.

Q: How should organizations approach the governance of AI agents?

A: Organizations should adopt a governance framework that incorporates continuous visibility, adaptive IAM practices, and stringent policy-based controls.

Practitioner guidance

• Define the five AI visibility domains Separate token usage, cost, user activity, API usage, and agent usage into distinct control objectives, then assign owners and evidence sources for each.
• Move policy into the request path Enforce controls where prompts, API calls, and agent actions are initiated so policy can apply before completion.
• Inventory all AI access paths Map direct provider accounts, gateways, console usage, and embedded API integrations so no usage path is treated as invisible.

What's in the full article

DataBahn's full whitepaper covers the operational detail this post intentionally leaves for the source:

• A breakdown of where token, cost, user, API, and agent telemetry each actually live in the enterprise stack.
• The distinction between monitoring data after the fact and placing controls in the request path before completion.
• Implementation detail on how to map visibility gaps to the right source systems, control points, and owners.
• The full telemetry model for deciding which AI actions can be observed, enforced, or only audited after the event.

👉 Read DataBahn's whitepaper on AI observability and governance gaps →

AI observability gaps: what security teams can actually govern?

Explore further

View Full Forum →  |  NHI Foundation Course →