AI governance programmes routinely fund a posture tool and assume they now know how their AI behaves, or fund an observability platform and assume they now know how their AI is configured, and neither assumption survives the first audit question. AI observability records what AI systems actually do while they run: traces of model calls, retrieval, tool execution, tokens, latency, evaluation scores and who made each request. AI security posture management (AI-SPM) records how AI assets are built, exposed and permissioned. This guide compares the two as evidence sources: what each sees, who owns it, where each fails, and how their outputs combine into the inventory, monitoring and incident records that AI governance frameworks ask for. It is written for AI governance leads, security architects, platform and SRE teams and risk owners. The split between posture and inline protection (guardrails and prompt filtering) is covered in AI-SPM vs AI Runtime Security and is not repeated here.
Key takeaways
- Observability is behaviour, posture is configuration. Observability answers "what did this model, agent or tool call do, for whom, and at what cost"; AI-SPM answers "which AI assets exist, how are they exposed and which identities can reach them".
- Observability is not a control. Traces and metrics do not block anything; they explain, alert and prove. Treat observability as the evidence layer that runtime controls, incident response and governance reviews depend on.
- Each one's blind spot is the other's strength. AI-SPM cannot see a correctly configured endpoint being abused through content; observability cannot see an endpoint that was never instrumented, which is exactly the asset AI-SPM is most likely to find.
- The telemetry store becomes a sensitive asset. Prompt and response capture is opt-in under the OpenTelemetry GenAI conventions because the content is likely to contain personal data, so the observability backend itself needs posture checks, access control and retention rules.
- Governance needs both, joined on one inventory. NIST AI RMF GOVERN 1.6 (inventory) is fed mainly by posture, MEASURE 2.4 and MANAGE 4.1 (production monitoring) mainly by observability; the join key is a shared AI asset identifier.
Short answer
Choose AI observability when the question is about behaviour: an agent looped and spent a month's token budget, a retrieval step returned the wrong documents, an answer leaked customer data, or an auditor wants to see what the system was given before it made a decision. Choose AI-SPM when the question is about the estate: which models and endpoints are running, which are public, which still accept API keys, which managed identity can read the training bucket, and which agents are wired to which tools. In practice an organisation that runs AI in production needs both, and the design decision is less "which one" than "who owns each, how do they share an asset identifier, and which governance records does each feed".
Side-by-side comparison
| Dimension | AI observability (runtime telemetry) | AI-SPM (posture) |
|---|---|---|
| Core question | What did the AI system do, for whom, how well and at what cost? | What AI assets exist, how are they configured and how exposed are they? |
| Unit of evidence | Trace, span, metric, event per request (model call, retrieval, tool call, agent step) | Resource, setting, identity, network path, finding, attack path |
| Data sources | Application and framework instrumentation, AI gateways, provider invocation logs, evaluation jobs | Cloud control-plane APIs, infrastructure as code, container images, identity graphs, data classification |
| Timing | Continuous, per request, near real time | Scheduled scans and change-triggered checks |
| Typical signals | Token usage per caller, latency, finish reasons, tool arguments, evaluation scores, guardrail verdicts, errors | Public endpoint, key authentication enabled, over-broad identity, unencrypted data source, vulnerable AI library, unregistered model |
| Sees content? | Yes, if content capture is opted in | No |
| Sees uninstrumented assets? | No | Yes, within connected accounts |
| Primary owner | AI engineering, platform or SRE, with security consuming signals | Cloud security or posture team |
| Main data risk | Telemetry holds prompts, outputs and personal data | Findings reveal attack paths and must be access controlled |
| Governance output | Production monitoring records, incident timelines, quality and drift evidence | AI inventory, configuration baseline evidence, exposure trends |
AI observability in depth
Threat model: what it helps you catch
Observability does not assume an attacker path; it assumes that something will go wrong and you will need to explain it. The security-relevant cases are the ones that look like normal traffic to infrastructure monitoring:
- Stolen or leaked model credentials: a key used from an unexpected principal or at unusual volume, mapped in MITRE ATLAS to AML.T0055 Unsecured Credentials followed by AML.T0040 AI Model Inference API Access. Token usage grouped by caller identity is usually the first signal.
- Cost harvesting and denial of wallet: ATLAS AML.T0034 Cost Harvesting and OWASP LLM10:2025 Unbounded Consumption show up as token and request spikes long before an invoice does.
- Prompt injection after the fact: where a guardrail missed an injection (OWASP LLM01:2025, ATLAS AML.T0051), the trace shows the retrieved document, the unexpected tool call and the data that left. Observability turns a suspicion into a timeline.
- Excessive agency: an agent calling tools it rarely uses, with arguments outside its normal range (OWASP LLM06:2025). Tool spans with arguments make this measurable.
- Quality and safety drift: a model version change or a prompt edit that degrades answers or raises refusal rates. Continuous evaluation scores attached to traces show when and after which change.
Architecture in words
Instrumentation in the application, the agent framework or an AI gateway emits spans for each operation. The OpenTelemetry GenAI semantic conventions, which now live in the dedicated open-telemetry/semantic-conventions-genai repository and are marked Development, define operation names such as chat, embeddings, retrieval, execute_tool and invoke_agent, with attributes such as gen_ai.operation.name, gen_ai.provider.name and gen_ai.response.finish_reasons. Spans flow through a collector, where redaction and sampling happen, into a tracing backend. Evaluation jobs read stored traces and attach scores. Provider-side logs add a second, independent record: Amazon Bedrock model invocation logging, which is disabled by default, captures request and response bodies, token counts and the caller's identity.arn to CloudWatch Logs or Amazon S3. Microsoft Foundry monitoring uses Azure Monitor Application Insights for token consumption, latency, exceptions and response quality, with alert rules built from the underlying KQL. Security teams consume a filtered stream in the SIEM.
Where it fails
- Instrumentation gaps: a team that calls a model API directly, or a model served from a GPU virtual machine outside the platform, produces no traces. The dashboard looks healthy because the unhealthy part is invisible.
- Content capture by accident: the conventions say instrumentations should not capture instructions, inputs and outputs by default, and warn that output messages are "likely to contain sensitive information including user/PII data". Teams that switch capture on for debugging and forget to switch it off build a shadow copy of their most sensitive data.
- Convention drift: because the GenAI conventions are still in Development, attribute names change between releases. Unpinned instrumentation breaks detections and dashboards silently.
- Telemetry without ownership: engineering collects traces for debugging, security assumes they are monitored, and nobody owns alerting on misuse.
- Sampling away the evidence: head sampling at 1 percent is fine for latency trends and useless for an incident where you need the one request that leaked data.
Controls it drives
Turn on provider invocation logging in every account and region that hosts models; standardise on OpenTelemetry GenAI conventions and pin the version; capture content only where justified, preferably by storing it externally and recording references, which the conventions recommend for production; redact at the collector before export; tail-sample to keep every errored, guardrail-flagged or high-cost trace; set retention for content shorter than for metadata; and route token, tool and identity anomalies to the SOC. For the full instrumentation pattern, see the LLM Observability Guide, and for agents and kill switches the AI Agent Observability and Incident Response Guide.
AI-SPM through an observability lens
The posture half is covered in depth in the AI Security Posture Management (AI-SPM) Guide. What matters for this comparison is what AI-SPM gives observability and governance that telemetry cannot.
- The asset list to instrument against: Microsoft describes Defender for Cloud discovering "the generative AI Bill of Materials (AI BOM), which includes application components, data, and AI artifacts from code to cloud", across Azure OpenAI, Azure AI Foundry, Azure Machine Learning, Amazon Bedrock and Google Vertex AI. Google's Security Command Center AI Protection discovers declarative assets, inferred assets (compute and storage processing AI data) and Model-as-a-Service usage seen only through API calls. Comparing that inventory against the services emitting traces is the single best test of observability coverage.
- Exposure context for alerts: a token spike on a private, keyless endpoint and the same spike on a public endpoint with key authentication enabled are different incidents. Posture findings tell the analyst which one they are looking at.
- Posture of the observability stack itself: the tracing backend, the S3 bucket receiving invocation logs and the evaluation store are data stores like any other. AI-SPM, CSPM or DSPM checks should cover public access, encryption and who can read them.
- What it cannot give: no posture tool sees what a model said, how much a given caller consumed, or whether answers got worse after a prompt change.
Ownership and operating model
Most failures in this area are organisational. Observability is usually built by the teams that ship AI features, for debugging and cost, while AI-SPM is usually bought by cloud security as an extension of CSPM. Neither group naturally owns the question "is our AI behaving acceptably and do we have the evidence".
| Activity | AI engineering / platform | Cloud security (posture) | SOC | AI governance / risk |
|---|---|---|---|---|
| Instrumentation standard and convention version | Owns | Consulted | Consulted | Informed |
| Content capture and redaction decisions | Implements | Consulted | Consulted | Approves (with privacy) |
| AI asset discovery and inventory reconciliation | Consulted | Owns | Informed | Uses for register |
| Misuse detections on telemetry | Supplies fields | Informed | Owns | Informed |
| Quality, safety and drift evaluations | Owns | Not involved | Not involved | Sets thresholds |
| Posture of telemetry stores | Remediates | Owns checks | Informed | Informed |
| Governance reporting | Supplies metrics | Supplies metrics | Supplies incidents | Owns |
Small organisations often have one platform team holding all four columns; the table still helps, because it names the decisions that must be made explicitly. Large organisations should give the AI governance function the right to require instrumentation as a condition of production approval, in the same way that posture onboarding is a condition of a new cloud account. See Centralised vs Federated AI Governance for how that authority is usually split.
How each feeds AI governance
Governance frameworks ask for records, not tools. The table maps the main requirements to the evidence each discipline produces. NIST AI RMF wording is from the AI RMF Core; EU AI Act references are to the provider obligations for high-risk AI systems.
| Requirement | What it asks for | Main evidence source | Supporting source |
|---|---|---|---|
| NIST AI RMF GOVERN 1.6 | Mechanisms to inventory AI systems, resourced by risk priority | AI-SPM discovery and AI BOM | Observability: services emitting GenAI telemetry reveal unregistered usage |
| NIST AI RMF MEASURE 2.4 | Functionality and behaviour of the AI system and its components monitored in production | Observability traces, metrics and evaluations | AI-SPM: component list defines what "its components" means |
| NIST AI RMF MEASURE 2.7 | AI system security and resilience evaluated and documented | AI-SPM findings and attack paths | Observability: detections and incident records |
| NIST AI RMF MANAGE 4.1 | Post-deployment monitoring plans implemented, including capturing and evaluating input | Observability with content capture policy | AI-SPM: change detection on deployed configuration |
| EU AI Act Article 12 | High-risk systems technically allow automatic recording of events (logs) over their lifetime, for traceability | Observability and provider logs | AI-SPM: confirms logging is enabled on each asset |
| EU AI Act Article 72 | Providers establish a documented post-market monitoring system that collects and analyses performance data | Observability and evaluation data | AI-SPM: inventory scopes which systems are covered |
The practical consequence: an AI register that is maintained by hand drifts within weeks. Feed it from AI-SPM discovery, mark each entry with whether it emits telemetry and whether invocation logging is on, and let the governance review read exceptions rather than the whole list. See the AI Register and AI Inventory Template and the AI Governance Guide.
Decision criteria
- Start with AI-SPM if: you cannot list the models, endpoints and agents running in your cloud accounts; teams self-serve model deployments; you have self-hosted models on general compute; or audit has asked for an AI inventory you cannot produce.
- Start with observability if: AI runs mostly through one or two managed platforms whose inventory you already know; you operate customer-facing assistants or agents with tools; cost overruns or quality complaints are the current pain; or you will need production monitoring evidence for a high-risk use case.
- Buy versus build: posture for AI is usually a module of an existing CSPM or CNAPP, so check what your current platform covers before buying. Observability is more often built on OpenTelemetry with a backend of choice; the AI Agent and LLM Observability Buyer's Guide covers selection.
- Regulated decisions: where AI outputs affect people (credit, hiring, health), observability with retained inputs is closer to a legal record than a debugging aid; design retention and access with privacy and legal teams first. See the AI and GDPR Guide.
- Small organisations: provider invocation logging plus the cloud provider's built-in AI posture checks cover a large share of the need at low cost. Add a dedicated tracing backend when you run agents or retrieval pipelines you built yourself.
Worked scenario
A retailer runs a customer service assistant on a managed model platform, with a retrieval index over order data and two tools: order lookup and refund request. On a Monday the finance team reports that model spend tripled over the weekend.
- Observability first: token usage grouped by caller identity shows that 90 percent of the extra spend came from one service principal that normally handles a nightly batch summarisation job, calling a large model at three times its usual rate.
- Posture context: the AI-SPM inventory shows that this principal holds an API key created eight months ago, that key authentication is still enabled on the endpoint, and that the endpoint is reachable from the internet. Neither fact appeared in the telemetry.
- Containment: the platform team revokes the key and switches the batch job to workload identity. Spend returns to normal within the hour, which confirms the key was the path.
- Scope: traces show the abusive calls carried no retrieval or tool spans, so customer data was not queried through the assistant. Without tool spans this question could not have been answered.
- Governance: the incident record links the trace evidence and the posture finding. The posture team adds a policy that blocks key-based authentication on AI endpoints; the AI register entry gains an owner and a review date. See the LLMjacking Guide for the wider pattern.
Implementation approach
- Reconcile inventory with telemetry (weeks 1 to 3): run AI discovery across all connected accounts, then list which assets emit GenAI telemetry or have provider invocation logging on. The gap list is your first backlog.
- Turn on provider logs (weeks 2 to 4): enable invocation logging per account and region with metadata first, content only where approved, into encrypted, access-controlled destinations with defined retention.
- Standardise instrumentation (weeks 3 to 8): adopt OpenTelemetry GenAI conventions, pin the version, require spans for retrieval and tool execution, and route through a collector that redacts and tail-samples. Pilot on one application before mandating.
- Wire detections (weeks 6 to 10): send token, tool and identity fields to the SIEM; start with per-caller token anomalies, new caller identities on a model, and tool calls outside an agent's allow list. Tune in alert-only mode for two weeks.
- Join the evidence (weeks 8 to 12): add the AI asset identifier from the inventory to telemetry resource attributes so findings, traces and register entries join on one key.
- Govern and roll back: make instrumentation and posture onboarding release gates. If content capture causes privacy concerns, drop to metadata-only by changing the capture setting, which the conventions default to no content, without removing traces.
Detection and measurement
| Metric | Source | Good direction |
|---|---|---|
| Share of inventoried AI assets emitting telemetry or invocation logs | AI-SPM inventory joined to telemetry | Towards 100 percent for production |
| Share of production traces with retrieval and tool spans present | Tracing backend | Up |
| Time from token anomaly to identified caller | SOC case records | Down |
| Share of AI endpoints private and keyless | AI-SPM | Up |
| Telemetry stores with content capture and no approved retention rule | Posture checks on observability stack | Zero |
| AI register entries reconciled automatically in the last 30 days | Register and inventory | Up |
Common failure patterns
- Calling a dashboard governance: an observability dashboard nobody reviews against thresholds is not post-deployment monitoring. Assign thresholds and owners.
- Inventory from one source: registering only what posture discovers misses Model-as-a-Service usage from code; registering only what emits telemetry misses everything uninstrumented. Use both.
- Debug-level content capture in production: prompts and outputs land in a backend with broad engineering access and long retention, creating a second copy of customer data outside data protection controls.
- No identity on spans: traces without the calling principal cannot attribute misuse. Bedrock captures identity.arn automatically; self-built instrumentation must add it.
- Posture findings with no runtime context: hundreds of medium findings are triaged by severity alone, while the one endpoint carrying unusual traffic waits. Feed usage data into prioritisation.
Framework mapping
| Framework | AI observability | AI-SPM |
|---|---|---|
| NIST CSF 2.0 | PR.PS-04 log records generated and available; DE.CM-09 runtime environments monitored; DE.AE-03 correlation | ID.AM-02 inventories of software, services and systems |
| NIST SP 800-53 Rev. 5 | AU-2 Event Logging, AU-12 Audit Record Generation, AU-11 Audit Record Retention, SI-4 System Monitoring | CM-8 System Component Inventory, CM-6 Configuration Settings, RA-5 Vulnerability Monitoring and Scanning, AC-6 Least Privilege |
| Both | CA-7 Continuous Monitoring; SI-12 Information Management and Retention for telemetry content | |
| NIST AI RMF | MEASURE 2.4, MANAGE 4.1 | GOVERN 1.6, MEASURE 2.7 |
| MITRE ATLAS | AML.T0034 Cost Harvesting, AML.T0040 AI Model Inference API Access, AML.T0051 LLM Prompt Injection (detected after the event) | AML.T0055 Unsecured Credentials (prevented by removing keys) |
| OWASP Top 10 for LLM Applications 2025 | LLM06 Excessive Agency, LLM10 Unbounded Consumption | LLM03 Supply Chain |
Practitioner checklist
- Reconcile the AI-SPM inventory against services emitting GenAI telemetry and close the gaps for production assets.
- Enable provider invocation logging in every account and region that hosts models, metadata first.
- Adopt OpenTelemetry GenAI conventions, pin the version and review the changelog before upgrades.
- Require spans for retrieval and tool execution, with the calling identity on every span.
- Write a content capture policy: who approves it, where content is stored, redaction and retention.
- Run posture checks on the observability stack itself: tracing backend, log buckets and evaluation stores.
- Send per-caller token, new-caller and tool-anomaly signals to the SOC with named owners.
- Put a shared AI asset identifier on telemetry, findings and register entries.
- Map evidence to NIST AI RMF GOVERN 1.6, MEASURE 2.4 and MANAGE 4.1, and to EU AI Act Articles 12 and 72 where they apply.
Standards and references
- NIST: AI Risk Management Framework Core
- OpenTelemetry: GenAI Semantic Conventions
- European Commission AI Act Service Desk: Article 12 Record-keeping
- European Commission AI Act Service Desk: Article 72 Post-market monitoring
- NIST: SP 800-53 Rev. 5
- AWS: Amazon Bedrock model invocation logging
- Google Cloud: AI Protection overview
- Microsoft Learn: AI security posture management in Defender for Cloud
Related NHI Mgmt Group resources: AI-SPM vs AI Runtime Security, LLM Observability Guide, AI Security Posture Management (AI-SPM) Guide, Agent Observability vs Audit Logging, AI Governance Guide.