Notifications
Clear all
Topic starter
20/08/2026 5:34 pm
TL;DR: MCP servers are emerging as a black box inside Kubernetes because many of them expose no metrics, structured logs, or traceable signals, according to Stacklok. That leaves observability stacks blind to AI tool-use paths, slows incident triage, and turns telemetry gaps into governance gaps for NHI and agentic AI operations.
NHIMG editorial — based on content published by Stacklok: Blog Insights on the next observability gap for Kubernetes and MCP servers
By the numbers:
- Only 18% of MCP server deployments implement any form of access scoping for tool permissions.
- 53% of MCP servers expose credentials through hard-coded values in configuration files.
- 70% of organisations grant AI systems more access than they would give a human employee performing the exact same job.
Questions worth separating out
Q: What breaks when MCP servers do not expose telemetry in Kubernetes?
A: Teams lose the ability to trace tool use, identify which service handled a request, and distinguish normal automation from risky behaviour.
Q: Why do MCP servers create governance problems for AI workloads?
A: Because they mediate tool access for AI systems while often leaving little trace of what happened inside the interaction.
Q: How do security teams decide whether MCP observability is enough?
A: MCP observability is enough only when it produces actionable evidence for ownership, scope, and session-level behaviour.
Practitioner guidance
- Instrument MCP servers before production rollout Require each MCP server to expose metrics, structured logs, and trace context as part of the deployment gate.
- Map MCP tool access to identity governance controls Classify MCP servers as privileged broker points in your NHI and agentic AI inventory, then assign owners, approval paths, and audit requirements to each tool permission set.
- Tie observability alerts to access review triggers When telemetry shows unusual tool invocation patterns, route the event into access review and incident triage workflows so teams can determine whether the behaviour reflects misconfiguration, misuse, or compromise.
What's in the full article
Stacklok's full blog post covers the operational detail this post intentionally leaves for the source:
- Prometheus scrape and service-discovery patterns for Kubernetes workloads that emit metrics
- OpenTelemetry Collector deployment models for correlating metrics, logs, and traces across clusters
- Practical examples of how MCP servers remain invisible when they do not expose structured telemetry
- The planned ToolHive approach for surfacing MCP tool-usage data in existing monitoring stacks
👉 Read Stacklok's analysis of the MCP observability gap in Kubernetes →
MCP servers in Kubernetes: what observability teams are missing?
Explore further
20/08/2026 6:20 pm
Observability gaps are now governance gaps when AI systems act through MCP servers. The article shows that telemetry is not just an operations concern. When tool mediation happens inside infrastructure that does not expose metrics, logs, or traces, security teams lose the evidence needed to govern non-human behaviour. That makes MCP visibility a control-plane requirement for NHI and agentic AI programmes, not a nice-to-have for SRE teams.
A question worth separating out:
Q: Who should be accountable for MCP server permissions and agent actions?
A: Accountability should sit with the business or platform owner that approved the data path, plus the security team that set the guardrails. If the agent can access regulated data or take sensitive actions, the organisation needs a clear owner for the server, the credentials, and the downstream consequences of misuse.
👉 Read our full editorial: MCP servers expose the next observability gap in Kubernetes
