Runtime observability for MCP servers exposes the security gap

At a glance

What this is: This is a runtime security guide for MCP servers that says traditional observability sees performance, not abuse, and that effective detection depends on correlating protocol, execution, and agent behaviour.

Why it matters: It matters because MCP deployments create a new security blind spot where tools can read sensitive files, reach external endpoints, or alter agent behaviour without conventional logs proving what happened.

Context

Model Context Protocol creates a security problem that looks familiar at the dashboard level but behaves differently at runtime. Standard observability can show latency, errors, and request volume, yet it does not tell defenders which files a tool touched, which endpoints it contacted, or whether the agent’s tool chain changed in ways that signal abuse.

The governance gap is the split between protocol telemetry, operating-system execution, and agent decision-making. In MCP environments, those layers are often managed separately, which makes security decisions dependent on partial evidence instead of complete attack context. That is why runtime correlation, not isolated logging, becomes the meaningful control boundary.

Key questions

Q: What breaks when MCP security relies only on protocol logs?

A: Protocol logs prove that an MCP request occurred, but they do not show which files were read, which endpoints were contacted, or whether the tool process did anything harmful after invocation. That leaves defenders with evidence of access, not evidence of safe execution. In practice, attackers exploit that gap by keeping protocol messages normal while abusing the runtime layer.

Q: Why does runtime observability matter more when MCP tools may be compromised?

A: When the tool itself is part of the threat model, application-level telemetry can be stripped, falsified, or selectively suppressed. Runtime observability matters because kernel-level or host-level signals are harder for the workload to manipulate and can reveal the actual side effects of the invocation. That changes detection from trust-based reporting to independent verification.

Q: How do security teams know if MCP runtime monitoring is actually working?

A: They should test whether one investigation can reconstruct the full chain from protocol request to host behaviour to agent decision. If the team still needs separate tools to explain file access, network egress, and tool sequencing, the observability model is incomplete. Effective monitoring produces one coherent attack story.

Q: What should teams do after they detect anomalous MCP tool chaining?

A: They should contain the affected tool, preserve protocol and execution evidence, and compare the observed sequence against the tool’s normal invocation pattern before assuming user error. The priority is to stop further runtime access, because chained tool calls can move from benign actions to sensitive file reads and outbound transfer in a single session.

Technical breakdown

Why protocol logs stop at the wrong boundary

MCP traffic uses JSON-RPC methods, parameters, and responses, but the protocol layer only confirms that a tool was invoked. It cannot show whether the tool read a harmless config file or accessed credential stores, and it cannot prove what happened after the call entered the server process. That is why protocol monitoring is useful for audit context but weak for abuse detection. The security problem is not visibility into messages alone, but the loss of context when requests cross into execution. For MCP, the control boundary has to extend beyond the wire protocol into runtime behaviour.

Practical implication: treat protocol logs as evidence of invocation, not evidence of safe execution.

Why kernel-level execution telemetry changes the evidence quality

Once an MCP tool starts, it becomes a process issuing syscalls such as openat, read, execve, and connect. Those calls reveal what the tool actually did, including file access, child processes, and outbound network egress. eBPF is valuable here because it observes kernel events without relying on the application to self-report, which matters when the tool itself may be compromised. In other words, the kernel can show behaviour that application instrumentation might hide or falsify. For MCP security, execution telemetry is the closest thing to ground truth because it captures side effects, not just declared intent.

Practical implication: add kernel-level telemetry for tool processes so sensitive file access and unexpected egress are visible.

How cross-layer correlation turns noise into an attack story

A protocol anomaly by itself is often ambiguous, and an execution-layer alert by itself may look like ordinary container noise. Correlation becomes the multiplier because it ties a suspicious JSON-RPC call to a specific tool process and then to agent tool-call sequencing. That linkage creates a defensible narrative: who asked, what ran, what it touched, and where the data went. This is also where MCP overlaps with agentic AI risk, because the agent’s sequence of tool calls can reveal prompt injection, capability drift, or session abuse even when each individual event looks normal in isolation. The security value is in joining layers that are usually analysed separately.

Practical implication: build detections that join protocol, execution, and agent traces into one incident timeline.

Threat narrative

Attacker objective: The attacker wants to use trusted MCP tooling to access sensitive data and move it out of the environment without triggering isolated alerts.

1. Entry occurs when a normal-looking MCP request or injected prompt drives a tool invocation that appears valid at the protocol layer.
2. Credential access or data exposure follows when the tool reads sensitive files, credential stores, or internal resources that were not part of the intended request.
3. Escalation and exfiltration happen when the tool opens unexpected outbound connections or the agent chains tools in a new pattern to move data outward.
4. Impact is the loss of sensitive information and the collapse of confidence in the MCP server's runtime trust boundary.

Breaches seen in the wild

• Smithery.ai MCP hosting breach 2025: A Smithery.ai build flaw gave GitGuardian a live fly.io token controlling 3,000+ hosted MCP servers and the API keys their clients sent.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Runtime observability is becoming a control plane, not a troubleshooting layer. MCP security depends on runtime evidence because the threat model includes compromised tools, manipulated instrumentation, and behaviour that only appears after a request crosses from protocol to execution. Security teams that still treat observability as performance management are looking at the wrong layer. The practitioner conclusion is simple: security assurance for MCP starts where application telemetry ends.

Cross-layer correlation is the named concept that matters here. The article shows that protocol alerts, execution signals, and agent behaviour only become actionable when they are correlated into one attack narrative. That is a broader governance lesson for agentic AI and NHI-adjacent workloads: isolated telemetry creates false confidence, while joined telemetry exposes trust boundary violations. Practitioners should design monitoring around correlated evidence rather than source-specific dashboards.

Kernel-level observation is the practical answer to falsifiable tooling. When the threat model includes the tool itself being compromised, self-reported logs and code-level traces are insufficient because they can be altered or omitted. Ground-truth observation from the host or kernel narrows the blind spot and aligns with the logic behind MITRE ATLAS and OWASP Agentic Applications risk thinking. The conclusion for security programmes is to anchor detection where the workload cannot rewrite the evidence.

MCP governance must now account for non-human execution paths as first-class identities. Tools, agents, and their runtime privileges create access paths that behave like machine identities even when they are not managed that way operationally. That means identity governance, privilege scoping, and observability cannot be separated in AI-enabled environments. Practitioners should assume the agent will expose whatever the tool can reach unless runtime controls constrain and correlate that access.

Agentic AI security and NHI security now share the same failure mode. Whether the object is a service account, a tool server, or an AI agent, the core issue is unsafely broad runtime authority paired with weak behavioural evidence. The article reinforces that the security programme has to watch what actually executes, not just what was authorised on paper. The practitioner conclusion is to govern runtime authority as tightly as static access.

From our research library:

• 24,008 unique secrets were exposed in MCP configuration files in 2025 alone, the protocol's first year of widespread adoption, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

Cross-layer correlation is becoming the decisive MCP control: protocol anomalies, execution telemetry, and agent traces only become useful when they are treated as one evidence chain. That is the difference between triage noise and a defensible attack story, and it is why monitoring strategy now has to be designed around runtime context rather than around any single log source.

MCP security is also a warning sign for agentic AI governance more broadly. If an agent can chain tools faster than a team can correlate the runtime evidence, then access review and dashboard-level monitoring are already behind the threat surface.

Runtime-first monitoring is especially important where tools can reach secrets, cloud metadata, or internal endpoints. Once those paths are exposed, defenders need a control model that watches issuance, execution, and egress together instead of assuming one telemetry source will tell the full story.

For practitioners

• Instrument MCP servers with runtime evidence Capture protocol events, process lineage, file access, and network egress for every tool invocation so each request can be traced through the full runtime path.
• Correlate agent traces with execution telemetry Join tool-call sequences to syscall and network data so prompt injection, capability drift, and suspicious chaining are visible in one incident timeline.
• Define sensitive-path baselines for tools Classify expected file paths, databases, and endpoints for each tool and alert when runtime behaviour reaches credential stores, secret paths, or unknown destinations.
• Separate audit logging from detection Retain compact protocol logs for audit, but rely on execution-layer telemetry and correlation for real-time security decisions because logs alone cannot prove side effects.

Key takeaways

• Traditional observability is insufficient for MCP environments because it does not reveal the file access, egress, or tool chaining that determine whether a request was harmless or malicious.
• The central evidence problem is the gap between protocol messages, host execution, and agent behaviour, and attackers exploit that gap to hide in normal-looking traffic.
• Runtime observability changes the security model by tying each invocation to its real side effects, which is the only reliable basis for detection and investigation.

Key terms

• Model Context Protocol: Model Context Protocol is an open protocol that lets AI agents connect to tools and data sources. It expands what an agent can reach, so governance has to cover not only the model and its prompts, but also every system that can receive or return agent-driven data.
• Trace Observability: Trace observability is the ability to inspect a workflow step by step, including inputs, decisions, retries, and outputs. For AI agents, traces show where a loop is wasting tokens, hitting permission barriers, or failing to converge. That visibility supports debugging, cost control, and post-run review.
• Cross-Layer Correlation: The process of linking events from endpoint, SaaS, cloud, and identity systems into one coherent action path. It is not just log centralisation. Correlation turns separate observations into usable identity evidence for investigations, access review, and privilege governance.
• eBPF-Based Monitoring: A kernel-level telemetry method that observes system activity with low overhead and without relying on application self-reporting. It is useful in MCP security because compromised tools may falsify their own logs, while kernel events remain more trustworthy for file, process, and network evidence.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is designed for practitioners who need to govern machine and agent access with clearer evidence and stronger control models.