Why is mesh-level telemetry alone not enough to understand what is happening in front of proxied services?

Why mesh telemetry is incomplete at the proxy boundary

Mesh telemetry is valuable, but it is usually produced by the sidecar or dataplane layer, which sees the flow after traffic has already crossed the proxy boundary. That means it can describe service-to-service behaviour without fully explaining what the ingress proxy actually received, matched, transformed, or rejected. For proxied services, the boundary is part of the system, not just a transport detail.

That distinction matters because the gateway often makes the first material decision about the request: which route is selected, whether headers are rewritten, whether a request is dropped, and whether the upstream even sees traffic. If you only watch the mesh plane, you may mistake downstream service symptoms for upstream routing behaviour.

The practical consequence is that mesh metrics answer “what happened inside the service mesh,” while gateway metrics answer “what happened before the service was reached.” Those are related but not interchangeable views.

What gateway telemetry adds to troubleshooting and capacity planning

Gateway metrics fill the missing operational layer by exposing request volume, status codes, latency at the edge, and routing outcomes. In practice, that is where teams learn whether a spike is caused by client demand, path mismatch, retries, malformed requests, or an overloaded ingress tier. Without that layer, troubleshooting becomes inference rather than observation.

For capacity planning, gateway telemetry is often the more reliable signal for real demand because it includes requests that never make it into the application mesh. A service may appear healthy on mesh metrics while the gateway is absorbing bursts, rejecting malformed traffic, or concentrating load on a subset of backends. That can hide the true pressure on the front door and lead to undersized ingress capacity.

Gateway telemetry also helps distinguish application failure from routing failure. A rise in 5xx responses in the mesh may reflect upstream behaviour, but the same symptom at the gateway may come from misconfiguration, policy enforcement, or exhaustion of the proxy itself. The more centralized the front door, the more important it is to observe it directly.

What you should measure when traffic is proxied

For proxied services, the most useful view combines mesh and gateway signals rather than choosing one. Mesh telemetry is best for east-west service health, while gateway telemetry is best for north-south ingress behaviour and request attribution. Together they show where a request was accepted, where it changed shape, and where it failed.

• Gateway request rate and error rate, to see traffic before service admission.
• Route match and upstream selection outcomes, to confirm whether requests reached the intended backend.
• Request latency at both gateway and service layers, to isolate whether delay is at the edge or in the application path.
• Rejected, redirected, or rewritten requests, to capture behaviour that mesh-only views often miss.

This combined view is especially useful when multiple teams own different layers of the path. The ingress layer, the mesh layer, and the application all report different facts, and each can be correct within its own scope.

Risk and Threat Considerations

When teams rely only on mesh telemetry, they can miss abnormal activity that terminates or mutates at the gateway, including blocked requests, route probing, and load concentrated on the front door. That creates a visibility gap that weakens incident triage and can hide misrouting or abuse patterns until the backend starts failing.

Failure mechanism: Telemetry from the sidecar or dataplane observes traffic only after ingress decisions have already happened, so request drops, route mismatches, header rewrites, and edge-side saturation may never be reflected in the mesh view.

Impact: Operators can misread the health of the system, under-provision ingress capacity, and spend time investigating backend services that are not the real source of the problem.

Practitioner Guidance

What to verify: Confirm that gateway metrics capture the same request classes that matter for operations, including routed, rejected, rewritten, and delayed requests. If the front door handles auth, rate limiting, or path normalization, make sure those outcomes are visible separately from service telemetry.

Decision rule: If the symptom appears only in mesh data, validate the ingress path before treating it as a service issue. If the symptom appears at the gateway but not the mesh, assume the problem is likely at the edge or in routing policy until proven otherwise.

Practitioner takeaway: Mesh telemetry is a downstream view of proxied traffic, so it should be treated as one layer of evidence, not the full operational truth. The most accurate picture comes from correlating gateway and mesh signals across the entire request path.

Related resources from NHI Mgmt Group

• What do teams get wrong when they rely on Network Level Authentication alone to protect remote desktop services?
• How do security teams decide whether telemetry is good enough for enforcement?
• When is SPIFFE alone not enough for NHI governance?
• Why is cloud IAM alone not enough for agentic workloads?