What is the difference between automatic and manual instrumentation for LLM tracing?

Why This Matters for Security Teams

Tracing is not just observability plumbing for LLM apps. It is the evidence layer that shows what the model saw, what tools it called, where sensitive data moved, and whether a request stayed inside policy. Automatic instrumentation gives broad coverage quickly, but it often stops at the SDK boundary. Manual instrumentation closes the gaps around retrieval, prompt assembly, guardrails, and business logic that determine real risk.

That distinction matters because LLM applications fail in the seams. The OWASP Agentic Applications Top 10 and the NIST AI Risk Management Framework both push teams toward better visibility, but neither removes the engineering choice between convenience and completeness. If tracing only records model calls, security teams may miss the retrieval source that introduced a secret, the rule that downgraded a control, or the downstream action that actually caused impact.

NHIMG research on the AI Agents: The New Attack Surface report shows why that blind spot matters: 80% of organisations report AI agents have already acted beyond intended scope. In practice, many security teams discover trace gaps only after an incident investigation needs them, rather than through intentional observability design.

How It Works in Practice

Automatic instrumentation usually comes from the model SDK, framework, or wrapper library. It can emit spans for prompts, completions, token counts, latency, and sometimes tool invocations. That makes it useful for fast rollout and baseline telemetry. Manual instrumentation is added by developers at the points where the application makes security-relevant decisions: retrieval calls, prompt templating, policy checks, redaction, routing, approval flows, and post-processing.

For tracing to support security analysis, teams usually need both layers to share a common correlation ID and a consistent event schema. Without that, the trace may show the LLM response but not the retrieval step that injected sensitive context. The practical pattern is to let automatic instrumentation capture the model interaction, then add manual spans around control points that automatic hooks cannot see.

A workable implementation often includes:

• Automatic spans for SDK-level prompts, completions, latency, and token usage.
• Manual spans for retrieval-augmented generation, tool calls, policy decisions, and human approvals.
• Structured attributes for tenant, user, model, tool name, data source, and policy outcome.
• Redaction before export so traces do not become a secondary secret store.

The OWASP Top 10 for Agentic Applications 2026 and the CSA MAESTRO agentic AI threat modeling framework both align with this layered view: observability has to reach the decision points, not just the model runtime. NHIMG’s LLMjacking: How Attackers Hijack AI Using Compromised NHIs also reinforces the point that credentials, tool access, and trace data are part of the same attack surface. These controls tend to break down in high-throughput agent pipelines where async calls, nested tools, and third-party middleware make end-to-end correlation incomplete.

Common Variations and Edge Cases

Tighter tracing often increases engineering overhead, privacy exposure, and storage cost, so teams have to balance visibility against data minimisation. That tradeoff is real: more context improves incident response, but more captured data also increases the chance that traces contain prompts, secrets, or regulated content.

Current guidance suggests automatic instrumentation is best for coverage, while manual instrumentation is best for precision. There is no universal standard for this yet, especially across frameworks that abstract away tool calls or chain multiple agent steps. In those environments, automatic spans may overstate what happened by flattening the workflow, or understate it by omitting retrieval and policy checks entirely.

Edge cases matter in production:

• Streaming responses may need manual spans to mark when content became visible to the user.
• Multi-agent systems often need per-agent spans because one trace can hide cross-agent escalation paths.
• Serverless or ephemeral workers can drop context unless correlation IDs are propagated explicitly.
• Privacy-sensitive workloads may require sampling or field-level redaction before trace export.

For teams building governed AI systems, the DeepSeek breach is a reminder that visibility and exposure travel together. The practical rule is simple: use automatic instrumentation for baseline telemetry, add manual spans wherever security decisions happen, and validate that the resulting trace can explain the full request path without leaking the data it is meant to protect.

Related resources from NHI Mgmt Group

• What is the difference between tracing production LLM usage and running evals?
• What is the difference between workflow durability and LLM tracing in AI agent operations?
• What is the difference between tracing and evaluation in LLM operations?
• What is the difference between tracing for LLM applications and an end to end improvement workflow?