Go test caching fails when runtime inputs escape the cache key

At a glance

What this is: Testifysec examines how Go test caching can hide changed runtime dependencies when those inputs are not represented in the cache key.

Why it matters: This matters to practitioners because build and test integrity depends on knowing which inputs actually govern reuse, especially when CI traces, fixtures, and external services can diverge from the command’s tracked inputs.

Context

Go test caching is an execution optimisation, not a guarantee that every runtime dependency was observed. If a test spawns another program, reads files dynamically, or consults a service, the cache can still return a prior result when those dependencies were never part of the tracked inputs.

For identity and security teams, the governance issue is evidence quality. A traced run, a fresh run, and a cached run are different artefacts with different trust boundaries, so test acceptance, regression proof, and diagnostic telemetry must be kept separate.

The article’s central point is that this failure mode is typical whenever teams assume the command line alone fully describes test behaviour. That assumption is especially weak in CI pipelines where external fixtures, local files, and subprocesses shape outcomes.

Key questions

Q: What breaks when a Go test depends on runtime inputs that are not in the cache key?

A: The cache can replay a prior pass even though the test’s real behaviour has changed. That creates a false sense of regression coverage because the command saw the same declared inputs, while the test actually consulted files, subprocesses, or services that were not tracked. The fix is to make those dependencies explicit or force the affected path to run fresh.

Q: Why does a diagnostic trace not prove that a cached test result is trustworthy?

A: A trace only shows what the selected backend observed during that execution, and its coverage is limited by the tool and backend. Missing observations can mean limited visibility, not no activity. For trust decisions, teams need a fresh baseline and a separately governed acceptance artefact, not a trace alone.

Q: How can security teams tell when test caching is hiding a dependency problem?

A: Watch for tests that discover inputs at runtime, call out to services, or change behaviour when fixture files are added or altered. If a relevant input changes and the result still reports cached, the cache key is incomplete. That is the strongest signal that the test is not modelling its dependency set correctly.

Q: Should teams treat traced test runs and release evidence as the same thing?

A: No. Traced runs are useful for investigation, but release decisions need commit-bound evidence that matches the relevant policy and trust boundary. Mixing those roles weakens both functions. Keep traces for diagnostics and use separate evidence for acceptance, especially when cache reuse is involved.

Technical breakdown

How Go decides whether a test result is reusable

Go caches test results when the command, package inputs, and tracked environment match what it has already seen. That works well for deterministic tests, but it breaks down when the test depends on runtime-observed state that the command never declared. If a test discovers files late, reads a network resource, or shells out to another program, the cache can treat materially different executions as equivalent. The key problem is not caching itself, but incomplete dependency modelling. Once the declared input set and the real input set diverge, a cached pass can mask a current failure.

Practical implication: Treat cache eligibility as a dependency-design issue, not a performance default.

Why traces can mislead without a fresh baseline

Diagnostic traces record what a selected backend observed during a specific execution, but observation coverage is bounded by the tool and backend. An empty network or subprocess record does not prove absence of activity, only absence of visibility in that trace. That is why a bounded fresh run matters before any cached experiment: it gives you a controlled baseline, then a repeatable comparison. Decoding a DSSE payload may reveal captured data, but it does not itself establish signature trust or evidence suitability for acceptance decisions.

Practical implication: Use tracing to investigate behaviour, not to certify result validity.

How input-change experiments expose missing cache dependencies

The strongest validation method is simple: change one input the test is supposed to depend on and require the test to fail or rerun. If the result stays cached after a relevant input changes, the cache key is incomplete. The source article’s schema-inventory example shows the pattern clearly: adding a schema file did not invalidate the cached result until the schema package became an explicit dependency. That is a dependency-registration failure, not a cache bug in isolation.

Practical implication: Build regression tests that mutate one dependency at a time and verify the rerun occurs.

Threat narrative

Attacker objective: The practical objective is not malicious exploitation but incorrect trust in a cached test outcome that no longer reflects current inputs.

1. Entry occurs through a test that relies on subprocesses, file discovery, or external service calls that are not fully represented in the tracked inputs.
2. Credential or state exposure is not the issue here; the failure is that the cache key omits a real dependency, allowing the prior result to be replayed.
3. Impact is a false passing result that hides a regression until a fresh run or an external check finally exposes it.

Breaches seen in the wild

• LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Cache correctness is an evidence-governance problem, not just a build optimisation problem. Teams often talk about test caching as a speed feature, but the article shows that its real risk is trust misplacement. If the dependency graph is incomplete, a cached pass can masquerade as a valid regression check. Practitioners should treat cache reuse as a governed evidence decision, not a convenience setting.

Runtime-discovered inputs create a blind spot that ordinary test declarations do not close. A test that starts subprocesses, reads files at runtime, or consults a service can still appear deterministic to the command tracker. That makes the hidden dependency the real control gap. The correct response is to make those dependencies explicit so the cache key reflects the actual attack surface for bad evidence.

Diagnostic traces and acceptance evidence belong in different trust tiers. A locally signed trace can help explain behaviour, but it is not the same as commit-bound proof required by a push policy. The governance assumption that one artefact can satisfy both investigation and release acceptance fails here. Practitioners should separate investigative telemetry from release evidence and define which artefacts are authoritative for each decision.

Test freshness must be enforced where external dependencies are dynamic. The article’s guidance on deterministic fixtures and fresh runs points to a deeper pattern: any dependency outside the command’s tracked inputs should be treated as a cache-breaker until proven otherwise. That is especially relevant in CI systems where local state, fixtures, and transient services can vary between runs. Teams should make freshness an explicit control for non-deterministic tests.

Invisible dependency drift creates a repeatable false-pass pattern. Once a test’s real inputs drift away from its declared inputs, repeated cached passes become noise rather than assurance. This is the kind of failure that survives normal happy-path validation because the pipeline is optimised to trust reuse. The practitioner takeaway is to hunt for tests whose environment changes outside the declared cache boundary.

What this signals

Invisible dependency drift: build and test pipelines can produce stable-looking results while silently ignoring runtime inputs that were never declared. That changes the control point from execution time to evidence design time, because the real question becomes whether the cache key actually reflects the test’s dependencies.

For identity and governance teams, this is a useful analogy for any control that certifies state after the fact. If the measured object can change outside the measured boundary, the control will validate the wrong thing. The response is explicit dependency registration, not blind trust in reuse.

For practitioners

• Define cache-breaking dependencies for tests List every file, subprocess, service, and generated artefact a test actually consults, then make sure those inputs are declared or force a fresh run when they cannot be tracked.
• Run a bounded fresh baseline before cache experiments Capture one suspicious test in a private scratch environment with a timeout, using the installed tool’s tracing guidance, before comparing it to a cached rerun.
• Separate investigation traces from acceptance evidence Store diagnostic traces inside approved access boundaries and do not treat them as commit-bound proof for a push gate or release decision.
• Use input-change tests to prove invalidation Change one dependency at a time, rerun the package, and confirm the intended test executes rather than silently reusing the old result.
• Keep external-service tests fresh when the cache key is incomplete If an external service or subprocess input cannot be modelled reliably, preserve the regression assertion but disable reuse for that path.

Key takeaways

• Go test caching is only trustworthy when the tracked inputs match the test’s real runtime dependencies.
• Diagnostic traces help explain behaviour, but they do not by themselves establish evidence quality for acceptance decisions.
• The practical control is to force reruns when dependencies are discovered at runtime or outside the cache key.

Key terms

• Cache Key: The cache key is the set of request attributes a caching layer uses to decide whether two requests are the same. For authenticated systems, it must distinguish users or sessions when the response is private. If it only keys on the URL, confidential data can be replayed across identities.
• Runtime Dependency Risk: Runtime dependency risk is the gap between what a software manifest says will run and what a live workload is actually executing. It matters because malicious or altered packages can behave differently at runtime, especially when they touch secrets, spawn processes, or call external endpoints.
• Diagnostic Trace: A diagnostic trace is execution evidence captured to explain what happened during a run, usually for troubleshooting or investigation. It can be useful context, but it is not automatically authoritative proof that a result is trustworthy or policy-compliant.
• Commit-Bound Evidence: Commit-bound evidence is test or validation output that is explicitly tied to the code or change being reviewed. It supports acceptance decisions only when it is collected under the right policy, not when it is merely convenient diagnostic output.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is a practical route for security teams that need stronger control over credentials, lifecycle, and governance decisions.