Capture and replay exposes policy drift before production changes

At a glance

What this is: This is an architecture write-up on capture and replay for security policy changes, showing how a live proxy journal, shadow evaluation, and signed receipts reduce rollout surprises.

Why it matters: It matters because identity-adjacent policy enforcement, including MCP tool rules and request filtering, needs proof that the running control path will behave the same way after change as it did during review.

Context

Security policy changes fail when teams assume a config edit and a running enforcement path are the same thing. In practice, the live system can disagree with the intended policy because request shapes, session state, and scanner behaviour all affect the verdict. In this article's model, capture and replay is the mechanism that turns those disagreements into reviewable evidence before promotion, which is especially relevant where MCP tool policy and request handling intersect with runtime governance.

The core governance problem is not whether a policy can be written, but whether the enforcement path will actually apply it the way operators expect. That is why replay, shadow evaluation, and signed lifecycle receipts matter: they create a pre-promotion control loop instead of discovering mismatches under production pressure.

Key questions

Q: What breaks when security policy changes are promoted without replay testing?

A: Teams lose the ability to see how the live enforcement path will behave before users feel the impact. Hidden mismatches show up as blocked workflows, missed detections, or inconsistent verdicts after deployment. Replay testing converts those surprises into reviewable deltas, which lets operators refine the policy before ratification instead of reacting under production pressure.

Q: When should teams prioritise shadow evaluation over direct policy rollout?

A: Shadow evaluation should come first whenever a policy change can affect legitimate workflows, shared request shapes, or stateful request handling. That is the point where a candidate contract may look correct in theory but disagree with live traffic in practice. If the change touches access rules, request filtering, or tool-policy decisions, shadowing gives the safer sequence.

Q: What are the signs that path normalisation is too aggressive?

A: Operators should be cautious when distinct endpoints collapse into the same rule, when review output becomes too coarse to explain, or when a small change appears to affect many unrelated requests. Those are signs that the contract has lost useful fidelity. The fix is to pin or split the paths that carry materially different security meaning.

Q: How should teams govern runtime policy changes for MCP and similar tool paths?

A: Treat runtime policy changes as lifecycle events, not as file edits. Require observation, shadow review, ratification, and promotion to happen in sequence, with signed evidence at each step. That approach gives operators a defensible record of what was approved and what was enforced, which is especially important when tool calls can change execution paths quickly.

Technical breakdown

How capture journals preserve policy decisions for replay

The capture phase records each verdict site in a structured JSONL journal so replay can reconstruct what the proxy saw and decided. Each record includes the transport, the subsurface hook, a replayable input summary, the verdict, and session or trace context. That context matters because stateful behaviours such as rate limiting, cross-request exfiltration, and MCP tool baselines depend on ordering. When raw payloads are needed, encrypted sidecars preserve exact content without making the journal itself the sole source of truth. The architectural point is that replay works only when the captured evidence preserves enough fidelity to simulate the same policy path later.

Practical implication: Treat the capture schema as part of the control plane, not as a log format afterthought.

Why path normalization changes contract quality

Path normalization prevents a candidate contract from exploding into millions of nearly identical rules. A compiler that emitted one rule per observed URL path would create an unreadable, brittle policy that operators could not safely review. The normalizer collapses structurally similar paths, such as /users/123/profile and /users/456/profile, into a shared shape when the variable part looks like an identifier. It also supports pinning and splitting so operators can override bad heuristics. The design trade-off is precision versus reviewability: too little normalization creates noise, while too much normalisation hides meaningful distinctions that should stay separate.

Practical implication: Review normalization outcomes before ratification so the contract reflects real enforcement boundaries.

What fidelity gates prevent in replay

Replay is only useful if its verdicts match what the live proxy would have produced. Fidelity gates enforce that by pinning or preserving state that would otherwise change outcomes, such as DNS resolution for SSRF checks or request order for session-dependent behaviour. When the replay engine sees a surface that cannot be reproduced faithfully, it should mark it as stateful rather than pretending the result is deterministic. That distinction matters because the operator is not testing generic policy intent. The operator is testing the exact contract between observed traffic, replay engine, and enforcement logic, which is where false confidence usually enters.

Practical implication: Classify stateful surfaces explicitly so replay does not certify a different policy than production runs.

Breaches seen in the wild

• tj-actions/changed-files compromise 2025: A stolen bot token let attackers poison tj-actions/changed-files so pipelines printed their CI/CD secrets to public logs (CVE-2025-30066).

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Capture and replay is really a governance control for enforcement drift. The article shows that the important failure is not a malicious bypass, but a mismatch between what operators think a policy change will do and what the running proxy will actually enforce. That is the same class of problem identity teams face when access logic changes in one layer but persists differently in another. The operational lesson is that policy state must be proven, not assumed.

Signed lifecycle receipts turn policy change into an auditable chain of custody. Ratify, promote, rollback, and tombstone steps matter because they prove the active contract is the one reviewed by the operator. This is especially relevant where runtime controls touch delegated access, tool execution, or request filtering, because the control value depends on traceability as much as on syntax. Practitioners should treat approval evidence as part of the security boundary.

Path cardinality is a hidden source of governance debt. Without normalization, rule sets balloon into artefacts that no human can reliably review, compare, or retire. That does not just hurt usability; it weakens access governance because operators stop trusting the contract they are meant to approve. The named concept here is policy fidelity debt: the gap between a reviewable contract and the behaviour the production path actually applies. Teams need to reduce that debt before it accumulates into operational risk.

MCP policy enforcement needs the same lifecycle discipline as other identity-bound controls. The article's tool-policy examples show that runtime decisions around MCP calls can break or protect work in ways that resemble privilege changes elsewhere in the stack. Where a multi-agent delegation path exists, the enforcement proof matters even more because one bad assumption can cascade across multiple tool calls. The practitioner conclusion is to govern request policy with evidence, not just configuration intent.

What this signals

Policy fidelity debt: the longer teams rely on direct edits and hot reloads, the more they accumulate hidden differences between intended and actual enforcement. That debt becomes visible only when a replay or rollback exposes a rule that never behaved the way reviewers expected. The practical response is to move policy changes into a verifiable lifecycle before that gap turns into operational risk.

Capture-and-replay design is a reminder that the strongest security controls are often lifecycle controls. When a decision path can be observed, compiled, shadowed, and ratified before promotion, operators can govern the outcome instead of merely reacting to it. For teams running identity-adjacent enforcement points, that is the difference between configuration management and control assurance.

For practitioners

• Define the capture schema as a security control Record transport, subsurface, verdict, and session context so replay can reconstruct the same decision path later.
• Review path normalization before promotion Pin or split paths where structural collapsing would hide a meaningful rule boundary, especially for admin or versioned endpoints.
• Use shadow deltas as the ratification gate Require a signed delta receipt before any policy becomes active, and classify every disagreement as false-positive, false-negative, blocked-now-allowed, or allowed-now-blocked.
• Treat stateful checks as non-replayable when needed Preserve order and pinned resolution for replay surfaces such as SSRF, rate limiting, and cross-request behaviours, or exclude them from deterministic comparison.
• Tie promote and rollback to signed receipts Make the active manifest verifiable against the ratified contract so production cannot drift from approved policy state.

Key takeaways

• The article frames policy rollout risk as a mismatch between reviewed intent and live enforcement behaviour, not as a simple configuration problem.
• Its main control insight is that capture, replay, and signed receipts let operators detect disagreement before a policy change reaches production.
• The practical lesson is to govern policy changes as an evidence chain, because runtime enforcement cannot be assumed to match a YAML edit.

Related NHI Mgmt Group resources: Agentic AI Security Guide · Agentic AI Security Policy Template · Multi-Agent and A2A Security Guide · Red Teaming AI Agents for Identity Abuse

Key terms

• Capture and Replay: A testing and governance method that records live security decisions, then replays those same inputs against a candidate policy or configuration. It helps teams compare actual and intended enforcement before promotion, reducing surprises in production and creating evidence for review, audit, and rollback decisions.
• Shadow Evaluation: A parallel run in which a candidate policy makes decisions alongside the live policy without taking enforcement action. The goal is to surface disagreements, measure false positives and false negatives, and decide whether a change is safe to ratify before it affects real traffic or users.
• Policy Fidelity: Policy fidelity is the degree to which an access rule behaves the same way across different systems and environments. In hybrid identity programmes, it is a practical test of whether orchestration is truly consistent or only appears consistent from a central dashboard. Weak fidelity turns central control into centralised ambiguity.
• Path Normalization: Path normalization is the process of converting a file path into a canonical form so the application can evaluate what it really points to. It helps reveal hidden traversal sequences, mixed separators, and encoded variations before access decisions are made. Normalization is useful, but it must be paired with allowlisting and base-directory checks.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management. It helps practitioners connect identity controls to the broader change-management and runtime assurance decisions their programmes depend on.