Persistent workspace state should live in dedicated volumes or mounted directories, while rebuilt layers stay immutable and disposable in the image or container filesystem. Separate config, cache and data paths from code and dependency layers, then mount only the needed paths back into the runtime. That keeps rebuilds predictable and avoids overwriting state during image replacement.
Why workspace state should not be rebuilt with the image
The core design choice is to treat the image as a rebuildable artifact and the workspace state as a separate persistence boundary. Code, dependencies and compiled layers should be replaced freely; user data, caches, runtime state and other durable files should survive replacement only through a volume or mounted directory. That separation makes rebuilds repeatable and prevents accidental state loss when containers are recreated.
Teams usually get into trouble when they let application state accumulate inside the same filesystem area as the image contents. Once that happens, a new build can overwrite files that were never meant to be versioned, or preserve stale state that no longer matches the rebuilt application. Keeping state outside the image avoids both extremes.
A practical way to think about it is that the image should express what is immutable, while the mounted path should express what must outlive the process. If a file must remain available after a redeploy, it belongs in the persistence layer, not in the layered image itself.
What belongs in volumes versus rebuilt layers
Config, cache and data should be separated on purpose, because each behaves differently during deploys and rollbacks. Configuration may be injected through environment variables, files or mounted secrets, while caches can often be discarded and rebuilt. Durable application data, uploads, job queues and other stateful artifacts need a stable mount path that is not tied to the container image lifecycle.
Rebuilt layers should stay immutable and disposable. That means the application binaries, language dependencies and base operating system content should be replaced as a whole when the image changes, instead of being edited in place at runtime. When teams keep the runtime writable only where necessary, they reduce drift between what was tested and what is actually running.
The simplest operational test is whether the service still behaves correctly after you throw away the container and start a new one from the same image. If the answer depends on files inside the image filesystem, the boundary is probably wrong. If it depends on mounted storage that is intentionally preserved, the boundary is probably right.
How to mount only the paths the runtime actually needs
Mounting should be narrow, not broad. Rather than exposing a whole application tree, mount only the exact directories that need durability, such as a data directory, a cache directory or a config path. That limits the blast radius of a bad write and makes it easier to reason about which files are replaceable and which are persistent.
A useful discipline is to define each path by function: code path, dependency path, config path, cache path and data path. Once that separation is explicit, it becomes much harder for an image build to silently overwrite live state or for runtime writes to leak into a layer that should remain immutable. NIST Cybersecurity Framework 2.0 is helpful here because it reinforces asset boundaries, resilience and controlled recovery, all of which depend on knowing what is rebuilt and what is preserved.
This pattern also helps when a team scales from a single service to many replicas. Shared state should be deliberate, and per-instance state should not be hidden inside the image. If a directory must be writable, make that decision explicit in the deployment spec instead of assuming the runtime will sort it out.
Risk and Threat Considerations
When state and rebuilt layers are mixed together, deployments become fragile and compromise response becomes harder. A redeploy can destroy evidence, preserve malicious changes, or leave behind stale files that the new build will trust. The same confusion also creates operational risk, because teams cannot easily tell whether a failure is caused by the image, the mount, or a lingering runtime artifact.
Failure mechanism: Mutable files are written into paths that are assumed to be disposable, so image replacement overwrites live data or carries forward state that should have been reset. In a compromised environment, that same mistake can hide persistence in places operators expect to be rebuilt.
Impact: Teams lose predictability in rollouts, rollbacks and incident recovery. They may also miss unauthorized changes because rebuilds do not fully replace the true state of the service.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.DS-10 — Data in transit is protected | Mount separation preserves data flow boundaries and reduces accidental exposure during rebuilds. |
| PR.IR-01 — Networks, systems and assets are protected by resilience mechanisms | Persistent volumes and immutable layers improve recovery and replacement resilience. | |
| Recommendation — Protect mounted state paths and isolate data flows from disposable image layers. Use immutable rebuilds and separate storage to improve service recovery after redeployments. | ||
| NIST SP 800-53 Rev 5 | CM-2 — Baseline Configuration | Separating state from image layers supports a stable, repeatable build baseline. |
| CM-6 — Configuration Settings | Mounting only needed paths is a configuration control decision that constrains writable scope. | |
| Recommendation — Define the image as the baseline and keep mutable runtime state outside it. Restrict writable directories to the minimum set required by the runtime. | ||
| ISO/IEC 27001:2022 | A.8.9 — Configuration management | The design depends on controlled separation of immutable image content and persistent runtime paths. |
| Recommendation — Manage image and mount configuration so rebuilt layers stay immutable. | ||
Practitioner Guidance
What to verify: Confirm which paths are writable at runtime, which are mounted, and which are recreated from the image on every deploy. If the application writes outside the explicitly mounted state paths, treat that as a design defect rather than an implementation detail.
Decision rule: If the file must survive container replacement, put it on a dedicated volume or mounted directory. If the file can be regenerated safely, keep it in the disposable layer and make the application capable of rebuilding it cleanly.
Practitioner takeaway: The goal is not simply to make the container writable, it is to make persistence intentional, narrow and observable so rebuilds can be trusted to do exactly one thing: replace the immutable parts.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on November 16, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org