Dynamic garbage collection tuning for memory-heavy OCaml workloads

At a glance

What this is: This is a Semgrep engineering analysis showing that OCaml 5 introduced major-memory regressions for one workload and that dynamic garbage collector tuning restored acceptable performance across repository sizes.

Why it matters: It matters because practitioners running memory-intensive security and analysis pipelines need to understand when runtime tuning, not code changes, is the fastest path to stable operation and predictable resource use.

By the numbers:

• Only 20% have formal processes for offboarding and revoking API keys, and even fewer have procedures for rotating them.
• 71% of NHIs are not rotated within recommended time frames, increasing the risk of compromise over time.
• 91.6% of secrets remain valid five days after the targeted organisation is notified, showing a critical gap in remediation procedures.
• 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools.

👉 Read Semgrep's analysis of OCaml 5 memory regressions and GC tuning

Context

OCaml 5 changed the runtime model enough that a workload which behaved acceptably on OCaml 4 could consume far more memory after the upgrade. For teams that run analysis engines, scanners, or other long-lived services, that kind of regression is a governance problem as much as a performance problem because it affects availability, cost, and rollout risk.

Semgrep's experience also shows why runtime behaviour needs measurement before assumptions harden into policy. The article connects memory tuning to operational stability, which is relevant to identity and security platforms that depend on reliable tooling, predictable pipelines, and safe upgrades rather than default settings alone.

Key questions

Q: How should teams validate memory behaviour after a runtime upgrade?

A: Start with representative workloads, then measure peak memory, RSS, and runtime together before and after the change. Compare the allocation trace with operating system memory usage so you can tell whether the regression comes from the program, the collector, or fragmentation. That evidence should decide whether to tune, patch, or roll back.

Q: When does static garbage collector tuning stop working?

A: Static tuning breaks down when the same setting must serve workloads with very different allocation patterns or heap sizes. Small jobs may become slower because the collector works too hard, while large jobs may still exhaust memory. At that point, tuning should become workload-aware and bounded rather than fixed.

Q: What do teams get wrong about memory regressions in long-running services?

A: They often assume the problem is only application code, when the runtime may have changed the collector's behaviour or memory-return policy. That assumption leads to the wrong fix. Teams should first separate live allocation growth from process-level memory overhead before deciding where the fault sits.

Q: Who should own runtime performance regressions in security tooling?

A: Ownership should sit with the team that runs the service, but change control should include engineering, operations, and security stakeholders if the tool feeds risk or access decisions. A runtime regression is not just a performance issue when it threatens scan reliability, pipeline stability, or decision quality.

Technical breakdown

Why OCaml 5 changed memory behaviour

OCaml 5 introduced multicore support, which required a redesigned runtime, garbage collector, and allocator. Those changes improved parallel execution, but they also altered how long memory stays live and how aggressively the runtime returns unused memory. In Semgrep's workload, that meant peak memory could rise sharply even when the program logic had not changed. The key distinction is between live memory, which Memtrace can observe, and resident memory, which also includes allocator and heap effects. Practical implication: treat major runtime upgrades as performance changes, not just language version bumps.

Practical implication: Measure real workloads before rollout, because runtime defaults may no longer match your service's memory profile.

Memtrace versus resident set size

Memtrace records when allocations become live and when they are collected, which is useful for comparing program behaviour across versions. Resident set size adds the operating system view, capturing memory that may be wasted in heap fragmentation or not yet returned to the OS. Semgrep used both to separate actual allocation growth from collector behaviour. That distinction mattered because the problem was not only how much memory was allocated, but how long unreferenced memory remained around. Practical implication: pair runtime profiling with OS-level telemetry before deciding whether a regression is in the program or the collector.

Practical implication: Use both allocation tracing and RSS monitoring before you blame the application or the allocator.

Dynamic GC tuning with space_overhead

OCaml exposes major collector aggressiveness through the space_overhead parameter. Lower values make the collector work harder to reclaim memory, while higher values trade memory efficiency for speed. Semgrep found no single static value worked well across small and large repositories, so it used GC alarms and quick_stat to adjust space_overhead after each major collection based on heap size. This is a simple feedback loop: memory pressure rises, the collector becomes more aggressive, and the setting relaxes again when the workload is smaller. Practical implication: when one tuning value cannot fit all workloads, a bounded controller can stabilise performance without code changes.

Practical implication: Introduce workload-sensitive GC policies when a single static setting creates either memory blowups or time regressions.

NHI Mgmt Group analysis

Runtime tuning is part of operational governance, not a niche performance trick. Semgrep's experience shows that a platform upgrade can silently change the resource contract of a service even when application code is unchanged. For security and analysis tooling, that means upgrade management has to include measurement, rollback planning, and workload-specific validation. The practitioner lesson is to govern runtime behaviour with the same seriousness as access and deployment change.

Memory governance debt is the hidden cost of assuming one collector setting fits all workloads. Semgrep found that static tuning could not satisfy both smaller and larger repositories, which is a pattern many teams see in analysis pipelines, CI services, and security scanners. This is not just a tuning inconvenience. It is a sign that the control model is too coarse for the variance in workload shape. Practitioners should expect mixed performance unless they introduce policy that adapts to scale.

Instrumentation beats intuition when runtime behaviour changes under a major version. The combination of Memtrace, RSS polling, and controlled benchmarks let Semgrep isolate the real source of regression. That is the right pattern for identity and security programmes too: use evidence to distinguish application growth from runtime side effects. Without that separation, teams often overcorrect in the wrong place and create new instability. The practitioner conclusion is to make measurement a gate in any upgrade path.

Workload-aware control loops are becoming the practical model for reliable security tooling. Semgrep's dynamic GC utility mirrors a broader shift in engineering governance where systems are tuned continuously rather than configured once. That approach fits modern analysis and security platforms, which often see uneven request sizes, repository sizes, or processing depth. The right frame is not perfection, but bounded adaptability. Practitioners should design for controlled variation instead of static assumptions.

The identity and governance angle is indirect but real: platform reliability determines whether security controls stay trustworthy. When scanners, analyzers, and policy engines become memory unstable, teams lose confidence in the outputs that inform access, code, and risk decisions. That turns performance into a governance concern. The practical conclusion is to treat runtime regressions as control failures when they affect the fidelity of security workflows.

What this signals

Runtime regressions should be treated as governance signals when they affect analysis engines, scanners, or policy systems that inform security decisions. The practical lesson is to set explicit validation gates for memory-heavy services before upgrades move from test to production, especially where default runtime settings were tuned for older execution models.

Memory-pressure drift: when runtime defaults stop matching workload size, the service may remain functional while silently becoming more expensive, slower, or less stable. That drift is dangerous in security tooling because teams tend to notice the symptom late, after capacity is exhausted or scan confidence falls. The right response is continuous telemetry, not one-off tuning.

Where the tooling supports access, secret, or policy decisions, runtime stability becomes part of control assurance. Teams should link capacity monitoring to release management and, where identity workflows are involved, keep the lifecycle disciplines in Ultimate Guide to NHIs in view because unstable platforms undermine trust in downstream governance.

For practitioners

• Implement workload-specific runtime benchmarks Run the same scan or analysis job across representative small, medium, and large inputs before and after an interpreter or runtime upgrade. Track peak memory, RSS, and execution time together so you can see whether the regression is in allocation behaviour or collector behaviour. Use the benchmark set as a release gate, not an afterthought.
• Add dual-layer memory telemetry to production services Combine allocation tracing with OS-level RSS polling for the services that matter most. Allocation traces help explain what the program is doing, while RSS shows what the process is costing the machine. That separation is what lets you choose between code fixes, GC tuning, or capacity changes with confidence.
• Adopt bounded, workload-aware tuning policies If one static GC setting cannot fit all workloads, define lower and upper bounds and adjust the collector based on a concrete runtime signal such as heap size. Keep the policy simple enough to reason about, but tie it to the size of the live service rather than a global default. Revalidate the bounds after each major upgrade.
• Treat runtime upgrades as change-managed releases Require rollback criteria, capacity review, and validation data whenever a runtime version changes the memory model. That process matters for scanners, security analytics, and other long-running jobs because a collector change can be as disruptive as a code defect.

Key takeaways

• OCaml 5 changed memory behaviour enough to expose a real operational regression for Semgrep's interfile analysis workload.
• Static garbage collector settings did not generalise across workload sizes, which is why dynamic tuning restored acceptable memory and time characteristics.
• Teams running memory-intensive security tooling should treat runtime upgrades as measured change events, not simple version bumps.

Key terms

• Garbage Collector Tuning: Garbage collector tuning is the practice of changing runtime parameters so the collector reclaims memory more or less aggressively. In long-running systems, the goal is to balance throughput, latency, and memory footprint without changing application logic.
• Resident Set Size: Resident set size is the amount of physical memory a process is currently using on the host. It includes live data and overhead from the runtime, allocator, and heap behaviour, so it is often the best measure of real operational cost.
• Memtrace: Memtrace is a memory profiling tool that records where memory is allocated and when it is collected. It helps compare runtime behaviour across versions by showing how much memory stays live over time, even when the process still appears healthy.
• Space Overhead: Space overhead is an OCaml garbage collector parameter that controls how aggressively the runtime trades CPU time for lower memory usage. Lower values reclaim memory more aggressively, while higher values preserve speed at the cost of larger memory consumption.

What's in the full article

Semgrep's full post covers the implementation detail this analysis intentionally leaves at the architecture level:

• The exact Memtrace and RSS workflow used to compare OCaml 4 and OCaml 5 memory behaviour
• The dynamic_gc approach for adjusting GC settings after each major collection
• Repository-specific benchmark results across Juice Shop and Blaze Persistence
• The future work options, including live-memory tuning and more sophisticated controllers

👉 Semgrep's full post covers the benchmark comparisons, GC heuristics, and future tuning ideas in detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It is designed for practitioners who need to connect identity controls to broader security and operational programmes.