Continuous profiling in OCaml: what Pyro Caml changes for practitioners

At a glance

What this is: Semgrep describes how it built Pyro Caml, a continuous profiler for OCaml, to measure production performance under sandbox and runtime constraints.

Why it matters: This matters because security and platform teams often inherit codebases where profiling, observability, and runtime safety have to coexist, and those constraints shape what can actually be monitored and tuned.

By the numbers:

• Only 5.7% of organisations have full visibility into their service accounts.
• 97% of NHIs carry excessive privileges, increasing unauthorised access and broadening the attack surface.
• 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools.

👉 Read Semgrep's post on building Pyro Caml, a continuous profiler for OCaml

Context

Continuous profiling is a production telemetry pattern that samples runtime behaviour over time, rather than requiring engineers to reproduce a problem locally. In Semgrep’s case, the challenge was not just performance analysis but performance analysis under sandboxing, a small OCaml tooling ecosystem, and a security posture that avoids broad access to customer code. The primary topic here is continuous profiling, and the identity security angle is that tooling constraints often determine whether operators can safely inspect sensitive workloads at all.

The article’s real value is architectural: it shows how observability becomes a design constraint when a product runs at scale and must remain reliable while scanning customer code. That is a familiar governance problem in identity and security programmes too, where telemetry must be safe, low overhead, and compatible with the operating environment. The starting position is typical for mature production engineering, but atypical in the precision with which it combines language runtime limits, sandboxing, and profiler design.

Key questions

Q: How should teams choose profiling tools for production services with strict runtime constraints?

A: Prioritise tools that can run safely inside the real production boundary, with low overhead and a failure mode that does not affect application correctness. If a profiler requires weaker sandboxing, broad code access, or changes workload behaviour enough to distort results, it is not suitable for continuous use in sensitive systems.

Q: Why do sandboxed workloads make observability harder to implement?

A: Sandboxing can remove the kernel hooks, system calls, or runtime access that many observability tools expect. That forces teams to choose between weaker security boundaries and reduced visibility. The practical answer is to design telemetry around the sandbox, not against it, so measurement remains compatible with the protected workload.

Q: What do security teams get wrong about low-overhead telemetry?

A: They often assume any telemetry is better than none, even when the measurement path distorts the system being measured. In practice, a profiler that changes scheduling, allocation patterns, or execution time can produce misleading conclusions. Teams should validate that overhead remains stable across the workload they care about.

Q: How do you know if continuous profiling is working well enough?

A: It is working well enough when it consistently identifies bottlenecks that change tuning decisions without materially slowing the service. You do not need perfect fidelity, but you do need repeatable signals, acceptable overhead, and enough stack coverage to distinguish real hotspots from measurement artefacts.

Technical breakdown

Why continuous profiling differs from local profiling

A normal profiler is usually developer-driven and short-lived, while continuous profiling runs in production and collects data repeatedly over time. That difference matters because production workloads expose behaviour that local tests miss, especially when access to representative data is restricted. Semgrep’s situation also shows why continuous profiling is not just an engineering convenience: when you cannot safely copy the workload locally, production sampling becomes the only reliable path to root-cause analysis. The trade-off is that the profiler must remain low overhead and safe enough not to alter system correctness.

Practical implication: choose profiling methods that can run inside the real security boundary without forcing weaker access to sensitive workloads.

How Memprof sampling creates useful OCaml call stacks

Memprof is OCaml’s memory profiling mechanism, and in this design it is repurposed as an allocation-based sampling source. Instead of timing every function, the profiler triggers callbacks on a percentage of allocations and receives a call stack from the runtime at that moment. That works better for allocation-heavy CPU workloads, but it is not a perfect time sampler. Short-lived functions can be missed, and allocation frequency can bias the profile. The key technical insight is that accuracy comes from enough sampling density, not from instrumenting every execution path.

Practical implication: validate that your sampling method matches the workload’s behaviour before trusting the resulting hotspot data.

Why runtime events and Rust FFI reduce profiling overhead

OCaml 5 runtime events provide a low-overhead channel for emitting profiling data into a file-backed ring buffer, which another process can read and forward. That separation matters because the profiler should do as little work as possible in the target process. Semgrep then uses Rust as the transport and aggregation layer through FFI, leaving OCaml responsible for the stack capture and Rust responsible for packaging and delivery. This is a common production pattern: keep the measurement path thin, move processing out of band, and accept some loss of fidelity if the result is operationally safe.

Practical implication: separate sampling from export so the instrumentation path does not become the bottleneck you are trying to find.

NHI Mgmt Group analysis

Continuous profiling is a governance control, not just an engineering convenience. When a security product runs at scale, the ability to understand runtime behaviour becomes part of operational assurance. Semgrep’s approach shows that visibility tooling must fit the execution environment, especially when sandboxing and sensitive workloads limit what engineers can inspect directly. For IAM and security leaders, the lesson is that observability design should be treated like any other control dependency, because if it cannot run safely in production, it cannot support reliable governance.

Telemetry boundary mismatch is the core failure mode this article exposes. The profiler problem was not the absence of data, but the mismatch between the data the team needed and the mechanisms available in OCaml, gVisor, and the broader tooling ecosystem. That is a useful pattern for identity security teams as well: control effectiveness often fails where runtime boundaries, language runtimes, or sandboxing assumptions break the measurement path. Practitioners should treat missing telemetry coverage as a control gap, not a tooling inconvenience.

Low-overhead observability is essential when the workload itself is the protected asset. If the act of measuring adds 80% overhead, the measurement has become the problem. Semgrep’s choice to constrain profiler overhead to around 5% reflects a broader design principle that also applies to identity and access systems handling sensitive code, secrets, or customer data. The practical conclusion is straightforward: runtime controls must be safe enough to stay on by default, or they fail the governance test.

Open standards and separable components reduce governance debt. The article’s emphasis on OpenTelemetry, Pyroscope, and Rust FFI shows a preference for composable building blocks instead of closed measurement paths. That matters because identity and security programmes repeatedly inherit systems where every special case becomes an exception to govern. Standardised, inspectable telemetry architecture is easier to audit, easier to extend, and easier to align with operational controls. Practitioners should favour architectures that keep measurement, transport, and storage independently reviewable.

Profiling accuracy is a threshold question, not an absolute one. The article is explicit that the goal is not perfect flame graphs but enough accuracy to find bottlenecks that materially affect performance. That same logic applies to security governance: teams need enough signal to make decisions, not exhaustive visibility that never arrives. The right question is whether the control produces stable, decision-grade evidence under real constraints. Practitioners should define the minimum useful fidelity up front and measure against that target.

What this signals

Continuous profiling has a clear governance analogue in identity programmes: if the measurement layer cannot operate inside the protected boundary, the control story is incomplete. Telemetry-boundary mismatch: teams should treat missing runtime visibility as an operational risk, because blind spots in measurement often map directly to blind spots in assurance. For readers responsible for identity-led systems, that means aligning observability architecture with the same boundary discipline used in IAM and PAM.

The practical signal for security programmes is that low-overhead instrumentation must be evaluated like any other control dependency, not assumed to be harmless. Where workloads are sensitive or sandboxed, the monitoring path may need separate runtime components, explicit failure handling, and a tighter definition of what “good enough” evidence means. That is especially true when continuous profiling sits alongside broader identity and secrets governance, where a single blind spot can hide both performance and security issues.

For practitioners

• Define the production boundary for observability Map where profiling can run safely in production, including sandboxed services, restricted workloads, and any environments where attaching standard profilers is impossible. Document which telemetry paths are allowed to touch sensitive runtime data and which must stay out of process.
• Set an overhead budget before enabling continuous profiling Establish a hard ceiling for runtime impact, then test profiler configurations against that target under representative load. If the profiler changes application behaviour enough to bias results, treat it as invalid for operational use rather than assuming more data is better.
• Separate stack capture from data export Keep the in-process sampling path minimal and move formatting, aggregation, and transmission into a different runtime or service. That reduces skew, lowers failure blast radius, and makes it easier to reason about correctness when the profiler is active.
• Validate sampling fidelity against real workload shapes Check whether allocation-heavy or highly recursive code is being represented accurately enough to support tuning decisions. Pay special attention to workloads with FFI calls, truncated stacks, or low allocation rates, because those are the places where profiles become least trustworthy.

Key takeaways

• Continuous profiling is only useful when it can run inside the real production boundary without weakening the workload’s security or correctness.
• Semgrep’s architecture shows that allocation-based sampling, runtime events, and out-of-process export can deliver decision-grade visibility with controlled overhead.
• For security and identity practitioners, the lesson is to treat observability as a governed control plane, not a disposable engineering utility.

Key terms

• Continuous profiling: Continuous profiling is the practice of collecting performance samples from production systems over time rather than during a one-off debugging session. It gives operators a time-based view of hotspots, but only if the sampling method stays low overhead and reflects the workload accurately.
• Allocation-based sampling: Allocation-based sampling triggers profiler callbacks when memory allocations occur instead of at fixed time intervals. In OCaml, this can be a practical way to approximate runtime behaviour, but it may over-represent allocation-heavy code and under-represent logic that allocates less often.
• Runtime events: Runtime events are low-overhead signals emitted by a language runtime to describe what the program is doing while it runs. They are useful because they let tools observe production behaviour without adding as much instrumentation cost as in-process measurement often requires.
• Telemetry Boundary: A telemetry boundary is the point where operational logs stop being internal data and become governed data subject to new access, retention, and privacy rules. In migration projects, that boundary often shifts from the SIEM itself to the collection and routing layer before ingestion.

What's in the full article

Semgrep's full post covers the implementation detail this post intentionally leaves for the source:

• The Memprof sampling design and why allocation-based profiling was chosen over standard time-based profilers.
• The OCaml runtime events pipeline used to move profiling data out of process with minimal overhead.
• The Rust FFI integration pattern that connects OCaml sampling to the Pyroscope backend.
• The production caveats, including FFI blind spots, truncated stacks, and plans for memory profiling.

👉 Semgrep's full post covers the sampling architecture, runtime event pipeline, and production caveats in detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It is designed for practitioners who need to connect identity controls to broader security operations and assurance.