Notifications
Clear all
Topic starter
02/08/2026 11:27 am
TL;DR: A production-safe profiler that could work under gVisor, avoid perf_event_open, and stay below roughly 5% overhead was needed for an OCaml-based SAST engine, according to Semgrep, which led it to build Pyro Caml with Memprof, runtime events, and Rust FFI for continuous profiling. The deeper lesson is that observability choices are governance choices when performance tooling must fit real runtime and sandbox constraints.
NHIMG editorial — based on content published by Semgrep: Pyro Caml, a continuous profiler for OCaml
By the numbers:
- Only 5.7% of organisations have full visibility into their service accounts.
- 97% of NHIs carry excessive privileges, increasing unauthorised access and broadening the attack surface.
- 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools.
Questions worth separating out
Q: How should teams choose profiling tools for production services with strict runtime constraints?
A: Prioritise tools that can run safely inside the real production boundary, with low overhead and a failure mode that does not affect application correctness.
Q: Why do sandboxed workloads make observability harder to implement?
A: Sandboxing can remove the kernel hooks, system calls, or runtime access that many observability tools expect.
Q: What do security teams get wrong about low-overhead telemetry?
A: They often assume any telemetry is better than none, even when the measurement path distorts the system being measured.
Practitioner guidance
- Define the production boundary for observability Map where profiling can run safely in production, including sandboxed services, restricted workloads, and any environments where attaching standard profilers is impossible.
- Set an overhead budget before enabling continuous profiling Establish a hard ceiling for runtime impact, then test profiler configurations against that target under representative load.
- Separate stack capture from data export Keep the in-process sampling path minimal and move formatting, aggregation, and transmission into a different runtime or service.
What's in the full article
Semgrep's full post covers the implementation detail this post intentionally leaves for the source:
- The Memprof sampling design and why allocation-based profiling was chosen over standard time-based profilers.
- The OCaml runtime events pipeline used to move profiling data out of process with minimal overhead.
- The Rust FFI integration pattern that connects OCaml sampling to the Pyroscope backend.
- The production caveats, including FFI blind spots, truncated stacks, and plans for memory profiling.
👉 Read Semgrep's post on building Pyro Caml, a continuous profiler for OCaml →
Continuous profiling in OCaml: are your performance controls keeping up?
Explore further
02/08/2026 12:43 pm
Continuous profiling is a governance control, not just an engineering convenience. When a security product runs at scale, the ability to understand runtime behaviour becomes part of operational assurance. Semgrep’s approach shows that visibility tooling must fit the execution environment, especially when sandboxing and sensitive workloads limit what engineers can inspect directly. For IAM and security leaders, the lesson is that observability design should be treated like any other control dependency, because if it cannot run safely in production, it cannot support reliable governance.
A question worth separating out:
Q: How do you know if continuous profiling is working well enough?
A: It is working well enough when it consistently identifies bottlenecks that change tuning decisions without materially slowing the service. You do not need perfect fidelity, but you do need repeatable signals, acceptable overhead, and enough stack coverage to distinguish real hotspots from measurement artefacts.
👉 Read our full editorial: Continuous profiling in OCaml: what Pyro Caml changes for practitioners
