Web application firewall actions expose the real tradeoffs at scale

At a glance

What this is: This is an analysis of how ngrok built and tuned WAF actions into its traffic policy system, with the key finding that safe scale required shared runtime design, dry-run validation, and selective rule exclusions.

Why it matters: It matters to IAM and security practitioners because runtime enforcement, observability, and policy tuning increasingly sit alongside identity and access controls in protecting exposed services and APIs.

By the numbers:

• ngrok blocked about 1.2% of requests from reaching upstream services after running its WAF actions for more than six months.
• The WAF actions triggered 98 different CRS rules, showing that attack patterns remained diverse rather than concentrated in one signature.
• ngrok said its compiled Coraza instance consumed around 25MB of memory, which shaped the decision to use a singleton per node.

👉 Read ngrok's analysis of how it built WAF actions into traffic policy

Context

Web application firewalls are not just signature filters. They sit in the traffic path, inspect HTTP content, and decide whether a request should be allowed, challenged, or denied based on application-layer behaviour. In practice, that means the control must balance detection quality, latency, memory use, and false-positive handling at the same time, especially when it protects customer-facing endpoints and APIs.

For IAM and NHI practitioners, the governance lesson is that exposed services are only as resilient as the controls around the request path and the secrets that front them. Where applications depend on service accounts, API keys, or other non-human identities to expose functionality, edge enforcement and identity governance become complementary controls rather than separate disciplines.

Key questions

Q: How should teams roll out a WAF without breaking legitimate traffic?

A: Start with detection-only mode, collect false-positive examples, and only block once you can explain the decisions. The safest rollout pairs rule tuning with request logging, path-specific review, and a rollback path for rules that collide with normal application behaviour.

Q: Why do WAFs become harder to operate as endpoint counts rise?

A: Because each compiled rule engine consumes memory, and body buffering adds pressure during inspection. At scale, the control has to balance detection depth against latency, garbage collection, and tenant isolation, which turns runtime design into an operational constraint.

Q: What do security teams get wrong about false positives in WAFs?

A: They often treat false positives as a sign that the entire control is wrong. In practice, false positives usually show where rules need path-aware tuning, exclusions, or better context, especially for documentation pages, downloads, and rich application responses.

Q: Who is accountable for tuning WAF rules when business traffic is blocked?

A: The accountable team is usually the one that owns the application edge policy, but the process should include security, platform, and application owners. WAF governance works best when rule changes, exceptions, and block-mode decisions are all reviewed in a controlled change process.

Technical breakdown

How CRS rules turn HTTP traffic into block or allow decisions

The OWASP Core Rule Set works by matching request and response content against patterns associated with common web attacks. In this model, a WAF does not need to understand the business logic of the application, only the structure and content of HTTP fields such as headers, parameters, and body content. Scoring-based evaluation matters because many real attacks are ambiguous at first glance. That is why anomaly scoring can reduce dependence on brittle one-rule, one-decision logic. A traffic-policy system that exposes request variables, expression logic, and deny actions can host a WAF without replacing the broader access control layer.

Practical implication: keep WAF decisions explainable so operators can distinguish real attack traffic from false positives quickly.

Why shared WAF state matters in multi-tenant platforms

At scale, a WAF is not just a detection problem, it is a memory and tenancy problem. Compiling rule engines for every endpoint can create large per-instance overhead, while request-specific context still needs isolation. The architecture ngrok describes uses a shared Coraza instance per node and per-request transaction state, which reduces duplicated memory while preserving tenant separation. That separation is essential because the engine must hold global rule state without leaking request data across customers. Disabling shared logging inside the engine and attaching decision data to per-request result variables are part of that safety model.

Practical implication: prefer shared rule engines with isolated transaction context when protection must scale across many endpoints.

Why dry-run mode is the correct first phase for WAF rollout

Dry-run is not a convenience feature, it is a governance control. Real traffic contains legitimate patterns that can resemble attack signatures, and the only safe way to learn those boundary conditions is to observe decisions before blocking. ngrok’s rollout surfaced false positives in documentation content and download paths, which led to targeted rule exclusions rather than wholesale disabling. That approach reflects a mature deployment model: detect first, measure false positives, then enforce. Without that sequence, security teams tend to convert attack detection into user-facing outage.

Practical implication: run WAF protections in observation mode first, then tighten enforcement only after false positives are understood.

NHI Mgmt Group analysis

WAF tuning is now a governance problem, not only a detection problem. The article shows that effective application-layer protection depends on observing real traffic, handling false positives, and making selective exclusions without losing control coverage. That shifts WAF operation into the same policy discipline as access governance, where decisions must be explainable and reversible. Practitioners should treat protection rules as governed controls, not static signatures.

Edge protection and identity governance meet at the point where secrets front exposed services. When applications expose endpoints through API keys, service tokens, and other non-human identities, a WAF becomes part of the control stack that protects those access paths. The identity angle is indirect but real: the service identity is what makes the endpoint callable, while the WAF controls what reaches it. Practitioners should align edge filtering with secret hygiene and lifecycle control.

Shared runtime enforcement is the only sustainable model for large endpoint fleets. A per-endpoint security engine may be simple to describe but becomes expensive to operate when traffic volume and tenant count rise. The article’s singleton-per-node approach illustrates a broader principle of security engineering: controls must be resource-aware to remain operable. That aligns with NIST CSF and CIS Controls thinking around resilient, scalable protective architecture. Practitioners should evaluate memory footprint as part of control design, not after deployment.

False positives reveal more about operational context than about rule quality alone. The same signatures that catch attack traffic can also catch ordinary product content, especially when applications serve documentation, downloads, or technical strings. That means WAF governance must include path-aware tuning and business-context review, not just signature updates. Practitioners should build a review loop that separates benign application language from hostile payloads before they escalate to block mode.

What this signals

Runtime enforcement now needs the same change discipline as identity controls. Once a WAF sits in front of exposed services, policy drift, rule exceptions, and observability gaps become part of the security model. Teams that already manage privileged access and non-human identities should recognise the pattern: controls fail quietly when the exception process is weak.

The next step for most programmes is not broader blocking, but better control visibility. That means mapping where request-layer protection depends on service identities, where exceptions are accumulating, and where application owners need a faster path to validate rule changes before they reach production.

For practitioners

• Run new WAF rules in observation mode first Start with dry-run or continue mode so the engine evaluates live traffic without denying it. Use that phase to identify false positives in documentation, downloads, and API responses before switching to blocking.
• Set explicit exclusions for known false-positive rules Track which CRS rule IDs map to legitimate traffic patterns in your environment, then exclude only those rules for the affected traffic classes or paths rather than disabling the full ruleset.
• Measure memory cost before scaling the control fleet Profile compiled rule-engine footprint and request-body buffering against the number of endpoints and tenants you expect to protect, then choose a shared runtime model if the per-instance cost is high.
• Tie deny events to operational alerts Export denial metadata into your logging and monitoring stack so on-call teams can inspect matched rule IDs, anomaly scores, and request context when traffic patterns change.

Key takeaways

• WAF effectiveness depends on operational tuning, not just rule coverage.
• ngrok’s rollout shows that shared runtime design and dry-run validation are necessary at scale.
• Security teams should treat WAF policy changes as governed changes that require context, logging, and rollback.

Key terms

• Web Application Firewall: A web application firewall inspects HTTP traffic and applies rules to detect or block malicious requests and responses. Unlike a network firewall, it understands application-layer content such as headers, parameters, and body fields, which lets it catch attacks like injection and path probing.
• Anomaly Scoring: Anomaly scoring is a detection method that assigns points to rule matches instead of making a single yes-or-no decision. It helps a WAF tolerate noisy traffic by combining multiple weak signals into one block decision, which is useful when individual signatures are not decisive on their own.
• Dry-Run Mode: Dry-run mode allows a security control to evaluate traffic and produce logs without actively denying requests. It is used to measure false positives, validate rule behaviour, and tune enforcement safely before switching the same control into blocking mode.
• False Positive: A false positive is a scanner result that looks like a secret but is not actually sensitive. In secret governance, false positives matter because they consume analyst time, weaken trust in alerts, and can delay response to the findings that truly change exposure and access risk.

What's in the full article

ngrok's full post covers the implementation detail this analysis intentionally leaves for the source:

• The exact Traffic Policy actions used to run CRS rules in request and response phases.
• The rule-exclusion workflow that removed specific false positives without disabling the entire WAF.
• The load-test approach used to set the 4KB body-processing limit.
• The logging and alerting pattern that sends deny metadata into Datadog for on-call response.

👉 The full ngrok post covers the CRS rollout details, false-positive tuning, and scale limits.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management. It is designed for practitioners who need to connect identity control with broader security operations.