Most WAF purchases are judged on a demo of blocked SQL injection, then fail in production on false positives, unprotected APIs and rules nobody tunes. A web application firewall (WAF) inspects HTTP traffic in front of an application; web application and API protection (WAAP) is the broader category that adds API discovery and schema enforcement, bot management, DDoS mitigation and often client-side script protection. This vendor-neutral guide covers how to define requirements, which capabilities matter, how to score them and how to run a proof of concept on your own traffic. It is for security architects, AppSec leads and procurement teams replacing an ageing WAF or buying WAAP for the first time.
Key takeaways
- Start from your traffic, not the datasheet. The mix of browser apps, public APIs, partner APIs and mobile back ends decides whether you need a WAF or a full WAAP.
- Measure false positives as hard as detection. A WAF left in monitor mode because it blocks customers protects nothing.
- APIs need a positive security model. Signature rules miss broken object level authorization (API1:2023); schema enforcement and per-client behaviour analysis do better.
- PCI DSS 4.0 makes it mandatory for card environments. Requirement 6.4.2 expects an automated solution in front of public-facing web applications that blocks or alerts, logs and stays current.
- The WAF is a privileged system. It holds TLS private keys, API tokens and a cloud role, so its own identity and the vendor's security belong in the evaluation.
Define requirements
Write these down before speaking to vendors. They decide the deployment model and which capability areas are mandatory.
- Estate: number of applications, domains and APIs; which are public, partner-facing or internal; whether OpenAPI definitions exist. If you cannot list your APIs, discovery is a requirement (OWASP API9:2023 Improper Inventory Management).
- Deployment model: cloud-delivered reverse proxy (DNS or CDN based), cloud provider native WAF attached to a load balancer or API gateway, appliance or virtual appliance, or a software engine such as ModSecurity running the OWASP Core Rule Set (CRS) in your ingress. Data residency and where TLS is terminated follow from this choice.
- Compliance drivers: PCI DSS 4.0 requirement 6.4.2 for public-facing applications in scope; 6.4.3 (payment page script inventory, authorisation and integrity) if you want the same platform to cover client-side scripts.
- Threat profile: injection and other OWASP Top 10:2025 classes, credential stuffing and scraping, API abuse of business flows (API6:2023), resource exhaustion (API4:2023) and application-layer floods (ATT&CK T1499.003 Application Exhaustion Flood).
- Operating model: who tunes rules and handles false positives (your team, the vendor's managed service, or a partner), and how changes move through CI/CD.
- Integration: SIEM and SOAR targets, log format and latency, infrastructure-as-code support, and ticketing for rule change requests.
Capability areas
| Area | What it should do | Why it matters |
|---|---|---|
| Core WAF (negative model) | Managed signatures for injection, XSS, path traversal and protocol abuse; anomaly scoring rather than single-rule blocking; per-rule exclusions scoped by path and parameter | Baseline for OWASP Top 10:2025 A05 Injection and similar classes |
| Positive security and API schema enforcement | Import OpenAPI definitions, reject undeclared endpoints, methods, parameters and types; learn schemas from traffic where none exist | Catches attacks that look like valid requests and shadow endpoints |
| API discovery and posture | Inventory endpoints from traffic, flag sensitive data in responses, unauthenticated endpoints and deprecated versions | API9:2023 inventory failures; feeds the API programme |
| API abuse and authorisation anomalies | Per-identity baselines to spot object ID enumeration and abuse of business flows | API1:2023 and API6:2023 rarely match a signature |
| Bot management | Classify automated clients, challenge or rate limit by behaviour, protect login and checkout | Credential stuffing, scraping, inventory hoarding |
| Rate limiting and DDoS | Per-client and per-route limits; layer 7 flood mitigation; network layer scrubbing for cloud-delivered services | API4:2023 and ATT&CK T1498 / T1499 |
| Virtual patching | Fast custom rules for a disclosed CVE, testable in log-only mode first | Covers the gap until a code or vendor fix ships |
| Client-side protection | Inventory and integrity monitoring of scripts on payment pages | PCI DSS 6.4.3 and skimming attacks |
| Logging and operations | Full request context per alert (rule ID, matched data, client identity) with sensitive field redaction; streaming export; rules as code | Tuning, investigation and audit evidence |
The OWASP CRS documentation is a useful vendor-neutral reference for how mature rule engines behave. Each matching rule adds to an anomaly score (by default 5 for critical, 4 error, 3 warning, 2 notice) and the request is blocked when the inbound total reaches the threshold, 5 by default. Paranoia levels 1 to 4 trade detection depth for false positives, and an executing paranoia level higher than the blocking level lets stricter rules log without blocking. Ask commercial vendors for the equivalent controls. See the Web Application Firewall Guide for rule design and the API Security Guide for API controls behind the gateway.
Evaluation criteria
| Criterion | Weight (example) | How to evidence it |
|---|---|---|
| Detection on your applications | 20% | Replay of your own DAST and penetration test findings through the candidate in blocking mode |
| False positive rate after tuning | 20% | Legitimate requests blocked per 10,000 during a mirrored or shadow period; hours of tuning needed |
| API protection depth | 15% | Schema import and enforcement, discovery accuracy against your known API list, detection of BOLA test cases |
| Operability | 15% | Rules and exclusions as code, staged rollout, per-rule log-only mode, one-click rollback |
| Bot and DDoS coverage | 10% | Results of controlled credential stuffing and layer 7 load tests |
| Logging and integration | 10% | Log fields, redaction, delivery latency to SIEM, API for automation |
| Platform and vendor security | 5% | Key custody model, admin MFA and SSO, audit of admin actions, assurance reports |
| Commercials | 5% | Pricing basis (requests, applications, bandwidth) modelled on peak and attack traffic |
Adjust the weights: a small organisation with a few web apps and no staff to tune rules should weight managed rules and managed service quality higher; a large estate with hundreds of APIs should weight discovery, schema enforcement and rules as code higher. The Vulnerability Management Buyer's Guide uses the same scoring approach.
Questions to ask vendors
- How is anomaly or confidence scoring tuned per application, and can we run stricter rules in log-only mode alongside the blocking set?
- How narrowly can an exclusion be scoped (rule, parameter, path, method), and is every exclusion versioned and attributable?
- Which OpenAPI versions can you import, what happens to requests that violate the schema, and how do you learn a schema where none exists?
- How do you detect broken object level authorization and business flow abuse when every request is syntactically valid?
- How quickly are managed rules released for a newly exploited CVE, and how are we notified before they move to blocking?
- Where are our TLS private keys stored, who at your company can access them, and do you support keyless or customer-held key options?
- Which request fields are logged by default, and how are passwords, tokens and card data redacted?
- What fails open or closed if your service or the inspection engine is degraded, and can we choose?
- How are your own administrative credentials, API keys and cloud roles protected, and what incidents have you disclosed?
Red flags
- Detection claims without your traffic: high scores on generic test suites but no willingness to run in blocking mode against your applications.
- Exclusions only by disabling whole rules: the CRS project warns that broad exclusions and edited rule files create gaps and maintenance debt.
- "API protection" that is only rate limiting: no schema enforcement, no discovery, no per-identity behaviour.
- Logs that capture secrets: full bodies or headers stored without redaction, which turns the WAF into a store of credentials and card data.
- Over-privileged deployment: a cloud role or service account for the WAF with data access it does not need. In the Capital One breach a misconfigured WAF's role credentials could read customer data in S3.
- Vague answers on key custody: in the Imperva breach a stolen administrative AWS key exposed Cloud WAF customers' API keys and SSL certificates, which customers then had to rotate.
- Pricing that punishes attacks: per-request billing without caps for attack traffic, which encourages turning protection off.
Proof of concept
- Choose three to five targets: one high-traffic browser application, one public API with an OpenAPI definition, one API without a definition and, if in scope, a payment page.
- Deploy in monitor or mirrored mode first: run for at least one full business cycle (including month-end or peak events) and record every alert.
- Tune and record effort: write scoped exclusions as code, count hours and changes needed to reach an agreed false positive rate.
- Switch to blocking for a pilot slice: one host or a percentage of traffic, with a documented rollback (DNS or routing change, or a per-application policy toggle) tested in advance.
- Attack it under controlled conditions: replay prior penetration test findings, OWASP Top 10:2025 test cases, BOLA tests against the API, credential stuffing with test accounts and a rate-limited layer 7 load test agreed with the vendor and your hosting provider.
- Test a virtual patch: ask the vendor or your team to write a rule for a known CVE in your stack, deploy it log-only, confirm it catches the test and not normal traffic, then enforce. The OWASP Virtual Patching Cheat Sheet describes this cycle.
- Verify logging and evidence: confirm alerts arrive in the SIEM with rule ID, matched data and client identity, sensitive fields are redacted, and the output would satisfy a PCI DSS 6.4.2 assessor (blocking or immediately investigated alerts, audit logs, current rules).
- Score and decide: apply the weighted criteria, and track after go-live: percentage of applications in blocking mode, false positives per 10,000 requests, median time to deploy a virtual patch, and API endpoints discovered that were not in the inventory.
For layer 3 and 4 volumetric attacks see the DDoS Protection Guide, and for automated abuse the Bot Management Buyer's Guide. Primary sources used here: the OWASP CRS anomaly scoring and paranoia level documentation, the OWASP Top 10:2025, the OWASP API Security Top 10 2023, the OWASP Virtual Patching Cheat Sheet and the PCI DSS 4.0 requirement 6 text as reproduced by Microsoft Learn. In NIST SP 800-53 Rev. 5 terms a WAF supports SC-7 Boundary Protection, SI-4 System Monitoring and SI-10 Information Input Validation.
How NHI Mgmt Group can help
We provide independent requirements, RFP and evaluation support. Browse vendors in our NHI and AI products directory or contact us.
Related NHI Mgmt Group resources: Web Application Firewall Guide · API Security Buyer's Guide · Application Security Guide · PCI DSS 4.0 Guide · AWS Security Guide