Bot management products are easy to demonstrate and hard to evaluate, because any vendor can block a noisy script in a demo while the bots that cost you money are slow, distributed and built to look human. Offerings range from CDN and WAF add-ons with a bot score, through dedicated platforms with client-side telemetry and mobile SDKs, to fraud tools that score accounts and transactions. They now also decide which automation to let in: search crawlers, partner integrations and AI agents acting for customers. This vendor-neutral guide is for application security, fraud and platform teams writing requirements or running an RFP. Our Bot Management Guide explains the threats and controls; this page is about choosing a product.
Key takeaways
- Buy against named threats and flows. List the OWASP automated threats you face, such as OAT-008 Credential Stuffing, OAT-005 Scalping or OAT-011 Scraping, and the endpoints they hit, then score vendors on those.
- Coverage of every door beats detection claims. A product that protects the web form but not the mobile API moves the attack rather than stopping it.
- Good bots and AI agents need a policy, not an exception list. Require verification by reverse DNS, published ranges or signatures (the IETF Web Bot Auth work), with rules per class of crawler and agent.
- Friction and privacy are part of the product. Score challenge accessibility against WCAG 2.2 and the lawful basis for the device signals collected, not only block rates.
- Prove it on your own traffic in monitor mode. A four to six week trial with labelled outcomes is the only fair comparison.
Define requirements
- Threats and losses: map each problem to an identifier from the OWASP Automated Threats to Web Applications project and attach a cost: account takeover (OAT-007, OAT-008), fake accounts (OAT-019), card testing (OAT-001), voucher enumeration (OAT-002), inventory hoarding (OAT-005, OAT-021), scraping (OAT-011) and the infrastructure cost of automated traffic. See the Account Takeover Prevention Guide.
- Protected flows: inventory endpoints, not pages: login, password reset, one-time passcode sending, sign-up, checkout, gift card balance, search, pricing APIs and the mobile app's own API. OWASP API6:2023 Unrestricted Access to Sensitive Business Flows notes that APIs "consumed directly by machines" are often the easiest target.
- Channels: browsers, native iOS and Android apps, single-page apps calling JSON APIs, and partner APIs. Each needs a different signal source.
- Automation you want: search engines, monitoring, partners, accessibility tools and AI agents buying or booking for customers. Decide whether each class is allowed, rate limited, charged or blocked. See the Agentic Commerce Identity Guide.
- Constraints: edge latency budget, content security policy for injected scripts, data residency for signals, accessibility commitments, and which team operates the product.
Size changes the answer. A small organisation with a credential stuffing problem often gets most of the benefit from its CDN or WAF managed rules, a server-validated challenge on login and sign-up, MFA and per-account rate limits. Retail, ticketing, travel and financial services, where each successful bot transaction is monetised, usually need a dedicated product with client-side signals, mobile coverage and someone tuning it weekly.
| Deployment model | How it works | Limits to test |
|---|---|---|
| CDN or WAF add-on | Scores requests at the edge from network, TLS and header signals, sometimes with an injected challenge; fast to deploy | Little view of account outcomes; native app APIs give no browser signals; tied to one edge |
| Dedicated platform (edge, proxy or connector) | Client script and mobile SDK collect device and behavioural telemetry; a decision service scores each request | Integration effort; another processor of personal data; decision latency |
| Server-side scoring API | Your application or gateway sends request and session context and enforces the result | You build enforcement and choose fail-open or fail-closed |
| Fraud or identity platform | Bot score is one input to login, account and payment risk | Weak on scraping and inventory abuse that never touches an account |
The "WAF versus bot management" question comes down to this table. WAF rules look for malicious payloads; most automated threats send valid requests to legitimate functions, so what matters is whether the product sees behaviour across a session and an account.
Evaluation criteria
Weight criteria to your losses. An illustrative retail split is 35% detection on priority flows, 20% channel coverage, 15% response and operations, 15% good-bot handling, 15% privacy, accessibility and platform security.
| Criterion | What good looks like | Weak signs |
|---|---|---|
| Detection signals | Layers network reputation, TLS and HTTP fingerprints, browser checks, behaviour and business signals such as login success ratio; explains each decision | One opaque score; reliance on IP reputation, which residential proxies defeat |
| Challenges | Non-interactive by default, interactive only for a risk band; tokens validated server-side, single use, bound to session and action | Puzzles for everyone; tokens replayable across endpoints |
| Mobile SDKs | Use Google Play Integrity and Apple App Attest, verified on the server with a one-time challenge; documented fallback for unsupported devices | A secret embedded in the app; cached attestation verdicts |
| API protection | Policies per API operation, headless client coverage, per-key and per-account limits for partners | Only HTML pages protected |
| Good bots and AI agents | Verified-bot directory, reverse DNS and range checks, HTTP message signature verification, separate policies for training crawlers, search crawlers and user-initiated agents | Allowlisting by user agent; a single "AI bots on or off" switch |
| Response options | Log only, rate limit (HTTP 429), challenge, step-up via your identity provider, alternative content, block; set per flow and rule | Block or allow only; changes need vendor support |
| False positives | Per-decision explanations, feedback API for confirmed fraud and customers, support lookup by request ID, per-rule switch to log-only | No way to see why a customer was blocked |
| Accessibility | Meets WCAG 2.2 Success Criterion 3.3.8 Accessible Authentication (Minimum), level AA, with a non-cognitive alternative; audio CAPTCHA is not accepted as that alternative | Image puzzles as the only path; no conformance report |
| Privacy of signals | Inventory of every signal collected, retention, regional processing, no reuse beyond security and fraud, written into the processing agreement | Vague "device intelligence"; signals pooled for other purposes |
| Integration and operations | Decisions streamed to SIEM and fraud tools; rules API; SSO and MFA on the console; audit log; published latency and failure mode per flow | Dashboard only; shared admin accounts; unknown timeout behaviour |
Good bots, AI agents and verification
User-agent strings are trivially spoofed, so ask vendors to show verification. Google documents a reverse DNS lookup ending in googlebot.com, google.com or googleusercontent.com, confirmed by a forward lookup, or matching published files such as common-crawlers.json and user-triggered-agents.json (Google Search Central). The cryptographic route is the IETF Web Bot Auth working group's draft-ietf-webbotauth-httpsig-protocol: the bot signs requests with HTTP Message Signatures (RFC 9421) and a Signature-Agent header points to a key directory at /.well-known/http-message-signatures-directory. It is a working group draft (version 00, September 2026), so test support rather than assume it. robots.txt declares crawler policy, but RFC 9309 states that "These rules are not a form of access authorization", so the product must enforce your AI crawler policy itself.
Where identity fits
Most bot attacks end in an account, so the product should share signals with your identity stack: a login scored as automated should raise step-up risk in your identity provider, and confirmed takeovers should feed back. The 23andMe breach showed credential stuffing on about 18,000 accounts reaching data on nearly 7 million people. AI agents acting for customers need an identity you can verify and authorise, not a blanket bypass. See the Customer IAM (CIAM) Guide. The vendor's own access matters too: an edge or proxy deployment sees session cookies, login bodies and payment pages. Ask which vendor staff and systems can see request content, and treat decision-service and rules API keys as privileged machine credentials, scoped per environment and rotated. See the API Key Management Guide.
Questions to ask vendors
- For each priority flow and OAT identifier, which signals do you use, and what happens when the attacker uses residential proxies and real browsers on a device farm?
- How is a challenge token validated, how long is it valid, and can it be replayed on another endpoint? Show the server-side check.
- How do your SDKs verify Play Integrity and App Attest results on the server, and what happens on unsupported devices? Google says the Play Integrity API "works best when used alongside other signals" and that caching verdicts increases the risk of proxying (Android Developers).
- Show how you verify a search crawler, a partner and an AI agent, and how we set different policies for each. Do you verify Web Bot Auth signatures today?
- Which data do your script and SDK collect, where is it processed, how long is it kept, and is it used for anything except protecting us? In the UK the ICO says the PECR strictly necessary exception covers "preventing or detecting fraud", but "If you use it for any other purpose as well, the exception does not apply" (ICO).
- What is the non-cognitive path for a user who cannot solve a challenge, and can you share a WCAG 2.2 conformance report?
- How does our support team find why a customer was blocked, and release them?
- What happens per flow if the decision service is slow or down, and what is 99th percentile latency?
- Does the price count requests, endpoints, active users or app installs? How are attack spikes billed, and are verified good bots counted?
Red flags
- Detection rates from the vendor's own traffic, with no offer to measure on yours.
- No mobile SDK, or mobile protection that relies on a secret embedded in the app.
- Good-bot handling that trusts the user-agent header, or an AI crawler toggle with no verification behind it.
- Image puzzles with no accessible alternative; a W3C working group note prefers non-interactive approaches "because these pose no accessibility challenges" (W3C Inaccessibility of CAPTCHA).
- No signal inventory, unclear retention, or signals reused for advertising or data products.
- Per-request pricing with no attack allowance, so you pay more while under attack.
Proof of concept
- Pick flows and labels: choose two or three flows with measurable outcomes, such as web and mobile login, sign-up and a limited-stock checkout. Agree outcome labels (confirmed takeover, chargeback, fake account, genuine sell-through) and pass criteria first.
- Run in monitor mode: deploy every candidate log-only on the same traffic for at least two weeks, ideally across a known peak. No vendor enforces during the comparison.
- Allowlist good automation: verify your crawlers, monitors and partners and confirm none would have been blocked. Send a signed test agent if Web Bot Auth support is claimed.
- Red team it: with written authorisation, run a headless browser, a stealth-patched browser through residential proxies, direct calls to the mobile API and a replayed challenge token against test accounts. Record what each product caught.
- Measure friction: compare challenge rate, pass rate and conversion per flow, and test challenges with a screen reader and keyboard only.
- Enforce on one flow, then roll back: switch the leader to enforce on login, compare takeover reports and support contacts with the baseline, and confirm one switch returns the rule to log-only within minutes.
- Close out: remove losing candidates' scripts and SDK builds, revoke their API keys and get written confirmation of data deletion.
Keep tracking after go-live: account takeover rate per 100,000 active accounts, login success ratio, fake account rate, genuine sell-through on limited stock, false positive support contacts and the share of origin traffic from unwanted automation.
How NHI Mgmt Group can help
We provide independent requirements, RFP and evaluation support. Browse vendors in our products directory or contact us.
Related NHI Mgmt Group resources: Bot Management Guide · Account Takeover Prevention Guide · Agentic Commerce Identity Guide · Fraud Prevention Guide for Digital Businesses · CIAM Buyer's Guide