An AI bug bounty programme pays or credits researchers for reporting security flaws in AI products, but most of them draw a firm line between a security vulnerability and a jailbreak, and researchers and programme owners who miss that line waste weeks. A prompt that makes a chatbot say something it should not is usually a content or safety issue routed to a separate channel; a prompt hidden in an email that makes an assistant forward a victim's files is a security vulnerability with a reward. This guide explains how AI bug bounty and vulnerability disclosure programmes work, how the major AI providers scope and route reports, the misconceptions that cause rejected submissions, and how to scope an AI vulnerability disclosure programme (VDP) for your own AI features. It is for security researchers, product security teams and anyone adding AI to an existing bounty. General programme design is covered in the Bug Bounty and Vulnerability Disclosure Guide; this page covers what changes for AI.
Key takeaways
- Security impact decides scope: AI programmes reward flaws that change a victim's data, account or actions, or leak sensitive information; output that only affects the tester's own session is generally out of scope.
- Jailbreaks usually go elsewhere: providers route jailbreaks, harmful content and hallucinations to safety or in-product feedback channels rather than the security bounty.
- Indirect prompt injection is the common payable case: instructions planted in content the AI processes (email, documents, web pages, tool output) that cause data exfiltration or unwanted actions are treated as security bugs.
- Routes differ by provider: each publishes its own platform, scope and safe harbour terms, and a flaw that affects several providers needs a report to each.
- Your own AI features need explicit scope: say in your VDP which AI issue types are in scope, how they are rated and where content issues go, or researchers will guess.
What it is
An AI bug bounty is a vulnerability reward programme whose scope covers AI-specific attack classes: prompt injection, sensitive data exfiltration through model output or tools, unauthorised actions by agents, training data extraction, model theft and manipulation of model behaviour. Some providers run a dedicated AI programme; others fold AI products into their main bounty with AI-specific rules. A VDP is the unpaid equivalent: a published route, scope and safe harbour for reporting, without guaranteed rewards.
The defining feature is the split between two kinds of finding:
| Finding type | Typical example | Where it usually goes |
|---|---|---|
| Security vulnerability in an AI product | Hidden instructions in a shared document cause the assistant to send a victim's data to an attacker | Security bounty or VDP |
| Conventional web or API flaw in an AI service | Access control bypass on a conversation-sharing endpoint, SSRF through a browsing tool | Security bounty or VDP |
| Model theft or training data extraction | Recovering confidential model weights or sensitive training examples | Security bounty where the provider lists it |
| Jailbreak or policy bypass in the tester's own session | Getting the model to produce content it is meant to refuse | Safety or content reporting channel |
| Hallucination or factually wrong output | A confident but false answer | Product feedback |
| System prompt disclosure with nothing sensitive in it | Revealing the assistant's instructions | Usually out of scope unless the prompt holds secrets or enables further impact |
The published policies below show the same logic: attacks that change a victim's account or data, or leak sensitive information, are treated as security findings, while jailbreaks, hallucinations and content produced in the tester's own session go to a content or safety channel.
How it works
The provider programmes below are listed alphabetically; the order is not a ranking. Each summary is taken from the provider's own published policy or programme page as of November 2026, and scopes change, so read the current page before testing.
Anthropic
- Route: security vulnerabilities are submitted through Anthropic's HackerOne form, linked from its Responsible Disclosure Policy (last updated 14 February 2025).
- Scope: Anthropic-owned or controlled internet-facing systems, including anthropic.com, its subdomains and the web or mobile apps hosted on them. Red teaming and adversarial testing of models, and content issues with prompts and responses, sit outside this security policy.
- Safety issues: reports of safety issues and jailbreaks are welcomed at [email protected] with enough detail to reproduce.
- Researcher terms: one vulnerability per report, a separate report to each AI provider affected, acknowledgement within three business days, and safe harbour for good-faith research with unconditional disclosure.
- Route: AI vulnerabilities are reported through Google Bug Hunters under the dedicated AI Vulnerability Reward Program, announced in October 2025, which has its own rules page.
- Scope: the AI VRP announcement lists categories including rogue actions that modify a victim's account or data, sensitive data exfiltration, phishing enablement, model theft, cross-account context manipulation, access control bypass, unauthorised product usage and cross-user denial of service (volumetric testing is not permitted).
- Safety issues: direct prompt injection, jailbreaks and alignment issues are outside the AI VRP; Google asks for content issues to be reported through the in-product feedback channel.
- Researcher terms: rewards and qualifying rules follow the Bug Hunters programme rules for each product.
Microsoft
- Route: the Microsoft Copilot Bounty is run by MSRC; reports go through the MSRC Researcher Portal under "Copilot, AI+ML, and LLMs", with the conversation ID obtained using the /id chat command.
- Scope: Copilot on the web, in Microsoft Edge, in Windows, in the mobile apps and on WhatsApp and Telegram, tested with a personal account. Award categories include inference manipulation and inferential information disclosure alongside conventional classes such as code injection and improper access control. Model manipulation was removed as an award category on 16 September 2025.
- Safety issues: content-related issues are submitted through the same portal with "AI derived harm" selected; hallucinations, system prompt leaks and prompt injection with no impact beyond the attacker are out of bounty scope.
- Researcher terms: severity follows Microsoft's Vulnerability Severity Classification for AI Systems and the Online Services Bug Bar; reports must be Critical, Important or Moderate and reproducible on the latest version.
OpenAI
- Route: OpenAI's bug bounty is managed on Bugcrowd, with participation rules on the Bugcrowd engagement page; its Coordinated Vulnerability Disclosure Policy also accepts encrypted reports to [email protected].
- Scope: the disclosure policy covers internet-facing infrastructure operated by OpenAI, including openai.com and subdomains, openai.org, api.openai.com and OpenAI-operated public cloud resources.
- Safety issues: questions about whether a finding fits the policy go to [email protected] before testing proceeds; check the Bugcrowd brief for the current treatment of model behaviour issues.
- Researcher terms: the policy asks for confidentiality until release is authorised, aims to authorise within 90 days, and treats good-faith research under the policy as authorised.
Reward amounts, tiers and eligibility rules are on each programme page and change often, so this guide does not reproduce them.
Why it matters
- Agents turn content bugs into security bugs: once an assistant can read mail, browse, call tools or act on a user's behalf, an injected instruction can move data or trigger actions with the user's privileges. Those paths are exactly what AI bounties reward. See the OWASP LLM Top 10 LLM06 Excessive Agency Guide.
- Agent credentials are NHIs: many payable findings end with an agent using an OAuth token, API key or connector account it should not have used in that context. Programme owners should rate these by the scope of the identity abused. See the Agent Identity and Access Guide.
- External testers find what internal teams miss: indirect prompt injection paths depend on the many content sources an AI reads, and outside researchers try combinations an internal red team will not. Bounties complement, not replace, the AI Red Teaming Guide work.
- Clear routing protects researchers: a published safe harbour and route reduce the risk that good-faith AI testing is treated as abuse of terms of service.
Common misconceptions
- "Any jailbreak is a bounty": output policy bypasses in your own session are generally routed to safety or content channels. A jailbreak becomes a security report when it is the first step in a chain with victim impact.
- "System prompt leakage always pays": it is commonly out of scope unless the prompt contains secrets or the disclosure enables a further attack.
- "Prompt injection against myself counts": injection whose only effect is on the attacker's own session has no victim; programmes look for injection delivered through content a victim's AI processes.
- "One report covers every model": where an issue affects several providers, each needs its own report under its own terms; Anthropic's policy says so explicitly.
- "Volume testing is fine because it is just prompts": denial of service, rate-limit abuse and large automated campaigns are excluded or restricted by most policies. Read the testing rules first.
- "A VDP for the website covers our AI features": unless the policy names AI issue types, researchers cannot tell whether prompt injection or retrieval data leakage is in scope or how it will be rated.
Related concepts
Scoping an AI VDP or bounty for your own product
| Decision | Options | Guidance |
|---|---|---|
| Which AI issue types are in scope | Indirect prompt injection with impact; data exfiltration via output or tools; unauthorised agent actions; cross-user context manipulation; model or data extraction | Tie each to a demonstrable victim impact; reference OWASP LLM Top 10 IDs so reports use shared names |
| Where content and safety issues go | Separate safety inbox, in-product feedback, or a tracked category in the same portal | Publish the route so out-of-scope reports still reach the right team |
| How to rate severity | Existing bug bar extended with AI classes; a published AI severity classification | Rate by the data and identities reachable, not by how clever the prompt was |
| Reproducibility rules | Single success, repeated success rate, conversation or trace ID | Ask for a trace or conversation ID and repeat attempts, since model output is non-deterministic |
| Testing limits | Personal or test accounts only, request caps, no volumetric testing | Provide test tenants or accounts with spending caps so researchers do not touch real users' data |
| Third-party models | Forward to the model provider, or handle as integration flaw | Say which issues you fix (your prompts, tools, permissions) and which you pass to the model provider |
- Disclosure process: AI reports still follow ISO/IEC 29147 for receipt and disclosure and ISO/IEC 30111 for internal handling. See the ISO/IEC 29147 and 30111 Vulnerability Disclosure and Handling Guide and the Vulnerability Disclosure Policy Template.
- Shared vocabulary: the OWASP Top 10 for LLM Applications 2025 (LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM06 Excessive Agency, LLM07 System Prompt Leakage) and MITRE ATLAS technique AML.T0051 LLM Prompt Injection give triage teams consistent labels. See the OWASP Top 10 for LLM Applications Guide.
- Internal testing first: run a structured assessment such as the OWASP AI Testing Guide before opening a bounty, so the first wave of reports is not dominated by issues you could have found yourself.
- Platforms: if you plan to buy a managed programme, see the Bug Bounty and VDP Platform Buyers Guide.
Measuring an AI programme
- Valid AI report rate: share of AI submissions accepted as in scope; a low rate usually means the scope text is unclear.
- Misrouted reports: safety or content reports arriving in the security queue, and the reverse.
- Time to triage and fix: median days for AI findings compared with your other findings.
- Repeat classes: the same injection path reappearing after a model or prompt change, which signals missing regression tests.
Practitioner checklist
- Read the provider's current policy and programme brief before testing an AI product; scopes change often.
- Test only with your own or provided test accounts and stay within published rate and volume limits.
- Show victim impact: data moved, account changed, action taken or sensitive information exposed.
- Include conversation or trace IDs, model and product versions and a reproduction success rate.
- Send jailbreak and content findings to the provider's safety or content channel, not the security queue.
- Report multi-provider issues to each affected provider separately.
- For your own product, list AI issue types, severity rules and the content route in the VDP.
- Rate agent findings by the scope of the identity and tokens abused, and fix the tool permissions, not only the prompt.
- Add every fixed AI finding to a regression test suite.
Standards and references
- Anthropic: Responsible Disclosure Policy
- Google Bug Hunters: Announcing Google's new AI Vulnerability Reward Program
- Google Bug Hunters: AI Vulnerability Reward Program rules
- Microsoft MSRC: Microsoft Copilot Bounty Program
- OpenAI: Coordinated Vulnerability Disclosure Policy
- Bugcrowd: OpenAI bug bounty engagement
- OWASP GenAI Security Project: Top 10 for LLM Applications 2025
- ISO: ISO/IEC 29147 Vulnerability disclosure
Related NHI Mgmt Group resources: Bug Bounty and Vulnerability Disclosure Guide · Vulnerability Disclosure Policy Template · Prompt Injection Guide · AI Red Teaming Guide