Promptfoo Guide: Configure Red Team Scans and CI Gates

Every change to a prompt, model, retrieval source or tool definition can quietly reopen a prompt injection or data leakage path in an LLM application, and teams need tests that run as routinely as unit tests. Promptfoo is an open source command-line tool and library for evaluating and red teaming LLM-based applications: it runs declarative test cases with assertions, generates adversarial test cases from red team plugins and strategies, and reports the results by vulnerability and framework. This guide covers the configuration file, the red team plugins and strategies, data handling choices, CI gates and reporting, and how to turn findings into fixes. It is for application security engineers, AI platform teams and developers testing systems they own or are authorised to assess. Test methodology is covered in the AI Red Teaming Guide. Facts were checked against the Promptfoo repository and documentation on 8 November 2026.

This is an independent guide from NHI Mgmt Group, based on Promptfoo's public documentation as of November 2026. It is not sponsored or reviewed by Promptfoo or OpenAI. Product features change, so confirm settings against Promptfoo's current documentation.

Key takeaways

• Two workflows, one config. promptfoo eval runs your own test cases and assertions; promptfoo redteam run generates adversarial cases from plugins and strategies and runs them against the target, both driven by promptfooconfig.yaml.
• Plugins choose the risk, strategies choose the delivery. Plugins (for example bola, bfla, rbac, pii, indirect-prompt-injection) generate payloads; strategies (for example basic, jailbreak:meta, jailbreak:composite, base64, multi-turn) decide how they are sent.
• Decide where generation and grading run. Without your own model key, red team generation and grading use Promptfoo's hosted inference at api.promptfoo.app; setting your own provider or PROMPTFOO_DISABLE_REMOTE_GENERATION=true changes that.
• Exit codes make CI gates easy. eval returns exit code 100 when any test fails or the pass rate is below PROMPTFOO_PASS_RATE_THRESHOLD, and 1 for other errors.
• Treat configs as code. Custom JavaScript and Python assertions, providers, transforms and hooks execute code, so the security policy advises running untrusted configs and pull requests only in isolated runs with scoped secrets.

What the tool does

Promptfoo is MIT licensed and distributed through npm (npm install -g promptfoo), Homebrew and pip, or run without installing via npx promptfoo@latest. The README states that Promptfoo is now part of OpenAI and remains open source and MIT licensed; OpenAI announced the acquisition on 9 March 2026. On 8 November 2026 the repository's package.json showed version 0.125.1, and the CI documentation lists Node.js 22.22.0 or later (Node.js 24 LTS recommended). The documentation describes evals as running locally, communicating directly with the configured model provider.

Configuration building blocks

The red team flow

The configuration docs describe three steps: promptfoo redteam init creates a starting config, promptfoo redteam run generates test cases and runs them (it combines redteam generate and redteam eval so the cases stay in sync with the config), and promptfoo redteam report shows the results. Generated suites record metadata.generation with a timestamp and token usage, which helps when you need to show what was tested and at what cost.

Defender use cases

• Agent authorisation testing: the agent red teaming guide pairs the rbac, bola and bfla plugins to test whether an agent can be talked into reaching objects or functions outside the user's rights. Its mitigation advice matches identity practice: deterministic, non-LLM permission checks, granular RBAC on tools and least privilege for agent operations. See the AI Agent Authorisation Guide.
• Prompt injection in RAG and tools: the indirect-prompt-injection plugin and injection strategies test whether instructions hidden in retrieved content change behaviour. The Prompt Injection Guide covers the design controls.
• Data leakage: pii plugins and custom policy plugins test for disclosure of personal data, credentials and confidential content.
• MCP and tool ecosystems: the MCP security testing docs describe testing an MCP server through a trusted client, tool poisoning in multi-server setups and direct MCP testing. Our MCP Security Testing Guide covers the wider method.
• Prompt regression: plain evals with assertions catch quality and safety regressions when a prompt or model changes, the cheapest gate to run on every pull request.
• Model upgrade acceptance: the OWASP LLM Top 10 page suggests running identical security suites across model versions or providers and requiring new models to pass a standard suite before deployment.

Deployment and safe use

Choose a data handling mode first

Some plugins and strategies are documented as remote-only (marked in the plugin docs), including the harmful:* and bias plugins, ssrf, bola, bfla, indirect-prompt-injection and strategies such as goat and jailbreak:composite. Check the plugin list against your chosen mode before scoping, and use self-hosted inference where those plugins are needed on sensitive systems. Set PROMPTFOO_DISABLE_TELEMETRY=1 to turn telemetry off, and use --no-share or PROMPTFOO_DISABLE_SHARING=true where results must not be uploaded. The docs list the hosts used in remote mode (api.promptfoo.app, api.promptfoo.dev, a.promptfoo.app) for firewall rules.

Phased rollout

1. Authorise. Agree targets, plugins, volumes and windows with the system owner; check hosted model terms. Use the Red Team Rules of Engagement Template.
2. Write the purpose well. A specific redteam.purpose (users, data, tools, what the agent must never do) produces far more relevant tests than an inferred one.
3. Pilot with a small plugin set. Start with a framework preset such as owasp:llm or a handful of plugins, low numTests, against staging; review grader verdicts by hand.
4. Secure credentials. Use test-only provider keys with spending limits, held in your CI secret store or secrets manager. Do not paste real credentials or auth headers into the target setup forms: the data handling docs say values entered there are sent to hosted services. See the API Key Management Guide.
5. Isolate untrusted inputs. Run configs, prompt packs or pull requests from untrusted contributors only in isolated runners with secrets scoped to that run, as the security policy advises.
6. Keep the local UI local. promptfoo view, the MCP server and setup pages are single-user development tools; if you expose them, put network restrictions and authentication in front.
7. Promote to CI. Add eval gates on every change and scheduled red team runs; tag runs with --tag ci.run-id=... and --tag git.sha=....
8. Roll back. If a gate is too noisy, lower its scope rather than removing it, and revoke keys when an engagement closes.

Detection and hardening value

CI gates

promptfoo eval exits with code 100 when at least one test fails or the pass rate falls below PROMPTFOO_PASS_RATE_THRESHOLD (default 100 per cent), and 1 for other errors; PROMPTFOO_FAILED_TEST_EXIT_CODE overrides the failure code. The CI documentation shows examples for GitHub Actions, GitLab CI and Jenkins, and caching via PROMPTFOO_CACHE_PATH to cut cost. The GitHub Action (promptfoo/promptfoo-action@v1) runs a before and after evaluation for changed prompt files and takes the provider key from repository secrets. Give the workflow token the narrowest permissions it needs.

Reporting

The red team report groups findings by vulnerability and framework and assigns each a risk score from 0 to 10, built from impact, exploitability, human factor and complexity, with ranges following CVSS v3.x and v4.0 severity levels. Use redteam.frameworks to show only the frameworks your organisation reports against.

Promptfoo runs support NIST SP 800-53 SA-11 (Developer Testing and Evaluation), CA-8 (Penetration Testing) and NIST CSF 2.0 PR.PS-06 (secure software development integrated and monitored).

Metrics

• Eval pass rate per pipeline and the number of builds blocked by a gate.
• Open red team findings by risk score band, and median days to fix and retest.
• Coverage: share of LLM applications with a red team run in the last quarter and an eval gate in CI.
• Grader agreement: share of sampled verdicts confirmed by a reviewer.
• Data mode compliance: share of runs on sensitive systems using own or local providers.

Common failure patterns

• A vague purpose. Generic attacks miss the application's real tools and data.
• Not checking the data mode. Teams assume everything runs locally while generation and grading use hosted inference by default.
• Real secrets in setup forms or configs. Credentials end up in hosted services, shared links or repositories.
• Running fork pull requests with production keys. Code-executing assertions and providers run with whatever secrets the runner holds.
• Gates that only warn. A red result that never blocks a merge stops being read.

Alternatives

• Probe-based scanners: broad libraries of fixed and adaptive probes against a model endpoint.
• Scripted adversarial frameworks: code-first multi-turn campaigns for deep testing of a single system.
• Manual expert red teaming: for business logic and high-impact agents; see the OWASP GenAI Red Teaming Manual Guide.
• Continuous managed testing: see the Continuous Automated Red Teaming (CART) Guide.

Practitioner checklist

• Written authorisation covers each target, plugin set and test window.
• The data handling mode is chosen and recorded for each system before the first run.
• redteam.purpose describes users, data, tools and forbidden actions.
• Test-only provider keys live in the CI secret store; none in configs or setup forms.
• Untrusted configs and fork pull requests run in isolated runners with scoped secrets.
• Telemetry and sharing settings match policy (PROMPTFOO_DISABLE_TELEMETRY, --no-share).
• Eval gates block merges via exit code 100 with an agreed pass rate threshold.
• Red team runs are tagged with run ID and commit SHA and reports are archived.
• Findings map to OWASP and ATLAS IDs with an owner and retest date.

Standards and references

• Promptfoo: Repository
• OpenAI: OpenAI to acquire Promptfoo (9 March 2026)
• Promptfoo: Documentation
• Promptfoo: Red team configuration
• Promptfoo: Data handling and privacy
• Promptfoo: CI/CD integration
• OWASP: Top 10 for LLM Applications 2025
• MITRE ATLAS: LLM Prompt Injection
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: AI Red Teaming and Testing Tools Buyer's Guide · AI Red Teaming Guide · AI Agent Authorisation Guide · MCP Security Testing Guide