Continuous Automated Red Teaming (CART) Guide: How CART Works and Where It Fits

A penetration test or red team tells you how exposed you were on the days it ran, and the environment starts drifting away from that answer the moment the report is delivered. Continuous automated red teaming (CART) is the label for tools and services that run adversary-style testing on a schedule or continuously, so that new exposures, broken controls and detection gaps are found in days rather than at the next annual test. The term is used loosely, and it overlaps with breach and attack simulation (BAS) and with the validation stage of continuous threat exposure management (CTEM). This guide gives vendor-neutral definitions, a comparison of CART, BAS, penetration testing and red teaming, the safety controls needed to run automated attacks against production, how to measure results and how to act on them. It is written for security programme owners, vulnerability management leads and SOC managers. It does not cover attack techniques or tool commands.

Key takeaways

• CART is a delivery model, not a standard. There is no standards body definition; treat it as automated, repeatable adversary emulation against your real environment, and check what each offering actually does.
• It complements human testing rather than replacing it. Automation is good at breadth and repetition; human red teams are better at novel paths, judgement and social engineering.
• It belongs in the validate stage of CTEM, confirming which exposures are really exploitable and whether detection and response work.
• Production safety is the main design problem: scope allowlists, safe actions, rate limits, kill switches, credential handling and deconfliction with the SOC must be agreed before the first run.
• Value comes from closing the loop: every validated path or missed detection needs an owner, a fix and an automatic retest.

What CART means, and what it does not

NIST defines a red team exercise, in SP 800-53 Rev. 5 and SP 1800-21B, as a simulated adversarial attempt to compromise organisational missions or business processes in order to give a comprehensive assessment of security capability. CART takes that adversarial perspective and makes it automated and recurring. Vendor descriptions vary; one vendor learning page, for example, describes CART as regular, automated simulations of real-world attacker tactics, techniques and procedures, and contrasts it with traditional red teaming that is typically done once or twice a year. In practice offerings sold as CART usually combine some of the following:

• External attack surface discovery: continuous enumeration of internet-facing assets, domains, cloud services and exposed services from an outsider's view.
• Automated exploitation checks: safe confirmation that a discovered weakness is actually reachable and exploitable, rather than only flagged by a scanner.
• Attack path analysis: chaining weaknesses, credentials and permissions to show how an attacker could move from an exposure to a valuable asset.
• Internal adversary emulation: automated execution of ATT&CK-mapped behaviours from inside the network, often from an assumed breach starting point.
• Detection validation: checking whether each action generated telemetry and an alert in the SIEM or EDR.

Open-source projects show the building blocks. MITRE's Caldera is an open-source platform built on ATT&CK that performs automated adversary emulation for red teaming, purple teaming and security validation, and is moving to the Apache Software Foundation as Apache Caldera (Incubating). Red Canary's Atomic Red Team is a library of small, portable detection tests mapped to ATT&CK. The MITRE Center for Threat-Informed Defense Adversary Emulation Library publishes machine-readable emulation plans designed for automation tools.

What CART is not: it is not a vulnerability scanner with a new name, it is not a substitute for a threat-led red team that tests people and process, and it is not something that can safely be pointed at an environment without scoping.

BAS vs CART vs penetration testing vs red teaming

The UK NCSC's penetration testing guidance makes the argument for continuous approaches directly: a test can only validate that systems are not vulnerable to known issues on the day of the test, gaps can persist for a year or more between tests, and regular functional testing of controls should continue between them. See the Breach and Attack Simulation (BAS) Guide, the Penetration Testing Guide and the Red Teaming Guide.

Where CART fits in CTEM

Gartner describes continuous threat exposure management as five stages: scope the attack surface, discover assets, vulnerabilities and misconfigurations, prioritise the threats most likely to be exploited, validate how attacks might work and how systems would respond, and mobilise people and processes to act. Gartner's article frames validation as confirming that attackers could actually exploit a vulnerability, mapping possible attack paths to an asset and checking that the response plan is fast and strong enough. It also predicts that organisations prioritising security investment on a continuous exposure management programme will be three times less likely to suffer a breach; treat that as Gartner's forecast, not measured evidence.

See the Exposure Management and CTEM Guide.

Safety controls in production

Automated attacks against live systems can cause outages, lock accounts, trigger real incident response or expose data. The same governance principles regulated red team schemes use apply: the Bank of England's CBEST guide requires a risk assessment before work begins, reviewed regularly, and gives the control group power to order a temporary halt if there is risk of disruption to an important business service. For automation, translate those principles into technical guardrails:

• Explicit scope allowlist: IP ranges, domains, cloud accounts, subscriptions and applications in scope, maintained by the asset owner. Default deny for anything discovered but not approved, especially third-party and SaaS assets you do not own.
• Safe action catalogue: agree which action classes are allowed (discovery, authentication checks, non-destructive exploitation proofs) and which are forbidden (destructive actions, denial of service, encryption, real data exfiltration). Use marker files instead of real data.
• Rate and time limits: throttle requests, avoid account lockout thresholds, and run heavy tests in agreed windows for fragile systems such as OT, legacy or payment systems, or exclude them.
• Kill switch: a tested way to stop all activity within minutes, with named people authorised to use it, as the CBEST control group can.
• Deconfliction with the SOC: tag test traffic or sources so analysts can confirm activity is the platform, without creating blanket suppression rules that a real attacker could hide behind.
• Change and incident integration: register the platform as a standing change, and pause runs automatically during real incidents or change freezes.
• Cleanup: every run lists the artefacts it created (files, accounts, scheduled tasks) and removes them; removal is verified.
• Legal and supplier authorisation: written authorisation from the system owner, and from cloud and hosting providers where their terms require it. The Adversary Emulation Library, for example, states that its content should be used only with prior, explicit authorisation.

Where identity fits

A CART platform is itself a high-risk non-human identity. Internal emulation agents run with system privileges, and platforms usually hold credentials for directories, cloud accounts and SaaS APIs so they can map attack paths. If the platform is compromised, its credentials and its map of your weaknesses are exactly what an attacker wants.

• Scope the platform's identities: dedicated service accounts and cloud roles with the minimum read permissions needed, separate from human admin accounts. See the Service Account Security Guide.
• Vault and rotate: store platform credentials and API keys in a secrets manager, rotate them, and alert on their use from anywhere other than the platform. See the API Key Management Guide.
• Test identity paths explicitly: include leaked-credential and service-account starting points in the plan, because identity paths are the routes automated discovery often reveals and humans rarely have time to check exhaustively.
• Protect the results: attack path data is sensitive; restrict access, and treat the SaaS tenant as a privileged system with phishing-resistant MFA for its administrators.

Measuring detection and exposure

CART produces a lot of data. Choose a few metrics that show whether exposure is falling and detection is improving, and report trends rather than raw counts.

Government red team reports explain why detection coverage needs to be measured, not assumed. In CISA's AA24-193A, a known-bad, indicator-based approach to detection missed alternate techniques and EDR alerts were not reviewed daily; in AA24-326A an Active Directory account-creation alert had been disabled and a compensating web application firewall was left in monitor-only mode. Continuous checks are designed to find exactly that kind of silent failure. CISA's AA24-326A also recommends that organisations test their security controls against the ATT&CK techniques described in the advisory, which is a practical starting set of scenarios. See the Detection Engineering Guide.

How to implement CART

1. Define the questions: decide whether the priority is external exposure, internal attack paths, detection validation or all three. This drives the choice between CART, BAS or both.
2. Agree governance: name a programme owner, asset owners, the SOC contact and who can stop testing. Write the rules of engagement and the safe action catalogue.
3. Start passive and external: begin with discovery and non-intrusive validation of internet-facing assets. Compare results with your asset inventory; the differences are often the first findings.
4. Pilot internal emulation in a ring: deploy to a non-production or low-risk segment first, run a small set of ATT&CK techniques, and confirm the SOC sees and deconflicts them correctly.
5. Expand by ring: extend to production segments one at a time, with rate limits and change windows, watching for side effects such as account lockouts or EDR performance issues.
6. Integrate with ticketing: send validated findings to the owning team with evidence and a severity aligned to your vulnerability management policy, and trigger an automatic retest when the ticket closes.
7. Calibrate with humans: run periodic purple team sessions to review what automation found and missed, and commission human red teams for objectives automation cannot test.
8. Roll back if needed: if a run causes impact, use the kill switch, roll back created artefacts, record the incident and adjust the safe action catalogue before resuming.

Decision guidance

• Choose BAS first when the main concern is whether existing controls (email gateway, EDR, web proxy, DLP) still work, and when detection maturity is low enough that you need a library of known behaviours to tune against.
• Choose CART first when the external attack surface changes quickly (frequent cloud deployments, acquisitions, many SaaS tenants) and the main concern is finding exploitable exposure before attackers do.
• Keep human testing for business logic flaws, social engineering, novel chains, and for regulated threat-led testing such as CBEST, TIBER-EU or DORA threat-led penetration testing, which automation does not satisfy.
• Small organisations: a managed external exposure and validation service plus an annual penetration test is usually more realistic than running an internal emulation platform without staff to triage it.
• Large organisations: expect to run CART, BAS and human red and purple teams together, with a single backlog and shared ATT&CK coverage reporting.
• AI systems: automated testing of models and agents is a separate discipline with its own tooling. See the AI Red Teaming Guide.

Framework mapping

Common failure patterns

• Buying a label: assuming "CART" means internal emulation and detection validation when the product only does external discovery, or the reverse. Test claims in a proof of concept.
• Unbounded scope: automated discovery finds assets the organisation does not own, such as supplier or shared SaaS infrastructure, and tests them without authorisation.
• Blanket SOC suppression: whitelisting the platform's sources so widely that real attacker activity from the same hosts is ignored.
• Findings without owners: a dashboard of validated exposures that no team is accountable for. Exposure numbers rise and the tool is blamed.
• Over-privileged platform accounts: domain admin or broad cloud roles given to the platform for convenience, creating a new crown jewel.
• Dropping human testing: cancelling red team engagements because the dashboard is green, and losing the people and process findings automation cannot produce.

Practitioner checklist

• Write down which questions CART must answer and which stay with BAS, penetration testing and red teaming.
• Maintain an owner-approved scope allowlist and default-deny everything else.
• Agree a safe action catalogue that forbids destructive actions and real data exfiltration.
• Test the kill switch and name who can use it.
• Set up SOC deconfliction that does not create broad suppression rules.
• Give the platform least-privilege, vaulted, rotated credentials and alert on their misuse.
• Pilot internally in rings, starting outside production.
• Route validated findings to owners with evidence and automatic retest on closure.
• Report four to six trend metrics, including detection coverage by ATT&CK tactic and regression rate.
• Review automation results with human testers at least quarterly.

Standards and references

• NIST: SP 800-53 Rev. 5
• NCSC: Penetration testing
• CISA: AA24-193A, Red Team Operations Against a Federal Civilian Executive Branch Organization
• CISA: Enhancing Cyber Resilience: Insights from CISA Red Team Assessment of a US Critical Infrastructure Sector Organization
• MITRE: Caldera
• MITRE Center for Threat-Informed Defense: Adversary Emulation Library
• Gartner: How to Manage Cybersecurity Threats, Not Episodes
• Bank of England: CBEST Threat Intelligence-Led Assessments Implementation Guide

Related NHI Mgmt Group resources: Breach and Attack Simulation (BAS) Guide · Exposure Management and CTEM Guide · Purple Teaming Guide · Assumed Breach Testing Guide · AI Red Teaming Guide