Security teams with a fixed testing budget are often told to pick between a breach and attack simulation (BAS) platform and a purple team programme, yet one tells you whether your controls still fire and the other teaches your people to see what the controls miss. BAS runs a library of safe, ATT&CK-mapped scenarios on a schedule and reports prevented, detected or missed per technique. Purple teaming puts attackers and defenders in the same room, emulating a chosen threat step by step so detection engineers can watch the telemetry, write or tune analytics and agree what "good" looks like. This guide compares the two as programme choices: threat model, architecture, failure modes, what each proves, how they combine into continuous control validation, and how to decide by maturity and size. It is for SOC managers, detection engineering leads and testing programme owners. It covers programme design only, not attack techniques or tooling.
Key takeaways
- BAS measures, purple teaming improves. BAS tells you at scale and on a schedule whether known behaviours are blocked or alerted; a purple team exercise changes the analytics, playbooks and skills that produce those results.
- Coverage is not quality. A green BAS heatmap can sit on top of brittle detections that catch one implementation of a technique; purple teams test the variations and the analyst response that a pass or fail cannot show.
- The validation loop needs both halves. CISA's loop is select a technique, align controls, test, analyse, repeat and tune; BAS automates the test and repeat steps, people do most of the analyse and tune steps.
- Purple teaming has formal standing in regulated testing. The ECB's TIBER-EU Purple Teaming Guidance (January 2025) makes a closure-phase purple team exercise a standard element of a TIBER-EU test.
- Sequence by maturity. Without basic logging and someone to act on misses, BAS produces red dashboards nobody fixes; without regression checks, purple team gains decay after the next change.
The short answer
If your main risk is drift (controls silently stop working after agent upgrades, policy edits, exclusions and SIEM changes), buy or build continuous automated testing first: that is the BAS model. If your main risk is blindness (you are not sure your analytics would catch a capable intruder, or analysts would recognise what they saw), invest in purple teaming first. Most organisations with a working SOC need both: purple teaming to raise detection quality on the techniques that matter, then BAS to keep checking that the gains survive. The Breach and Attack Simulation (BAS) Guide and the Purple Teaming Guide explain each discipline; the Breach and Attack Simulation vs Penetration Testing page covers BAS against human discovery testing, and Red Team vs Blue Team vs Purple Team covers team structure. This page is only about BAS versus purple teaming.
What each one is for
| Dimension | Breach and attack simulation | Purple teaming |
|---|---|---|
| Core question | Do our controls still prevent or detect these known behaviours today? | Can we see and respond to this adversary, and how do we get better at it? |
| Primary output | Pass, detect or miss per technique, per control, per asset, trended over time | New or tuned analytics, updated playbooks, documented telemetry gaps and a remediation list |
| Who does the work | A platform, owned by security operations or a validation team | Offensive operators, detection engineers, SOC analysts and threat intelligence, together |
| Cadence | Daily to weekly, and after change | Per exercise, typically monthly to quarterly, or a standing function |
| Depth per technique | One or a few safe implementations from the library | Several variations, discussed live, with log review at each step |
| Tests people and process | Only indirectly, if alerts are routed to analysts during runs | Directly: triage, escalation and the analyst's reading of evidence |
| Source of scenarios | Vendor or community library, often mapped to MITRE ATT&CK | Threat intelligence on actors relevant to you, emulation plans, incidents and test findings |
| Main cost driver | Licences, agent deployment and integration upkeep | Skilled people's time on both sides |
The two share tooling more than buyers expect. MITRE describes Caldera as an open-source platform for automated adversary emulation built on ATT&CK and lists red teaming, purple teaming and security validation among its uses. The MITRE Center for Threat-Informed Defense Adversary Emulation Library offers full plans for named groups, with machine-readable procedures for automation, and micro emulation plans for compound behaviours that it says can be run without a red team, manually or through automation. The same plan can drive a facilitated purple team session one month and an automated regression run the next. The difference is who interprets the result.
BAS: threat model, architecture and failure modes
Threat model: a known adversary using catalogued techniques against controls you already own. The failure BAS looks for is regression: an EDR policy moved to audit mode, a new exclusion, a disabled SIEM rule, a broken log forwarder, a proxy category changed during a migration.
Architecture in words: agents or virtual appliances on representative endpoints, servers and network segments execute benign stand-ins for attacker behaviour; simulated email, web and lateral traffic exercises gateways and network controls. The platform queries EDR, SIEM and gateway APIs to correlate what each control logged, blocked or alerted, and summarises results per technique, control and asset. Schedules and change triggers run scenarios again after deployments.
Failure modes:
- Library-shaped coverage: the heatmap reflects what the vendor wrote, not what threatens you. Techniques that matter to your sector may be absent or represented by a single variant.
- Detecting the simulator: analytics that key on the BAS agent's process name, hash or test markers pass every run and fail against a real intruder.
- Silent correlation errors: a broken API integration reports "not detected" for controls that fired, or "detected" from stale data. Results need periodic manual spot checks.
- Nobody owns the misses: results land in a dashboard rather than a ticket queue with a named control owner and a due date.
Purple teaming: threat model, architecture and failure modes
Threat model: a specific, capable adversary, chosen from threat intelligence, whose behaviours your detection stack and analysts may not recognise. The failure purple teaming looks for is blindness and misinterpretation: telemetry not collected, analytics too narrow, alerts raised but misread or closed.
Architecture in words: threat intelligence selects an actor and its techniques; operators prepare emulation steps under rules of engagement; detection engineers and analysts sit with them. For each step the operator executes, the defenders check whether telemetry arrived, whether an analytic fired, and whether the analyst would have escalated. Where a gap appears, the team records the evidence needed, writes or tunes a detection, and replays the step until it is caught. The Purple Team Exercise Framework (PTEF, version 4, MIT licence) describes this as combining cyber threat intelligence, red and blue teams, and distinguishes ad hoc exercises, an operationalised purple team and a dedicated purple team.
Failure modes:
- One-off theatre: a well-run day produces findings, but nobody turns fixed gaps into regression tests, so the next platform change undoes the work.
- Too few variations: the operator runs the same implementation the detection was written for and declares success.
- Prior knowledge bias: defenders know exactly when and where a step runs, so the exercise measures detection logic, not whether analysts would notice it in a normal shift. That is acceptable, but report it honestly.
- Dependency on a few people: results vary with the operator's and engineer's skill, and the programme stalls when they leave.
Continuous control validation: where the two meet
CISA's red team advisory AA24-326A recommends exercising, testing and validating your security programme against the ATT&CK behaviours in its advisories, in six steps: select a technique, align your security technologies to it, test them, analyse detection and prevention performance, repeat for all technologies, and tune people, processes and technologies based on the data. It also recommends continually testing your security programme at scale in a production environment. Mapped to that loop, the split of labour is clear.
| Loop step | BAS contribution | Purple team contribution |
|---|---|---|
| Select technique | Library and threat feed suggestions | Intelligence-led choice of actor and techniques that matter to you |
| Align controls | Integration map of which control should see what | Expected telemetry and data sources agreed per step |
| Test | Automated, repeatable, at scale across many hosts | Live, varied, with operators adapting to what defenders see |
| Analyse | Automatic correlation: prevented, detected or missed | Root cause: missing log source, brittle logic, triage error |
| Repeat | Scheduled and change-triggered reruns | Replay of failed steps until caught |
| Tune | Evidence for control owners | New analytics, playbooks and analyst training |
Continuous control validation is the programme that runs this loop permanently. Inside continuous threat exposure management it is the validation stage: see the Exposure Management and CTEM Guide. Where vendors sell fully automated "continuous purple teaming", read it as BAS with richer reporting unless people review variations and response, as the Continuous Automated Red Teaming (CART) Guide also notes for automated red teaming.
Coverage versus detection quality
The Center for Threat-Informed Defense's Summiting the Pyramid project argues that an ATT&CK heatmap is not enough. Its 2026 release measures implementation coverage, how many behaviourally distinct ways of performing a technique an analytic can observe, and detection quality, how robust and precise the signals behind that visibility are. This is the clearest way to explain the BAS versus purple teaming split to leadership.
- BAS answers breadth: across hundreds of techniques and thousands of hosts, which ones produce any prevention or alert today.
- Purple teaming answers depth: for the 20 or 30 techniques your priority actors use, whether detections key on the behaviour rather than a tool artefact, and whether a different implementation would still be seen.
- Reporting rule: never let a single BAS pass mark a technique as covered on the threat-informed defence map. Mark it "covered, one variant" until a purple team or detection engineer has tested variations.
Small, portable tests sit between the two. Atomic Red Team describes its library as small, highly portable detection tests mapped to ATT&CK; detection engineers use such tests during and after purple team sessions to confirm an analytic before it goes into the regression set. See the Detection Engineering Guide and Detection as Code Guide for how tuned analytics are versioned and tested.
Purple teaming in regulated threat-led testing
In the EU financial sector, purple teaming is now a defined part of threat-led testing. The ECB's TIBER-EU Purple Teaming Guidance, published January 2025, defines purple teaming as a collaborative activity involving the red team testers and the blue team, and sets out two uses:
- Limited purple teaming in the testing phase: a last resort to continue a test that would otherwise end early, proposed by the control team and approved by the test manager. Forms include catch-and-release, collaborative proof of concept and, in particular cases, a war game.
- Purple team exercise in the closure phase: a standard element on topics chosen jointly by testers and defenders, held after the replay exercise and no later than ten weeks after active red team testing ends. Forms include a tabletop discussion, re-exploration of attack steps on live systems while the blue team supplies defensive information, and exploration of alternative scenarios.
Outcomes feed the remediation plan. BAS has no equivalent role in TIBER-EU, but the closure-phase findings are exactly the scenarios that should become automated regression tests afterwards. See the DORA Guide for how threat-led penetration testing sits in DORA.
Decision guidance: which first, and how much of each
| Situation | Lead with | Why |
|---|---|---|
| Small team, outsourced SOC, few controls | Neither tool yet; a facilitated purple team day with your MDR provider | You need to know what your provider sees before you pay to measure it continuously |
| Mid-sized, in-house SOC, frequent endpoint and SIEM change | BAS for regression, quarterly purple team on priority actors | Change volume makes drift the dominant risk; quarterly sessions raise quality |
| Large, detection engineering function, several EDR or SIEM estates | Standing purple team plus BAS across estates | Breadth needs automation; depth needs a permanent team feeding it |
| Regulated entity facing TIBER-EU or DORA TLPT | Purple team capability first | Closure-phase purple teaming is expected; BAS evidence supports remediation tracking |
| Recent incident or red team showed analysts missed alerts | Purple teaming | The gap is people and process, which BAS does not exercise |
| Audit asks for evidence controls operate continuously | BAS | Timestamped, repeatable results per control suit control assurance |
Trade-offs to state plainly: BAS is cheaper per technique tested but shallow, and its value depends on integration upkeep. Purple teaming is expensive per technique but changes outcomes, and its value depends on retaining skilled people. Open emulation libraries and frameworks lower tooling cost for both; they do not remove the people cost of purple teaming or the integration cost of automation.
Running them as one programme
- Choose priority actors and techniques. Use threat intelligence to pick three to five actor profiles and their techniques; this list drives both the purple team calendar and the BAS assessment templates.
- Baseline with automation in audit mode. Run BAS scenarios for those techniques on a pilot ring of representative hosts, with results going to a report only. Fix integration and correlation errors before anyone trusts the numbers.
- Run a purple team exercise on the worst gaps. Take the missed and "detected but not alerted" techniques into a facilitated session. Record the telemetry, write or tune analytics, replay until caught, and document variations tested.
- Promote fixes into regression. Every closed gap becomes a scheduled BAS scenario or a portable test in the detection pipeline, tagged with the analytic it validates.
- Widen rings and add change triggers. Extend BAS to more hosts and segments, and trigger reruns after EDR policy, SIEM content and network changes.
- Close the loop monthly. Review BAS regressions with control owners, feed new intelligence and incident lessons into the next purple team scope.
- Roll back safely. Keep a kill switch for BAS agents and a per-scenario disable; in purple team sessions the rules of engagement name who can halt the exercise. Pause automation during incidents so test activity does not muddy investigations.
The Purple Team Exercise Plan Template provides the session paperwork.
Worked scenario
A mid-sized logistics company with an in-house SOC of eight analysts, one EDR and one SIEM, has budget for either a BAS licence or two external purple team engagements a year. Threat intelligence points to ransomware affiliates that rely on credential dumping, remote service abuse and backup deletion.
- Finding from a trial: a two-week BAS trial shows strong prevention on commodity malware but "logged, not alerted" on most credential access and lateral movement techniques.
- Decision: the SOC lead buys one external purple team engagement first, scoped to those behaviours, and builds in-house capability by pairing two analysts with the external operators.
- Result of the session: eleven new or tuned analytics, two missing log sources (domain controller audit policy and backup console logs) and a revised escalation playbook.
- Follow-on: in the next budget year the company buys BAS, scopes it first to the eleven analytics and the ransomware techniques, and schedules weekly runs plus change-triggered reruns. A later EDR upgrade that reset a policy to audit mode is caught within a week.
The order mattered: BAS first would have produced a long list of misses with nobody skilled to fix them; purple teaming alone would have let the EDR regression go unnoticed until an incident.
Where identity fits
Both approaches create privileged non-human identities. BAS platforms hold API credentials into EDR, SIEM, email and cloud consoles, often with read access to every alert and sometimes write access to policies; their agents run with local administrator or root rights. Purple team operators use test accounts and service accounts that must be time-bound and monitored. Treat these as privileged NHIs: inventory, least privilege, vault storage, rotation and removal when an engagement ends. Identity techniques themselves (credential access, token theft, service account abuse) deserve a place in both test sets. See the NHI Authentication Guide.
Detection and measurement
- Technique detection rate: share of priority techniques that produce an alert (not just a log) in the latest BAS run, trended weekly.
- Variant coverage: for priority techniques, number of distinct implementations tested and caught in purple team sessions.
- Time to restore coverage: median days from a BAS regression to a confirmed fix.
- Gap closure rate: purple team findings closed within the agreed window, and the share promoted into regression tests.
- Analyst response quality: in sessions where analysts are not pre-briefed, share of alerts correctly triaged and escalated.
- Correlation accuracy: share of sampled BAS results confirmed by manual review of the control's own logs.
Common failure patterns
- Buying BAS to avoid hiring: automation without detection engineers yields a growing list of misses and a stale dashboard.
- Excluding the BAS agent from controls: allowlisting the agent or its folders so it "runs cleanly" removes the very controls under test.
- Purple team findings with no regression: gaps closed in a session reopen after the next platform change and nobody notices.
- Heatmap reporting to the board: counting a technique as covered after a single library pass overstates protection; report variant coverage alongside.
- Running automation during an incident: simulated activity pollutes timelines and alerts; pause it under the incident process.
- Unmanaged test credentials: long-lived BAS API keys and purple team accounts left active after engagements become an attacker's foothold.
Framework mapping
| Framework | Reference | BAS | Purple teaming |
|---|---|---|---|
| NIST SP 800-53 Rev 5 | CA-7 Continuous Monitoring | Primary | Supporting |
| NIST SP 800-53 Rev 5 | CA-2(2) Specialized Assessments | Primary | Primary |
| NIST SP 800-53 Rev 5 | CA-8(2) Red Team Exercises | Supporting | Primary |
| NIST SP 800-53 Rev 5 | SI-4 System Monitoring; IR-3 Incident Response Testing | SI-4 | SI-4, IR-3 |
| NIST CSF 2.0 | DE.CM-09; DE.AE-02; ID.IM-02 | DE.CM-09, ID.IM-02 | DE.AE-02, ID.IM-02 |
| ISO/IEC 27001:2022 Annex A | 8.16 Monitoring activities; 5.35 Independent review of information security | 8.16 | 8.16, 5.35 |
| TIBER-EU | Purple Teaming Guidance (2025) | Not addressed | Closure-phase exercise, limited purple teaming |
Decision checklist
- Is our dominant risk drift after change, or blindness to capable intruders?
- Do we have people who can turn a missed technique into a working analytic?
- Have we chosen priority actors and techniques from threat intelligence?
- Can our EDR, SIEM and gateways expose the APIs a BAS platform needs?
- Does a regulator or scheme expect purple team activity in threat-led testing?
- Who will own BAS misses, and with what remediation deadline?
Practitioner checklist
- Agree one priority technique list that drives both BAS templates and purple team scope.
- Pilot BAS in report-only mode and verify correlation against control logs before trusting results.
- Never exclude BAS agents or test hosts from the controls being validated.
- Record telemetry, analytics and variations tested for every purple team step.
- Promote every closed purple team gap into a scheduled regression test.
- Trigger automated reruns after EDR, SIEM and network policy changes.
- Pause automated testing during live incidents.
- Store BAS API credentials in a vault, scope them to read where possible, and rotate them.
- Disable purple team test accounts at the end of each engagement.
- Report technique detection rate and variant coverage together to leadership.
Standards and references
- CISA: Enhancing Cyber Resilience, Insights from CISA Red Team Assessment (AA24-326A)
- ECB: TIBER-EU Purple Teaming Guidance
- NIST: SP 800-53 Rev 5 Security and Privacy Controls
- MITRE Center for Threat-Informed Defense: Summiting the Pyramid
- MITRE Center for Threat-Informed Defense: Adversary Emulation Library
- MITRE: Caldera
- SCYTHE: Purple Team Exercise Framework (PTEF)
- Red Canary: Atomic Red Team
Related NHI Mgmt Group resources: Breach and Attack Simulation Buyer's Guide, Detection Engineering Maturity Model, EDR Telemetry Guide, CTEM vs Penetration Testing.