A tabletop exercise shows whether people know the plan; it does not show whether they can execute it, whether the tools work, or how long containment and recovery really take. Drills, functional exercises and full-scale exercises close that gap by having teams perform real tasks under time pressure: isolating hosts, revoking credentials, restoring from backup, standing up out-of-band communications and handling regulator and media injects. This guide covers the operations-based exercise types defined in the US Homeland Security Exercise and Evaluation Program (HSEEP) and NIST SP 800-84, how to build a progressive exercise programme, the roles and documents needed to run one safely, how to evaluate performance, and how to measure improvement. It is written for incident response leads, resilience managers and CISOs who already run tabletops. For discussion-based exercises, see the Tabletop Exercise Guide.
Key takeaways
- Discussion tests plans; operations test capability. HSEEP separates discussion-based exercises (seminar, workshop, tabletop, game) from operations-based ones (drill, functional, full-scale).
- Build up in steps. Run drills on single tasks before a functional exercise, and functional exercises before a full-scale one. Each step finds problems that would derail the next.
- Measure time and outcome, not opinions. Operations-based exercises can time revocation, isolation and restore, which turns lessons into numbers.
- Safety is designed in. A control cell, a master scenario events list, trusted agents, an exercise-stop phrase and a no-play list keep live actions from harming production.
- Close the loop. An after-action report and improvement plan with owners and dates is the output; an exercise without tracked corrective actions is theatre.
Exercise types and what each tests
HSEEP (2020 doctrine, which superseded the 2013 version) defines a drill as an operations-based exercise often used to validate a single operation or function, a functional exercise as one designed to test and evaluate capabilities and functions in a realistic, real-time environment, and a full-scale exercise as typically the most complex and resource-intensive type. NIST SP 800-84 describes functional exercises as letting personnel validate operational readiness by performing their duties in a simulated operational environment, where deployments may be simulated or may actually occur.
| Type | Category | Cyber example | Typical duration | Tests |
|---|---|---|---|---|
| Seminar or workshop | Discussion | Walk-through of a new ransomware playbook | 1 to 3 hours | Understanding, plan content |
| Tabletop | Discussion | Executive scenario on data extortion | 2 to 4 hours | Decisions, roles, plan gaps |
| Game | Discussion | Team-based decision game with consequences per move | Half day | Decision trade-offs |
| Drill | Operations | Revoke all sessions and rotate keys for a test service account; restore one server from immutable backup; call tree activation at 3am | 15 minutes to 2 hours | One task, its timing and tooling |
| Functional exercise | Operations | SOC, IR and identity teams respond to a simulated intrusion in a test environment, while control cell plays executives, regulators and the attacker | 4 hours to 2 days | Coordinated functions in real time |
| Full-scale exercise | Operations | Multi-team response including business continuity, communications, suppliers and external partners, with real system actions in agreed scope | 1 to 3 days | End-to-end capability, including third parties |
Purple team exercises and breach and attack simulation overlap with drills on the detection side: they test whether controls see attacker techniques, not whether the organisation can run an incident. Use them alongside this programme. See the Purple Teaming Guide and the Breach and Attack Simulation (BAS) Guide.
Choosing the right exercise
- New or changed plan: start with a workshop or tabletop. Operations-based exercises against an unread plan mainly test confusion.
- One technical step is critical and untested: run a drill. Examples: mass password and session reset, disabling a federation trust, isolating a cloud account, failover of the identity provider, restoring Active Directory.
- Several teams must work together under time pressure: functional exercise. This is the main tool for testing SOC to CSIRT handover, identity and cloud team coordination, and decision-making with incomplete information.
- Regulator, board, supplier or sector expectations: full-scale exercise, often annually or every two years. Large cross-border examples include ENISA's Cyber Europe series, run every two years since 2010; the 2026 edition, the eighth, focused on rail and maritime transport.
Small versus large organisations: a small organisation gets most value from quarterly drills on its few critical recovery tasks plus an annual functional exercise built on a free package, such as the UK NCSC's Exercise in a Box (20 exercises across 12 topics) or CISA's Tabletop Exercise Packages adapted with live tasks. Large organisations need a multi-year programme with dedicated planners and an exercise environment.
Building a progressive programme
A programme is a multi-year schedule where each exercise builds on the last. A typical cycle for a mid-sized organisation:
| Quarter | Exercise | Objective |
|---|---|---|
| Q1 | Tabletop (executives) and drill (call tree, out-of-band channel) | Decision authority; reach every role within [30] minutes |
| Q2 | Drills: credential and session revocation; backup restore of a tier 1 system | Measure time to revoke and restore against targets |
| Q3 | Functional exercise: ransomware with data theft in the test environment | SOC, IR, identity, infrastructure and communications coordination over [8] hours |
| Q4 | Review and full-scale planning; drill retests of failed items | Close corrective actions; confirm improvements |
| Year 2 | Full-scale exercise with suppliers, insurer and external counsel | End-to-end response and recovery of a critical service |
Choose scenarios from the threats most likely to hit [Organisation]: credential theft leading to cloud compromise (T1078 Valid Accounts), phishing (T1566 Phishing), ransomware (T1486 Data Encrypted for Impact), supplier compromise and insider misuse. See the Tabletop Exercise Scenario Pack for scenario material that can be converted into operations-based exercises.
Roles, documents and safety controls
NIST SP 800-84 names an exercise director, controllers who manage the flow, data collectors who observe and record player actions, and simulators who represent people and organisations not taking part. HSEEP adds the exercise planning team and trusted agents: planning team members, generally not players, who must not reveal scenario details.
| Role or document | Purpose in a cyber exercise |
|---|---|
| Exercise director | Overall authority, including the decision to pause or stop |
| Control cell (controllers and simulators) | Delivers injects; plays the attacker, executives, regulators, media, customers and suppliers |
| Evaluators and data collectors | Record timings and actions against objectives using evaluation guides |
| Trusted agents | Scenario designers from each team who keep it secret and keep it realistic |
| Safety officer | Watches live actions against the no-play list and can halt any action |
| Exercise Plan (ExPlan) | For players: objectives, scope, rules, schedule; no scenario details |
| Master Scenario Events List (MSEL) | Timed list of injects and expected player actions, used by controllers |
| Controller and Evaluator Handbook | Detailed guidance for control and evaluation staff |
| Exercise Evaluation Guides (EEGs) | Standard forms to record performance against each objective |
| After-Action Report and Improvement Plan (AAR/IP) | Strengths, areas for improvement and tracked corrective actions |
Safety controls for live actions (practice recommendations):
- Scope by environment: run functional exercises in a test or cyber range environment where possible; in production, limit live actions to test accounts, test hosts and pre-agreed systems.
- No-play list: systems and actions that must never be touched (for example production payment systems, safety systems, real customer communications).
- Exercise-stop phrase: a fixed phrase any participant can use to halt play if a real incident or safety issue arises; agree how a real incident during the exercise is handled.
- Labelling: every inject and message starts and ends with "EXERCISE EXERCISE EXERCISE" so it is never mistaken for a real event.
- Change control: live drills on production (for example a backup restore) go through normal change approval with a rollback plan.
- Notify third parties: tell managed service providers, the identity provider's support team and any monitoring vendor in advance, so they do not escalate exercise activity as a real incident.
Choosing the exercise environment
Where the live actions happen decides both realism and risk. Most programmes use a mix.
| Environment | Realism | Risk to operations | Best for |
|---|---|---|---|
| Production, limited to test objects (test users, test service accounts, a sacrificial VM) | High: real consoles, real permissions, real approval routes | Low if scoped and change-controlled | Revocation, isolation and rotation drills |
| Staging or pre-production copy | Medium to high | Low | Restore drills, failover, functional exercises with technical play |
| Separate test tenant (identity provider, cloud account, SIEM workspace) | Medium: same tools, simplified data | Very low | Identity and cloud compromise scenarios with seeded artefacts |
| Commercial or in-house cyber range | Medium: realistic networks, not your own systems | None | Technical skills, forensic triage, red versus blue play |
| Simulated (control cell supplies outputs) | Low for technical steps, high for decisions | None | Executive and communications play inside a functional exercise |
A useful rule: decisions can be simulated, but the actions that recovery depends on (restores, credential revocation, break-glass access, out-of-band activation) should be performed for real at least once a year, in production where safe. The first real execution during an incident is the worst time to learn that a backup console authenticates through the compromised identity provider, or that only one person knows how to rotate a signing key.
Designing injects for operations-based exercises
Injects in a functional or full-scale exercise should require action, not discussion. Each MSEL entry records: time, source (which simulator), channel (ticket, phone, email, SIEM alert), content, expected player action, and the objective it tests.
- Technical injects: seeded alerts in a test SIEM tenant, artefacts on a test host for forensic triage, a leaked API key in a test repository, an attacker-created service principal in a test cloud tenant.
- Identity injects: a compromised administrator account requiring session revocation, MFA reset and review of new role assignments; a service account whose key must be rotated without breaking a dependent workload. See the Identity Incident Response Playbook.
- Business injects: a ransom note with a deadline, a journalist's call, a regulator's question about the 72-hour notification, an insurer requesting details, a key supplier reporting it is also affected.
- Friction injects: the primary chat platform is unavailable, the on-call manager does not answer, the backup console uses the compromised identity provider. These test out-of-band communications and break-glass access. See the Out-of-Band Communications for Incident Response Guide.
Running and evaluating the exercise
- Design: set objectives (no more than [five] per exercise), scope, participants and safety rules. NIST SP 800-84 uses design, development, conduct and evaluation as the four phases.
- Develop: write the ExPlan, MSEL, Controller and Evaluator Handbook and EEGs; prepare the environment and seed artefacts; run a dry run with the control cell.
- Brief: players receive the ExPlan, rules, stop phrase and communication channels.
- Conduct: controllers deliver injects on schedule and adjust pace; evaluators record timestamps for each expected action.
- Hot wash: immediately after play, gather participants' views while fresh.
- Evaluate: compare observed performance with objectives and targets; write the AAR within [two] weeks.
- Improve: the improvement plan lists each corrective action with an owner and due date, tracked in the same system as audit findings. Retest failed items with a drill. See the Tabletop Exercise After Action Report Template.
Rollback: every live action in the MSEL has a documented reversal (re-enable test account, restore isolated host, revert firewall change) and an owner who confirms the environment is back to normal before the exercise closes.
Detection and measurement
Operations-based exercises produce measurable times. Track them across exercises and against targets:
- Time to declare an incident from the first actionable inject.
- Time to contain: isolate a host, disable an account, revoke sessions and rotate a key.
- Time to restore a tier 1 system from backup, compared with its recovery time objective.
- Time to assemble the response team through the out-of-band channel.
- Percentage of exercise objectives met, and percentage of corrective actions closed by their due date.
- Repeat findings: issues found in more than one exercise, which signal that corrective actions are not working.
Framework mapping
| Requirement | Reference | How operations-based exercises meet it |
|---|---|---|
| Incident response testing | NIST SP 800-53 Rev. 5 IR-3, IR-3(2) Coordination with Related Plans, IR-3(3) Continuous Improvement | Functional exercises test the plan with related continuity and communication plans and feed metrics into improvement |
| Contingency plan testing | NIST SP 800-53 Rev. 5 CP-4, CP-4(1) Coordinate with Related Plans | Restore drills and full-scale exercises test recovery |
| Improvement from exercises | NIST CSF 2.0 ID.IM-02 | AAR/IP actions, including exercises with suppliers and third parties |
| Improvement from operations | NIST CSF 2.0 ID.IM-03 | Lessons from real incidents feed the next exercise scenario |
| Incident response exercises | CIS Controls v8.1 Safeguard 17.7 | Annual exercises covering communication, roles and decision making |
| ICT readiness for business continuity | ISO/IEC 27001:2022 Annex A 5.30 | Evidence that recovery capability has been tested |
Common failure patterns
- Skipping steps: jumping from a first tabletop to a full-scale exercise, which collapses on basic problems a drill would have found.
- Too many objectives: evaluators cannot observe them all and the AAR becomes impressions.
- Scripted success: injects that leak the answer, or players who know the scenario because trusted agents shared it.
- Exercise on the compromised channel: teams coordinate in the same chat and email the scenario assumes the attacker controls.
- No rollback: test accounts left disabled or firewall changes left in place, so the exercise causes a real outage.
- Same players every time: the incident lead and two senior analysts take part in every exercise while deputies, night shift staff and new joiners never do, so the capability exists only when those people are available. Rotate roles and run at least one exercise outside office hours each year.
- Actions not tracked: the AAR is filed, nothing changes, and the next exercise repeats the same findings.
Practitioner checklist
- Map current exercises to HSEEP types and identify which capabilities have never been tested hands-on.
- Write a multi-year programme that progresses from drills to functional and full-scale exercises.
- Name an exercise director, control cell, evaluators, trusted agents and a safety officer for each operations-based exercise.
- Produce an ExPlan, MSEL, Controller and Evaluator Handbook and evaluation guides.
- Agree a no-play list, stop phrase, labelling rule and rollback for every live action.
- Drill credential and session revocation, backup restore and out-of-band activation at least [twice] a year.
- Time every critical action and compare with targets and recovery time objectives.
- Issue an AAR/IP within two weeks with owners and dates, and retest failed items.
Standards and references
- NIST: SP 800-84 Guide to Test, Training, and Exercise Programs for IT Plans and Capabilities
- FEMA: Homeland Security Exercise and Evaluation Program (HSEEP) Doctrine
- NIST: SP 800-53 Rev. 5
- NIST: Cybersecurity Framework 2.0
- CISA: Tabletop Exercise Packages
- NCSC: Exercise in a Box
- ENISA: Cyber Europe Programme
Related NHI Mgmt Group resources: Tabletop Exercise Guide · Tabletop Exercise Scenario Pack · Purple Team Exercise Plan Template · Crisis Management Guide · Building a CSIRT Guide