Boards pay for penetration tests and red team exercises every year, yet many cannot say what those tests proved, what changed afterwards or whether the same weaknesses came back. This briefing is for directors, chief executives, audit and risk committee members, CISOs and general counsel who approve the budget, scope and risk appetite for authorised security testing. It explains what offensive security buys the organisation in business terms, why a clean test report is not the same as being secure, where regulation now sets the frequency, and how to test management's answers. It stays at board level and treats testing purely as an assurance activity carried out with written permission. The operating detail is in the Penetration Testing Guide, the Red Teaming Guide and the Offensive Security Programme Maturity Model.
The risk in one paragraph
Offensive security means paying skilled, authorised people or tools to behave like an attacker so the organisation learns where it would fail before a real attacker shows it. It covers penetration tests of systems and applications, red team exercises against detection and response, purple team sessions that improve detections, breach and attack simulation between exercises, and bug bounty or vulnerability disclosure programmes that invite outside researchers. The board's risk is not a lack of testing; most organisations do some. It is that testing is bought to satisfy a customer or auditor, scoped to avoid the systems that matter, reported to one person, and closed without fixing root causes, so the organisation spends money and still learns of its weaknesses from an attacker. A second risk is legal and operational: testing without clear written authorisation, safe limits and control over the credentials and data testers handle can itself cause an incident. The board influences the outcome through three choices: what questions testing must answer, who is accountable for acting on findings, and how much risk the organisation will accept when a finding is not fixed.
Why this is a board-level issue now
- Regulators now set testing clocks for financial services. The EU Digital Operational Resilience Act (DORA), applying from 17 January 2025, requires financial entities other than microenterprises to make sure appropriate tests are run at least yearly on all ICT systems and applications supporting critical or important functions (Article 24(6)). Entities identified for advanced testing must carry out threat-led penetration testing (TLPT) at least every three years (Article 26(1)), and where they use internal testers they must contract external testers every three tests (Article 26(8)). Article 5(2) makes the management body responsible for defining, approving and overseeing the ICT risk arrangements. See the DORA Guide.
- Supervisors run their own threat-led schemes. In the UK, CBEST is a regulator-led assessment overseen by the PRA, the Bank of England's FMI Directorate and the FCA; the Bank's implementation guide says the average CBEST project takes about 9 to 12 months, and remediation is supervised afterwards. In the EU, the ECB's TIBER-EU framework was updated in 2024 to align with the DORA regulatory technical standards on TLPT. See the Threat-Led Penetration Testing Guide.
- Government red teams keep finding the same board-level failure. In advisory AA24-326A, CISA describes a three-month red team assessment of a US critical infrastructure organisation. The team used Kerberos abuse such as golden tickets and DCSync without being detected, alerts were generated but not reviewed, and legacy hosts without endpoint detection gave long-term access. CISA notes that leadership had accepted the risk of known outdated software and miscalculated its likely impact and likelihood. That is a governance finding, not a technical one.
- Boards are expected to gain assurance, not just receive reports. The UK Cyber Governance Code of Practice, published on 8 April 2025 with the NCSC, frames directors' duties as gaining assurance on risk, strategy, incident planning and oversight, including at least annual exercising of the incident response plan. Independent testing is one of the few sources of evidence that is not produced by the team being assessed.
- Authorisation is a legal line, not paperwork. In the UK, section 1 of the Computer Misuse Act 1990 makes it an offence to cause a computer to perform a function with intent to secure access the person knows is unauthorised; the Bank of England requires CBEST testers to hold permission for in-scope systems for that reason. In the US, the Department of Justice announced on 19 May 2022 that federal prosecutors should not charge good-faith security research under the Computer Fraud and Abuse Act, which matters for disclosure programmes but does not replace a contract. See the Computer Misuse and Anti-Hacking Laws Guide.
What each activity buys the board
Each form of testing answers a different question. Buying the wrong one wastes budget and produces false comfort. NIST SP 800-53 Rev. 5 makes the same distinction: penetration testing (CA-8), independent testers (CA-8(1)) and red team exercises (CA-8(2)) are separate from vulnerability monitoring and scanning (RA-5).
| Activity | Question it answers for the board | What it does not tell you | Typical cadence |
|---|---|---|---|
| Vulnerability scanning | Do we have known flaws on the assets we scan? | Whether flaws can be chained, or what we do not scan | Continuous or weekly |
| Penetration test | Can a skilled tester exploit weaknesses in this defined scope? | Whether we would detect a real attack; anything out of scope | Annual and on major change |
| Red team exercise | Would we detect and stop a realistic adversary reaching a critical asset? | A complete list of weaknesses; it follows one path | Every one to three years once basics are in place |
| Purple team session | Do our detections work for specific techniques, and how do we fix them? | Surprise; defenders know what is coming | Quarterly or after each major test |
| Breach and attack simulation | Are our controls still working after this week's changes? | Creative, multi-step attacks a human would find | Continuous |
| Bug bounty or vulnerability disclosure | What do outside researchers find in our public services? | Internal exposure; coverage depends on researcher interest | Always on |
| Threat-led penetration testing | Does our resilience hold against intelligence-based scenarios on critical functions, to a supervisor's standard? | Everyday hygiene; it is a deep test of a few paths | As the regulator sets, for DORA at least every three years |
The UK NCSC describes penetration testing as a check on the organisation's own vulnerability management, comparable to an external audit of routine financial controls, and advises that the organisation should largely know what testers will find before they find it. A report full of surprises is evidence that internal processes are failing, not that the test was good value. See Breach and Attack Simulation vs Penetration Testing and Bug Bounty vs Penetration Testing.
Buying assurance, not reports
The value of a test is decided before it starts. Three decisions sit with senior management rather than the testing team.
- Scope by business risk: the NCSC recommends that risk owners take part in scoping and that the organisation shares its current vulnerability assessment with testers so they can judge its accuracy. A scope that excludes the payment platform, the identity provider or a recent acquisition because they are "too sensitive" leaves the board with assurance on the systems that matter least.
- Independence and accreditation: recognised schemes give buyers a baseline. CHECK is the NCSC scheme for testing public sector and critical national infrastructure systems; CBEST requires threat intelligence and penetration testing providers accredited by the Bank of England who are also CREST members. Rotation matters too: DORA's rule on external testers every three tests is a useful benchmark outside finance.
- Ownership of the outcome: the NCSC is clear that risk assessment and decisions on fixes remain the organisation's responsibility and that mitigation should not be wholly outsourced to the test team. Every finding needs a named business owner, not just a ticket in the security team's queue.
Price follows depth. A narrowly scoped external test costs far less than a months-long red team, and the board should expect management to explain which question each line of the testing budget answers. See How Much Does a Penetration Test Cost and the Penetration Testing Provider Buyer's Guide.
Five questions for the board to ask
- What question did each test in the last year answer, and who chose the scope? Expect a short list linking each test to a risk on the register, a major change or a regulatory duty, with risk owners involved in scoping. "The supplier recommended it" is not an answer.
- How many findings from the last test are still open, and how many had we seen before? A credible answer gives open high-severity findings by age and owner, and the share that repeated from earlier tests. Repeat findings show that fixes treat symptoms. See How to Read a Penetration Test Report.
- When testers got in, did we notice? Ask for time to detect and time to respond from the last red team or scenario test, and whether the security operations team saw the activity without being told. CISA's AA24-326A shows how often alerts exist but are not acted on.
- Who accepted the risk on findings we did not fix, and when does that acceptance expire? Exceptions should be time-limited, owned by a named executive and reported to the risk committee. Open-ended acceptance of outdated software is exactly the decision CISA criticised.
- How are testers authorised and controlled? Expect written authorisation signed by someone with the authority to grant it, rules of engagement, an emergency stop contact, deconfliction with the SOC, and control over any credentials, data and access granted to testers, removed at the end. See the Red Team Rules of Engagement Template.
Red flags in management answers
- "We passed our pen test": TIBER-EU states plainly that its tests are not pass or fail; neither is any other test. A short report usually means a narrow scope.
- "The red team got domain admin in two days, as usual": if every exercise ends the same way, the basics are not fixed and the organisation is paying for a known answer. Fix the basics and use purple teaming before the next full exercise.
- "The testers fix things for us": the NCSC warns against outsourcing mitigation to the test team. Ownership of fixes must sit inside the organisation.
- "We use an AI tool that tests continuously, so we no longer need people": automated testing finds breadth; it does not replace a human red team asking whether a determined adversary could reach a critical asset. See the AI-Assisted and Autonomous Penetration Testing Guide.
What good looks like
- A risk-led testing plan: an annual plan approved by the risk committee, linked to critical services, major changes and threat intelligence, with clear rules for testing on change.
- Testing that confirms what you know: internal vulnerability management finds most issues first, and the test validates it, as the NCSC advises.
- Detection measured, not assumed: red team and purple team results report time to detect and respond, and each exercise produces new or improved detections. NIST CSF 2.0 outcome ID.IM-02 expects improvements to be identified from security tests and exercises.
- Findings closed and verified: every finding has an owner and a deadline set by risk; closure needs retest evidence; root causes are tracked so the same weakness does not return.
- Safe and lawful by design: a testing policy, written authorisation per engagement, a small control group for covert exercises and controlled handling of test accounts and data. See the Penetration Testing Policy Template.
- Identity in scope: testers are asked to examine service accounts, API keys, tokens and cloud roles, because attackers increasingly move through credentials rather than exploits. CISA's red team relied on Kerberos and directory abuse, and the access the organisation grants to testers is itself a set of credentials to govern. See the Service Account Security Guide.
Where to invest first
| Priority | Investment | Why first | Typical owner |
|---|---|---|---|
| 1 | Testing policy, authorisation process and rules of engagement | Makes every later test lawful, safe and repeatable | CISO with general counsel |
| 2 | Risk-led annual penetration testing of critical and internet-facing services | Finds exploitable weaknesses where attackers land first | CISO with service owners |
| 3 | Remediation tracking with owners, deadlines and retests | Turns reports into reduced risk; without it testing spend is wasted | Service owners, reported by security |
| 4 | Purple teaming and detection improvement | Converts findings into detections the SOC can rely on | Head of security operations |
| 5 | Objective-based red teaming, and TLPT where regulated | Tests the whole organisation against a realistic adversary once the basics hold | CISO with an executive sponsor |
| 6 | Continuous validation and a vulnerability disclosure programme | Keeps assurance current between exercises and gives outsiders a safe route to report | Security engineering |
Smaller organisations get most of the value from items 1 to 3: a clear policy, one well-scoped test of what matters and disciplined remediation. Larger and regulated organisations need items 4 to 6, and financial entities in scope for DORA TLPT should plan the three-year cycle and supervisor engagement well ahead. See the Penetration Testing Requirements Map and the Bug Bounty and Vulnerability Disclosure Guide.
Metrics to track
| Metric | What it shows | Good direction |
|---|---|---|
| Critical services tested in the last 12 months (%) | Whether testing covers what matters | Rising towards full coverage |
| Open high and critical findings past deadline | Whether owners act on results | Near zero, each with an owner |
| Repeat findings (% of findings seen in an earlier test) | Whether root causes are fixed | Falling each cycle |
| Time to detect and contain in red team or scenario tests | Real detection and response capability | Falling, measured the same way each time |
| Test findings turned into new or tuned detections | Whether testing improves the SOC | Rising |
| Active risk acceptances on test findings, and overdue ones | Risk accepted instead of fixed | Few, all within expiry |
Report these per critical service and trend them over at least three cycles. A single test result tells the board little; the trend tells it whether the programme works.
A 90-day plan
- Days 1 to 30, take stock: list every test, exercise and disclosure programme in the last two years, its scope, provider, findings and current status; identify critical services that have never been tested.
- Days 1 to 30, check authorisation: confirm a testing policy exists, who may authorise testing, how third-party systems are handled and how tester credentials and data are controlled and removed.
- Days 31 to 60, clear the backlog: assign owners and deadlines to all open high and critical findings, retest closed ones that lack evidence, and put overdue items with no plan in front of the risk committee as explicit risk acceptances.
- Days 31 to 60, set the plan: approve a risk-led testing plan for the next 12 months covering critical services, major changes and any regulatory testing duty such as DORA.
- Days 61 to 90, measure detection: run a scenario test or purple team session on one critical service and record time to detect and respond.
- Day 90, review: the risk committee reviews the first metrics pack, approves or rejects open risk acceptances and agrees the budget split between testing, remediation and detection improvement.
Common failure patterns
- Compliance-driven testing: tests bought for a certificate or customer questionnaire, scoped to pass, with no link to business risk.
- Alerts without action: testers trigger alerts that no one reviews, so the report says "detected" while the organisation did nothing.
- Uncontrolled tester access: accounts, VPN access and API keys created for testers that outlive the engagement. Treat them as privileged non-human and temporary identities with expiry dates.
Framework mapping
| Framework | Reference | Board relevance |
|---|---|---|
| NIST SP 800-53 Rev. 5 | CA-8 Penetration Testing, CA-8(1) Independent Penetration Testing Agent or Team, CA-8(2) Red Team Exercises, RA-5 Vulnerability Monitoring and Scanning, CA-2 Control Assessments | Separates testing types and expects independence |
| NIST CSF 2.0 | ID.IM-02, GV.OV-01 | Improvements from tests and exercises; strategy reviewed and adjusted |
| CIS Controls v8.1 | Control 18 Penetration Testing | Programme, periodic tests and remediation. See the CIS Control 18 Penetration Testing Guide |
| DORA | Articles 5(2), 24, 25, 26 | Management body accountability, yearly testing, TLPT every three years |
| UK Cyber Governance Code of Practice | Principles D and E | Board assurance, plan exercising and oversight |
| MITRE ATT&CK | T1558.001 Golden Ticket, T1003.006 DCSync, T1078 Valid Accounts | Identity techniques government red teams used without detection |
Practitioner checklist
- Maintain a testing policy with named authorisers, rules of engagement and an emergency stop process.
- Link every test to a risk, change or regulatory duty, and involve risk owners in scoping.
- Share the current vulnerability assessment with testers and track how many findings were already known.
- Give every finding a business owner, a risk-based deadline and retest evidence before closure.
- Record time to detect and respond in every red team and scenario test.
- Run a purple team session after each major test and deploy the resulting detections.
- Issue tester credentials and access with expiry dates, log their use and remove them at engagement end.
- Report coverage, repeat findings, overdue findings and risk acceptances to the risk committee each quarter.
Standards and references
- NCSC: Penetration testing
- CISA: Advisory AA24-326A, Red Team Assessment of a US Critical Infrastructure Organization
- EUR-Lex: Regulation (EU) 2022/2554 Digital Operational Resilience Act
- European Central Bank: TIBER-EU framework
- Bank of England: CBEST Threat Intelligence-Led Assessments Implementation Guide
- NIST: SP 800-115 Technical Guide to Information Security Testing and Assessment
- GOV.UK: Cyber Governance Code of Practice
- US Department of Justice: New Policy for Charging Cases under the Computer Fraud and Abuse Act
How NHI Mgmt Group can help
We provide independent board briefings and risk assessments on the non-human identities that testing so often exposes: service accounts, API keys, tokens, certificates and the temporary access granted to testers themselves. Our NHI Foundation Level Training Course helps security and engineering teams govern these identities so test findings about credentials are fixed at the root. Contact us to arrange a briefing.
Related NHI Mgmt Group resources: Purple Teaming Guide · Red Team vs Blue Team vs Purple Team · Vulnerability and Exposure Management Executive Briefing · Penetration Test Remediation Tracker Template