What are common mistakes with Phishing Simulation?

What makes phishing simulations ineffective?

Phishing simulations fail when they are reduced to a vanity metric, run on a weak cadence, or built from lures that are too generic to mirror the tactics users actually face. A useful programme measures whether people report suspicious messages, whether risky behaviour changes over time, and whether the exercise teaches a repeatable response instead of simply catching people out.

That is why the Phishing Simulation Programme Guide emphasises report rate, ethics, difficulty calibration, and realistic delivery policy rather than click rate alone. Simulations should be hard enough to be credible, but not so artificial that the results reflect test design more than user behaviour.

Where do most phishing simulation programmes go wrong?

The most common mistake is using clicks as the only success measure. Clicks tell you something, but they do not show whether employees reported the message, escalated it quickly, or learned to recognise the indicators that matter in real incidents. A programme that ignores reporting can look successful while still leaving the organisation slow to detect genuine phishing.

A second failure is poor frequency. If campaigns are too rare, they become events people anticipate instead of a continuous signal of behaviour. That weakens the value of trend data and makes coaching less effective, because teams are measuring memorisation of the last exercise rather than sustained awareness.

A third mistake is using unrealistic or outdated lures. If the content does not resemble current attacker behaviour, the simulation may teach users the wrong lesson. That is why programmes should stay aligned with current phishing patterns, including modern delivery methods and social engineering techniques, not just old-style misspelled emails.

Phishing simulations also fail when they ignore the broader risk context. A user who falls for one test does not always need punishment; they may need better training, clearer reporting paths, or a review of whether the organisation has made safe reporting easy enough to use.

Internal programme design matters too. The right question is not whether someone clicked once, but whether the organisation can turn a failed simulation into better reporting behaviour, better coaching, and less repeat exposure.

The Top 10 Phishing Techniques is useful here because it keeps the testing model tied to the techniques attackers actually use, including AiTM, QR-code lures, OAuth consent phishing, device-code phishing, callback phishing, and AI-assisted lures.

How should a phishing simulation programme be run instead?

Phishing simulations work best when they are treated as behaviour measurement and coaching, not as a periodic humiliation exercise. The programme should reward reporting, use coaching for repeat offenders, and give employees a clear path to act on suspicion without fear of blame. That combination produces better signal than punitive naming and shaming.

Security Awareness and Human Risk Management Guide is the better parent resource for this operating model because it frames phishing testing as part of human risk management, not a standalone checkbox. It reinforces the idea that the point is behaviour change, culture, and measurable reporting improvement.

Security Awareness Training Buyer's Guide also fits naturally because it covers simulation ethics, reporting metrics, and privacy-aware evaluation criteria. Those are the details that keep a programme credible with employees and useful to defenders.

Practitioner Guidance

What to prioritise: Optimise for report rate, speed of reporting, and coaching outcomes before you worry about click-through percentages. A low click rate can hide a weak programme if users are not learning how to escalate suspicious messages quickly.

Common mistake: Do not build campaigns so unrealistic that they only test whether people can spot an obviously fake template. The better test is whether the lure matches the style, urgency, and delivery channels of the attacks your organisation is likely to face.

What to verify: Check that repeat clickers receive targeted follow-up and that employees are not punished for reporting mistakes. If staff think they will be blamed, they will under-report, and your detection signal will get worse even if the simulation metrics look cleaner.

Decision rule: If a simulation measures clicks but not reporting, treat it as incomplete. If it repeatedly uses outdated lures, treat the results as low confidence because the programme is measuring familiarity with the exercise rather than resilience to current phishing.

Practitioner takeaway: The best phishing simulation programmes change behaviour, not morale. If the exercise does not improve reporting and coaching, it is probably producing data, not defence.

Related resources from NHI Mgmt Group

• Who should own phishing simulation reporting in an identity programme?
• Who should own the workflow from phishing detection to simulation?
• How do you know if a phishing simulation programme is actually working?
• Why do identity and access systems matter in phishing simulation programmes?