Most security awareness training platforms are bought on the size of their content library and judged on completion rates, which says little about whether anyone reports the next phish. The category now runs from course libraries and phishing simulators to human risk management (HRM) platforms that score individual behaviour from your email, identity and endpoint data, which makes the purchase a data protection and integration decision too. This vendor-neutral guide is for awareness leads, CISOs, HR and procurement teams writing requirements, running an RFP or replacing a platform, and ends with a proof of concept you can run in one quarter. For programme design, see the Security Awareness and Human Risk Management Guide.
Key takeaways
- Buy for behaviour, not completions. Score platforms on how they measure reporting rate, time to report and repeat risky actions, not on how many modules they hold.
- Simulation ethics belong in the requirements. The NCSC warns that phishing simulations "can also create legal risk" and "erode trust between employees and security"; the platform must support a non-punitive programme, not a click league table.
- The report button is the integration that matters most. It must feed your SOC's reporting mailbox, give reporters feedback and not trigger training for real reports.
- Human risk scoring is workplace monitoring. Under UK GDPR, the ICO expects the least intrusive means, transparency and usually a DPIA; check that the platform lets you minimise, aggregate and delete.
- Treat the platform as a privileged SaaS supplier. It holds your whole workforce directory, SCIM and API credentials and sometimes mailbox access, so score its own security as hard as its content.
Define requirements
- Obligations: list what you must evidence. CIS Controls v8.1 Safeguard 14.1 requires training "at hire and, at a minimum, annually" with content reviewed annually, 14.9 requires role-specific training, and NIST SP 800-53 AT-2, AT-3 and AT-4 cover literacy training, role-based training and training records. ISO/IEC 27001:2022 Annex A 6.3 covers awareness, education and training.
- Population: headcount, contractors, frontline staff without a mailbox, languages and assistive technology users.
- High-exposure roles: service desk, finance, executives, developers and administrators. The MGM Resorts breach began with a help desk phone call. See the Account Recovery and Help Desk Security Guide.
- Channels to simulate: email, SMS, voice, collaboration tools, QR codes and MFA prompts. The Twilio breach started with SMS phishing of employees. See the Phishing Guide and the Vishing and Smishing Guide.
- Systems to integrate: email platform, identity provider, HR system, learning management system (LMS) and SIEM.
- Privacy position: decide with HR, legal and any works council whether you will score individuals at all, and who may see scores.
Size changes the answer. A small organisation can often meet its obligations with its email platform's built-in simulation and training plus a short course library. An HRM platform earns its cost in large or regulated organisations with many data sources, languages and role-based targeting, provided the privacy governance exists to support it.
| Platform type | What it does | Fits when | Watch for |
|---|---|---|---|
| Content library with LMS delivery | Courses, videos and quizzes, tracked by completion | Compliance evidence is the main need; small population | No behavioural data; annual-only cadence |
| Training plus phishing simulation | Adds simulated emails, landing pages and follow-up lessons | You want practice and a reporting baseline | Click rates without difficulty rating; punitive reporting |
| Human risk management platform | Correlates simulation, email security, identity, DLP and endpoint signals into risk scores and triggers nudges or lessons | Several data sources, role targeting at scale, mature privacy governance | Broad API access to your tools; individual scoring without a DPIA |
| Email platform built-in features | Simulation, report button and assigned training inside the mail suite | Small and mid-size organisations on one mail platform | Limited content, languages and non-email channels |
Evaluation criteria
Weight these to your requirements. An illustrative split is 25% measurement and reporting, 20% content, 20% integrations and identity, 15% privacy controls, 10% platform security, 10% accessibility and languages; adjust it rather than copying it.
| Criterion | What good looks like | Weak signs |
|---|---|---|
| Content quality and relevance | Short modules on current threats (deepfake calls, MFA fatigue, QR phishing); editable to reference your own policies and reporting route; reviewed at least annually | Long annual videos; content you cannot edit or localise |
| Role-based and just-in-time training | Assignment by HR attributes or IdP groups; short lessons triggered by a risky action; tracks for high-exposure roles | Same curriculum for everyone; "role-based" means a different video title |
| Phishing simulation | Email, SMS, voice and QR templates; difficulty rating (for example using the NIST Phish Scale); exclusion of sensitive lures; no storage of entered credentials | Bonus, redundancy or health lures by default; stored passwords |
| Measuring behaviour change | Report rate, time to first report, real threats reported, repeat risky actions and trends normalised for difficulty, by role and site | Headline metric is completion or click rate; no export of raw events |
| Reporting button integration | One-click reporting in every mail client; reports reach the SOC mailbox; real reports never trigger training; feedback to reporters | Separate button that bypasses the SOC; duplicate buttons; no feedback loop |
| Human risk scoring and privacy | Configurable signals; aggregate views by default; restricted access to individual data; retention settings; explainable, challengeable scores | Opaque scores; individual league tables for managers |
| Accessibility and languages | Training player and landing pages conform to WCAG 2.2 Level AA; captions, keyboard and screen reader support; reviewed translations | No accessibility conformance report; machine translation only |
| LMS, HRIS and IdP integration | SSO through SAML 2.0 or OpenID Connect; SCIM provisioning and deprovisioning; SCORM packages or xAPI statements for your LMS or learning record store; HR attributes for targeting | CSV user uploads; accounts that outlive the employee |
| Platform security | Admin SSO and MFA, tenant audit logs, named regions and sub-processors, independent assurance (for example SOC 2 or ISO/IEC 27001), vulnerability disclosure | Shared admin accounts; vague data location |
Phishing simulation ethics
The NCSC's phishing guidance is direct: "No training package, including phishing simulations, can teach users to spot every phishing attempt", "Employees who are afraid for their jobs will not report mistakes", and punishing people for clicking emails you sent "starts to resemble entrapment". It advises checking with HR before running simulations and measuring "how many people reported it" as well as clicks. Ask whether sensitive lures can be blocked, click data shown only in aggregate, and whether the landing page teaches rather than shames.
Report button and simulation delivery
Simulations must reach inboxes without weakening real filtering. In Microsoft 365, the advanced delivery policy accepts up to 50 simulation domains (the 5321.MailFrom or DKIM domain), up to 10 sending IPv4 addresses and, for non-email simulations, up to 30 URLs; matching messages are not blocked by Safe Links or detonated by Safe Attachments, so the entries must belong only to the simulation service. The same platform warns that if the user reporting mailbox is not configured as a SecOps mailbox, "a user reported message might trigger a training assignment by the phishing simulation product". Third-party buttons must deliver the original as an uncompressed .EML or .MSG attachment, and two buttons side by side create duplicate submissions. See the Email Security Guide.
Human risk scores and employee monitoring
Scoring individuals from email, identity and endpoint data is monitoring workers. The ICO's guidance says employers must "select the least intrusive means", carry out a DPIA before processing likely to cause high risk, should not rely on consent in employment, must tell workers about monitoring, and should let workers "see and, if necessary, explain or challenge the results". (the guidance is under review following the Data (Use and Access) Act). Require signal minimisation, short retention, aggregate reporting and subject access exports. See the DPIA Guide: Running a Data Protection Impact Assessment.
Where identity fits
The platform needs an accurate, current list of people. SCIM (RFC 7644) provisions users and groups through the /Users and /Groups endpoints with POST, GET, PUT, PATCH and DELETE, so joiners are enrolled and leavers are removed without CSV uploads. Prefer SCIM from your identity provider, with department, role and location mapped for targeting. Integration credentials, including SCIM bearer tokens, API keys for email security and endpoint tools, and any OAuth app with mailbox read or message-injection rights, are non-human identities with workforce-wide reach: scope them to the minimum, record an owner, rotate them and revoke them at contract end. See the SCIM Provisioning Guide, the Joiner-Mover-Leaver Guide and the SaaS and OAuth App Governance Guide.
Questions to ask vendors
- Show the report button in each mail client we use, routing a real phish to our SOC and a simulation to you, with feedback to the reporter.
- Can report rate and time to report be the headline metrics, normalised for template difficulty?
- Which signals feed your human risk score, how is each weighted, can we switch signals off, and can an employee see why their score changed?
- What OAuth scopes, API permissions and mailbox rights does each integration need? Can any of them read message content or send mail as users?
- Do you provision through SCIM, which attributes do you consume, and how fast is a leaver removed?
- Which languages are reviewed translations, and can you provide an accessibility conformance report against WCAG 2.2 AA?
- Can we export completions to our LMS (SCORM packages or xAPI statements) and raw events to our SIEM?
- Where is our data hosted, which sub-processors handle it, what is your retention default, and what independent assurance covers the platform?
- What does the price count (licensed users, modules, simulation channels such as SMS or voice, languages or integrations), and what happens when headcount drops or we add a channel?
Red flags
- Sales material built around ranking or disciplining people who click.
- Simulation landing pages that store entered passwords, or lures that exploit pay, redundancy or health fears by default.
- A report button that bypasses your SOC, or real reports that trigger "you failed" training.
- A single opaque risk score per employee, visible to line managers, with no explanation or appeal route.
- Integrations needing global administrator rights or long-lived mailbox write keys.
- Pricing that charges extra for every simulation channel or language, discouraging coverage of high-risk roles.
Proof of concept
- Scope and agree pass criteria: pick two high-exposure groups (for example finance and the service desk) and a comparison group, around one quarter, with weights agreed before the first demo. Complete a DPIA screening and brief HR before any simulation.
- Connect with least privilege: provision users through SCIM from your IdP, enable SSO and admin MFA, and record every permission each integration requests. Never use a personal administrator token.
- Test reporting end to end: configure the report button, reporting mailbox and simulation delivery settings; confirm a real phish reaches the SOC, a simulation report is recognised and thanked, and no real report triggers training.
- Run graded simulations: send templates of known difficulty across email and one other channel; measure report rate, time to first report and credential submission, normalised by difficulty.
- Test content: trigger a just-in-time lesson after a simulated risky action, assign a role track, and check accessibility with assistive technology users.
- Test privacy controls: run an aggregate report, a subject access export and a deletion; confirm managers cannot see individual scores unless you allow it.
- Test the exit and roll back: export completions (SCORM or xAPI records) and raw events, revoke SCIM tokens, API keys and OAuth grants, remove simulation delivery entries from your mail platform, and request written confirmation of deletion.
Keep tracking after go-live: report rate, time to first report, real threats reported per month, repeat risky actions, and leavers removed within one provisioning cycle.
How NHI Mgmt Group can help
We provide independent requirements, RFP and evaluation support. Browse vendors in our products directory or contact us.
Related NHI Mgmt Group resources: Security Awareness and Human Risk Management Guide · Phishing Guide · Vishing and Smishing Guide · Deepfake and AI Impersonation Guide · GRC Platform Buyer's Guide