What are the main risks of using generative AI in security awareness content?

Why generative AI changes the risk profile of security awareness content

Generative AI can speed up drafting, but speed is not the same as accuracy. For security awareness, the core problem is that content must be both understandable and operationally correct, or it trains people to trust the wrong signals. That is why generated material needs policy alignment, threat realism, and review before it reaches employees.

Awareness content is also a trust product. If it repeatedly sounds polished but contains errors, staff will eventually treat it as just another low-value message. That weakens the credibility of phishing guidance, reporting instructions, and safe-handling advice, especially when the organisation is trying to change behaviour rather than merely broadcast information.

Generative AI also tends to compress nuance. In security awareness, that can flatten distinctions between user groups, threat levels, and approved practices, which matters because a message that is safe for one audience may be misleading for another. Content quality is therefore not only a writing issue, it is a control issue.

What can go wrong when the content is generated without guardrails

The most immediate failure mode is factual drift: guidance can be outdated, inconsistent with current policy, or too generic to reflect the organisation’s actual tools and reporting paths. A related problem is unsafe example design, where a well-intended lesson accidentally normalises risky behaviour by showing the wrong thing too clearly or by framing an attack in a way that is easy to imitate.

Bias and tone issues also matter. Generated content may overemphasise certain threats, understate others, or sound accusatory in ways that reduce engagement. In practice, that can make employees tune out the message or assume the programme is detached from how the organisation really works.

There is also an information exposure risk. If prompts, source material, or drafts include sensitive internal details, the model can echo them into content that is far broader than intended. For a security awareness programme, that can turn an internal drafting aid into a leakage path for process details, system names, or incident-specific context.

How to use AI safely in awareness drafting and review

Use generative AI as a drafting accelerator, not as a policy authority. The content should be checked against approved controls, current reporting procedures, audience role, and the actual threats employees face. If the model cannot be constrained to those inputs, it is better suited to ideation than to publication.

One useful guardrail is to separate generation from approval. Drafts can be created quickly, but the final version should be reviewed by someone who can judge whether the message is technically correct, behaviourally practical, and aligned to the organisation’s tone. That review is especially important for examples, call-to-action language, and any reference to internal tools or workflows.

For a broader control view, NIST’s NIST AI 600-1 GenAI Profile is a useful reference point for content provenance, testing, and governance around generative AI use cases, while the Security Awareness and Human Risk Management Guide helps anchor the work in behaviour change, metrics, and programme design rather than simple content volume.

How to reduce risk before the content reaches employees

Start by defining what source material the model is allowed to use, and what it must not infer. The safest awareness content is usually built from approved policy language, current threat intelligence, and a small set of validated examples. That reduces the chance of invented advice, accidental disclosure, and inconsistent training messages.

Then test the output for three questions: does it match policy, does it reflect current threats, and would an employee act safely after reading it? If the answer is no to any of those, the content is not ready. This is where human review is doing real security work, not editorial polish.

For teams that need a structured way to decide when AI use is acceptable at all, the AI Risk Assessment Guide and the Security Awareness Training Buyer's Guide are useful complements, because they connect content decisions to risk treatment, vendor evaluation, and programme governance rather than treating AI as a content shortcut.

Practitioner Guidance

What to prioritise: Prioritise accuracy over variety. A smaller set of tightly reviewed messages is better than a high-volume stream of attractive but inconsistent content, because awareness programmes fail when employees cannot trust the advice.

What to verify: Verify that every generated asset maps to current policy and current reporting paths before publication. If the model introduces examples, labels, or actions that your environment does not actually use, treat that as a release blocker, not a wording issue.

Common mistake: The most common error is using AI to produce “good enough” awareness copy and assuming human review will catch everything later. In practice, review only works when the reviewer has a checklist for policy, threat realism, and sensitive-data leakage, not just grammar.

Practitioner takeaway: Generative AI can support awareness delivery, but it must be bounded by approved inputs and accountable review, or it becomes a credibility risk for the whole programme.

Related resources from NHI Mgmt Group

• How should organisations control the security risks of using AI content generation tools in the workplace?
• How should security teams start using generative AI safely?
• How should organisations adapt security awareness training for generative AI phishing?
• How should security teams build an AI cybersecurity awareness program for employees who use generative AI tools every day?