AI compliance and governance: what do security teams need now?

TL;DR: AI compliance now sits at the intersection of privacy law, model governance and security, with the EU AI Act, GDPR, NIST AI RMF and ISO/IEC 42001 shaping obligations around training data, profiling, transparency and monitoring, according to Osano. The practical issue is not only whether AI is allowed, but whether organisations can prove lawful data use, control inference risk and sustain audit-ready oversight.

Editorial analysis by NHI Mgmt Group, based on content published by Osano: “AI Compliance: Why Artificial Intelligence Systems Pose Risk & How to Contain It”.

By the numbers:

• The EU AI Act allows fines of up to €35 million or 7% of worldwide annual turnover for non-compliance.
• Other serious EU AI Act obligations can trigger fines of up to €15 million or 3% of worldwide annual turnover.
• Providing incorrect, incomplete or misleading information to authorities can lead to penalties of €7.5 million or 1% of worldwide annual turnover.

Key questions

Q: How should organisations govern access to data used by AI systems?

A: Treat AI data access as an identity governance problem, not just a data storage problem.

Q: Why do AI models create privacy risk even when the data looks anonymised?

A: Because models can infer, recombine or reveal sensitive attributes from inputs that appear harmless in isolation.

Q: What are the signs that AI governance is failing in the enterprise?

A: Common warning signs include rapid growth in AI use without matching policy coverage, sensitive files being copied into personal accounts, and a large share of AI apps carrying high or critical risk.

Practitioner guidance

• Define lawful-basis controls for AI training data Document the legal basis, source type, retention rule and intended use for every dataset that enters model training or evaluation.
• Require human review for consequential AI decisions Route hiring, lending, access and other material outcomes through a review and challenge path before the decision is finalised.
• Add model-specific security testing Test for prompt injection, inversion, data leakage and unsafe output handling as part of pre-production validation and ongoing monitoring.

Bottom line: AI compliance now depends on whether data use, model behaviour and disclosure can all be governed in one control model.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →