A GPAI system with systemic risk is a general-purpose AI model considered capable of causing broad downstream harm because of its scale, reach, or training compute. These systems face extra obligations, including adversarial testing and serious incident reporting, because their failures can propagate across many dependent use cases.
Expanded Definition
GPAI systemic risk refers to a general-purpose AI model whose scale, capability, and deployment reach make its failures likely to create widespread downstream harm. In the EU AI Act context, the term is used to distinguish high-impact foundation models from ordinary models that affect only one application or one tenant. The risk is not limited to model accuracy; it also includes the model’s ability to amplify unsafe outputs, enable misuse across many products, or propagate a weakness into a large dependency chain. For NHI and agentic environments, that matters because a single model may sit behind multiple agents, workflows, and tool integrations, turning one control failure into many operational failures. Definitions vary across vendors on how to measure “systemic” exposure, but the common thread is scale plus plausible cross-sector impact. Practitioners should read the term as a governance trigger, not merely a technical label, and align it with the obligations described in the EU AI Act and broader risk programs such as the NIST Cybersecurity Framework 2.0.
The most common misapplication is treating every large model as systemic by default, which occurs when organisations ignore actual deployment reach, downstream dependencies, and real incident blast radius.
Examples and Use Cases
Implementing GPAI systemic-risk controls rigorously often introduces release friction, requiring organisations to weigh faster deployment against deeper testing, logging, and escalation readiness.
- A foundation model embedded in multiple customer support agents is flagged for heightened review because a single prompt-injection weakness could affect many business units.
- An internal model used for code generation is subjected to adversarial testing because unsafe recommendations could spread across hundreds of repositories and CI pipelines.
- A vendor-hosted model feeding procurement, finance, and HR assistants is placed under stricter change control because one model update could disrupt several workflows at once.
- A model used in a safety-critical decision-support path is monitored for serious incident reporting thresholds, since errors may compound quickly across dependent systems.
- Security teams map model dependencies to NHI and secret pathways, using guidance from the OWASP NHI Top 10 and incident patterns discussed in the DeepSeek breach analysis.
These use cases are not limited to public-facing chatbots; they also include hidden dependencies where one model quietly becomes a shared control plane for many agents.
Why It Matters in NHI Security
GPAI systemic risk matters in NHI security because large models often sit inside agentic systems that rely on secrets, tokens, and delegated authority. When the model is compromised, the damage is rarely confined to one prompt or one user session. It can affect multiple non-human identities, tool permissions, and automated actions at once. NHI Management Group research shows how often identity-related weaknesses become operational incidents: in the 2024 ESG Report, 72% of organisations said they had experienced or suspected a breach of non-human identities, and compromised NHIs averaged 2.7 separate incidents in the prior 12 months. That pattern matters here because systemic-risk models increase the number of identity touchpoints exposed to a single failure. Organisations that do not classify and test these models early often discover the problem only after a cross-system failure, at which point governance, incident response, and containment become unavoidable.
That is why the Top 10 NHI Issues and the Ultimate Guide to NHIs — Why NHI Security Matters Now both emphasise shared identity control, secret discipline, and blast-radius reduction as foundational controls for AI-enabled operations.
Organisations typically encounter systemic-risk obligations only after a model failure has crossed product boundaries or triggered a reportable incident, at which point the term becomes operationally unavoidable to address.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP Non-Human Identity Top 10 address the attack surface, NIST AI RMF and NIST CSF 2.0 set the technical controls, and EU AI Act define the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| EU AI Act | Defines GPAI with systemic risk and adds testing and incident duties. | |
| NIST AI RMF | Provides risk framing for high-impact AI systems and downstream harms. | |
| NIST CSF 2.0 | GV.RM, ID.RA, DE.CM | Supports governance, risk assessment, and monitoring for critical AI dependencies. |
| OWASP Agentic AI Top 10 | A2, A5 | Agentic AI risks include tool abuse and unsafe autonomous actions from models. |
| OWASP Non-Human Identity Top 10 | NHI-01, NHI-02 | Systemic AI often depends on overprivileged identities and exposed secrets. |
Harden service identities and secret handling before treating the model as production-critical.
