Join our Newsletter — 33% off our NHI Course

Artificial General Intelligence (AGI)

Artificial General Intelligence refers to an AI system that can learn, reason, and adapt across many tasks at a level comparable to human capability. In security terms, AGI would require governance for autonomous decision-making, access control, identity assurance, and containment because its actions could span systems, data, and workflows without narrow task limits.

What AGI Means in Security Terms

Artificial General Intelligence is not just a stronger narrow model, it is a system that could transfer learning across domains, act with broader autonomy, and make decisions in unfamiliar situations. That changes the security problem from model performance alone to governance over agent behaviour, authority, and containment.

For practitioners, the important distinction is that AGI would not be treated like a single-purpose tool with one bounded workflow. Its operational reach could span multiple business functions, so security planning must assume that the system may reason across contexts, reuse knowledge in new settings, and create side effects beyond the original request.

Why AGI Changes the Security Model

AGI would expand the attack surface because one system could interact with many assets, services, and data sets. If its permissions, prompts, or tool access are too broad, a single failure could affect a much larger part of the environment than with a narrow application.

This is why AGI raises questions about who approves actions, how autonomy is limited, and when a human must intervene. The main risk is not only incorrect output, but also inappropriate execution with real operational consequences.

Governance, Identity, and Containment

AGI governance has to address decision authority, identity assurance, and the scope of access granted to the system. If an AGI can act through APIs, workflows, or delegated tools, organisations need clear rules for authentication, authorization, logging, and revocation.

Containment becomes part of governance, not just infrastructure. Isolation boundaries, least privilege, and step-up review matter because the system may be able to chain actions across environments faster than a human can supervise in real time.

Operational Readiness for AGI-Style Systems

Even before anything qualifies as AGI, organisations can prepare by testing how much trust they place in autonomous decisions, external tools, and cross-system execution. That includes examining where the system can write, trigger, approve, or retrieve sensitive information.

Readiness also means defining what happens when the system behaves unexpectedly, exceeds its intended scope, or cannot explain why it chose a path. The closer an AI system gets to broad general capability, the more important it becomes to treat reliability, oversight, and rollback as core controls rather than optional safeguards.

Risk and Threat Considerations

AGI would create concentrated risk because a highly capable system could amplify mistakes, misuse, or compromise across many systems at once. If an attacker, faulty policy, or unstable model behaviour influences that system, the resulting exposure could extend far beyond a single task or dataset.

Failure mechanism: Excessive autonomy, weak guardrails, or compromised credentials could let the system execute actions that were never intended, especially if it can chain tools or access multiple environments.

Impact: The result could be cross-system data exposure, unauthorized changes, privilege misuse, or cascading operational failures that are difficult to contain or unwind.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

NIST AI RMF, NIST SP 800-53 Rev 5 and NIST Zero Trust (SP 800-207) set the technical controls, while ISO/IEC 42001:2023 defines the regulatory obligations.

Framework Control / Reference Relevance
NIST AI RMF AI Risk Management Framework AGI is governed as a high-consequence AI system requiring risk, trust, and oversight management.
Recommendation — Apply AI RMF functions to govern autonomy, monitor harms, and constrain high-impact AI decisions.
ISO/IEC 42001:2023 AI Management System AGI requires organisational governance for accountable AI development, deployment, and oversight.
Recommendation — Establish an AI management system to assign accountability, controls, and review for advanced AI use.
NIST SP 800-53 Rev 5 AC-6 — Least Privilege AGI safety depends on limiting the authority and reach of any system actions it can perform.
IA-5 — Authenticator Management AGI operation depends on controlled handling of credentials, tokens, and other access material.
Recommendation — Constrain AGI-connected services to the minimum permissions needed for each approved function. Manage and rotate the credentials used by AI-connected tools and services under strict lifecycle control.
NIST Zero Trust (SP 800-207) Zero Trust Architecture AGI containment depends on verifying every request and limiting implicit trust across systems.
Recommendation — Apply zero trust principles to every AGI action, tool call, and cross-system access request.

Practitioner Guidance

Why practitioners should care: AGI changes the security boundary from a single application to an actor that may make decisions, call tools, and affect workflows across the enterprise. That means governance must focus on authority, not just model quality.

What to watch for: Pay attention to any design that gives an AI system persistent credentials, broad API reach, or the ability to act without explicit approval for high-impact steps. Those are the conditions that turn capability into material risk.

Practitioner takeaway: Treat AGI as a governance and containment problem first, because once broad autonomy is granted, the hardest control to recover is not the model, it is the authority it already has.