How should security teams govern agentic development when AI systems can write code and provision infrastructure with limited human review?

Why This Matters for Security Teams

Agentic development changes the control problem from “Can this code be trusted?” to “Can the decision chain behind this code be trusted?” When an AI system can generate files, call APIs, or provision cloud resources, the highest-risk events are often not the final artefacts but the unseen steps that created them. That is why guidance from the NIST AI Risk Management Framework is relevant: governance must cover the system lifecycle, decision accountability, and downstream impact, not just model output quality.

Practitioners frequently over-index on code scanning, secret scanning, or infrastructure-as-code review, then assume those controls are enough. They are not, because an autonomous agent can choose dependencies, alter configs, or chain tool use faster than a human reviewer can validate intent. Current guidance suggests that security teams should classify each agent by permitted actions, constrain where it can write, and require policy checks before execution as well as after generation. The operational question is not whether the output looks syntactically valid, but whether the agent had authority to produce it in the first place.

In practice, many security teams encounter drift only after an agent has already created risky access, exposed secrets, or deployed an unapproved service rather than through intentional governance design.

How It Works in Practice

Effective governance starts by treating the agent like a privileged system with narrowly defined duties. That means assigning a known owner, a documented purpose, explicit tool permissions, and enforced boundaries for code, cloud, and data access. The most reliable pattern is to require every significant action to be attributable to a prompt, policy decision, or human approval step so the organisation can reconstruct why the action happened.

In practice, this usually combines several layers:

• Prompt and task scoping so the agent only receives the minimum context needed for the work.
• Policy gating before execution, especially for package installation, infrastructure changes, and any secrets access.
• Ephemeral, tightly scoped credentials so the agent cannot keep standing privilege.
• Runtime monitoring for unusual tool use, repeated retries, or escalation beyond the approved task.
• Immutable logs that connect prompts, tool calls, code diffs, and infrastructure changes.

Security teams should align this with existing control frameworks rather than inventing a parallel process. NIST Cybersecurity Framework 2.0 is useful for mapping governance, protection, detection, and response, while NIST SP 800-53 Rev 5 Security and Privacy Controls helps translate agent oversight into concrete access control, audit, and configuration management requirements. For attack pattern analysis, the MITRE ATLAS adversarial AI threat matrix is useful when evaluating prompt injection, tool abuse, or manipulated model behaviour.

The key implementation point is that review cannot be a single human checkpoint at merge time. It has to be continuous, with pre-execution approval for high-risk actions and post-execution validation for drift, anomalies, and unauthorised side effects. These controls tend to break down in fast-moving platform engineering environments where agents can trigger nested tool chains across multiple accounts before logs, approvals, and policy engines are fully correlated.

Common Variations and Edge Cases

Tighter governance often increases delivery overhead, requiring organisations to balance developer speed against the risk of autonomous changes escaping review. That tradeoff becomes sharper in sandboxed experimentation, production infrastructure, and agent-to-agent workflows, where the same control can feel either appropriately restrictive or operationally unworkable depending on blast radius.

Best practice is evolving for multi-agent systems, and there is no universal standard for this yet. Some teams permit broader autonomy in non-production environments, then enforce stronger human approval and stronger credential scoping before anything can reach production. Others use policy-as-code to block high-risk actions automatically, such as network exposure, privilege escalation, or changes to identity and secrets systems. The right choice depends on whether the environment is optimised for rapid prototyping or regulated operations.

This is also where identity governance intersects with agentic AI. If an agent can assume roles, mint tokens, or call deployment APIs, then it behaves like a non-human identity and should be governed accordingly. The OWASP Agentic AI Top 10 and CSA MAESTRO agentic AI threat modeling framework are useful references for modelling those risks, while the NIST AI Risk Management Framework remains the clearest anchor for governance discipline.

Related resources from NHI Mgmt Group

• How should security teams govern AI systems that can act without human approval?
• How should security teams govern AI remediation systems that inspect proprietary code?
• How should security teams govern machine identity credentials in agentic AI environments?
• How should security teams govern AI agents that can access enterprise systems?