How should security teams govern AI agents and LLM tools in the SDLC?

Why This Matters for Security Teams

AI agents and LLM tools change the SDLC because they can read code, generate changes, invoke build systems, and interact with internal services at machine speed. That means they are not just development helpers; they can become privileged execution paths with access to source code, secrets, tickets, and release workflows. Security teams should govern them as production-adjacent workloads, with explicit identity, scoped permissions, and reviewable actions, consistent with the risk-based approach in the NIST AI Risk Management Framework.

The main failure mode is assuming the model is the risk, when the real exposure sits in the surrounding orchestration: tool connectors, prompt injection paths, delegated tokens, and overly broad service accounts. Current guidance suggests treating agent permissions like any other high-trust integration, except with tighter auditability because outputs are probabilistic and may be influenced by untrusted inputs. That is especially important in SDLC pipelines where a single mistaken action can alter code, infrastructure, or release integrity.

In practice, many security teams encounter agent risk only after a tool-enabled workflow has already touched a repository, signing key, or deployment target without anyone realising the scope was too broad.

How It Works in Practice

Effective governance starts with assigning each agent a bounded identity, then mapping that identity to a narrow set of tools, repositories, and data sources. The goal is to prevent the agent from becoming a universal automation layer. Security and platform teams should define what the agent may read, what it may propose, and what it may execute. For sensitive actions, best practice is evolving toward human approval gates rather than full autonomy, particularly for merges, releases, secret access, and privilege changes.

Operationally, this usually means combining SDLC controls with identity and monitoring controls:

• Register the agent as a distinct workload identity, not a shared developer account.
• Issue short-lived credentials and separate read, write, and execute permissions.
• Restrict tool calls to approved APIs, runners, and repositories.
• Log prompts, tool invocations, outputs, and approvals so actions are attributable.
• Validate generated code, configuration, and change requests before promotion.

Threat modeling should account for prompt injection, tool abuse, data exfiltration, and malicious instruction chaining. The OWASP Agentic AI Top 10 is useful for structuring these risks, while the MITRE ATLAS adversarial AI threat matrix helps teams think about attack patterns against AI systems and their surrounding workflows. Where agents are connected to code generation or automation in CI/CD, the CSA MAESTRO agentic AI threat modeling framework can help organise control ownership across developers, platform engineering, and security.

These controls tend to break down when agents are embedded into legacy CI/CD systems that still rely on shared secrets, long-lived service accounts, and loosely governed plugin ecosystems.

Common Variations and Edge Cases

Tighter agent governance often increases engineering overhead, requiring organisations to balance developer speed against change assurance. That tradeoff is real: some teams can tolerate autonomous code drafting but not autonomous deployment, while others may permit limited release assistance with mandatory approval. There is no universal standard for this yet, so policy should reflect the sensitivity of the environment rather than a blanket “allow” or “ban” approach.

High-risk edge cases include agents with access to production data, agents that can open pull requests across multiple repositories, and agents that chain into external SaaS tools or package registries. In those cases, the biggest issue is not only privilege but trust transitivity. A model can be well-governed internally and still become unsafe if a downstream connector is compromised or if untrusted content is allowed to steer tool use.

Where the agent supports secure software delivery, align governance with the NIST Cybersecurity Framework 2.0 for asset, access, and monitoring discipline, and use the NIST AI 600-1 Generative AI Profile where generative systems introduce extra validation and transparency requirements. When the agent is used for research, code review, or test generation only, the governance burden may be lighter, but it should still include explicit scope boundaries, logging, and rollback paths.

Related resources from NHI Mgmt Group

• How should security teams govern AI agents that can choose tools at runtime?
• How should security teams govern AI agents that choose tools at runtime?
• How should security teams govern AI agents that use service accounts and MCP tools?
• How should security teams govern AI agents that can invoke multiple tools in one session?