AI agentic security tools need stronger governance, not just more power

At a glance

What this is: PortSwigger argues that frontier AI is pushing AppSec toward agentic automation, but the real constraint is governance, not raw model capability.

Why it matters: For IAM and security teams, the same shift reinforces a familiar lesson: when software systems gain action power, policy enforcement, auditability, and least privilege become the controls that determine whether automation is usable at scale.

By the numbers:

• 17 minutes
• The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities.
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.

👉 Read PortSwigger's analysis of AI agentic security and the future of AppSec

Context

AI agentic security tools are moving the decision point from manual testing to governed execution, which changes the control problem for AppSec and adjacent identity programmes. The issue is no longer whether a model can propose an action, but whether the system can constrain, authorise, log, and reproduce that action safely at scale.

That matters for NHI governance because agentic tools behave like software systems with delegated access, not like passive assistants. Once a model can choose tools, run attacks, or touch sensitive data, the surrounding policy model starts to resemble privileged automation, where deterministic controls matter more than model output quality.

From our research: The State of Secrets in AppSec shows that leaked secrets take an average of 27 days to remediate, which is far longer than the window in which automated abuse often begins.

Key questions

Q: How should security teams govern AI-enabled workflows that can act on their own?

A: Treat them as identity-governed execution paths, not just software features. Assign a named owner, define least-privilege access, log every tool call, and require revocation paths for credentials and tokens. If the workflow can touch production systems or sensitive data, its permissions must be reviewed with the same discipline used for privileged machine identities.

Q: Why do agentic AI systems increase initial access and privilege abuse risk?

A: Because they can chain valid access into multiple tool calls without needing a human to approve each step. If a secret is exposed or a role is overbroad, the agent can turn that access into data movement, service interaction or recursive task execution. The risk rises when access outlives the task that created it.

Q: What do teams get wrong when they rely on human-in-the-loop controls for AI?

A: Teams often treat human-in-the-loop as a compliance checkbox, but the real test is whether the organisation understood the risk and placed controls around irreversible actions. A human review step helps only when it is tied to ownership, evidence, and a clear boundary for what the agent may do.

Q: How do I know if an agentic security workflow is actually safe?

A: Look for reproducibility, deterministic policy checks, and complete audit trails. A safe workflow should produce the same governed outcome when rerun under the same conditions, and every action should be attributable to a policy decision rather than model improvisation. If that evidence is missing, the workflow is not ready.

Technical breakdown

Why non-deterministic models need a governance cage

Large language models are probabilistic systems, which means the same prompt can produce different outputs. That is acceptable for ideation, but unsafe when the model can take actions, chain tools, or influence real systems. The core problem is not intelligence alone. It is that action-taking systems need deterministic policy enforcement, bounded permissions, and auditability. In practice, the model should propose or reason, while a separate control layer decides whether an action is allowed, logged, and attributable. This is the architectural difference between assistance and delegated execution.

Practical implication: keep the model outside the trust boundary and force every privileged action through deterministic policy enforcement.

Agentic AppSec workflows and the attack surface of tool access

When an AI system is given offensive AppSec tooling, its capability expands through the tools, not just the model. Curl, scanners, payload generators, and parsers turn a capable model into an execution engine that can test, enumerate, and exploit at scale. That is powerful, but it also creates failure modes that look like classic privileged automation risks: wrong-target execution, sensitive-data leakage, and poor traceability. The security question becomes whether tool access is scoped, monitored, and revocable in a way that matches the sensitivity of the task.

Practical implication: treat AI access to testing tools as privileged access and scope it with least privilege and revocation controls.

Human-in-the-loop oversight is a control, not a bottleneck

Human-in-the-loop decisioning only works if it is placed at the right points in the workflow. Approvals for high-risk actions, policy gates before target execution, and reviewable audit trails give practitioners a way to preserve accountability without forcing every low-risk step through manual review. This is closer to privileged access governance than to a chat interface. The important distinction is that human review validates the boundary conditions, while the machine handles repetitive execution inside the boundary.

Practical implication: reserve human approval for target selection, exploit execution, and data handling boundaries, not for every low-risk step.

Threat narrative

Attacker objective: The objective is to weaponise delegated AI execution so the operator loses control over targeting, data handling, and auditability.

1. Entry occurs when an agentic security tool is granted access to offensive testing capabilities and target context that exceed what a human operator intended.
2. Escalation follows when the model chains tool outputs into autonomous action, creating the possibility of wrong-target testing, sensitive-data exposure, or hidden persistence in logs and artefacts.
3. Impact is the loss of reliable assurance, because the system can act faster than reviewers can inspect and may generate unsafe or untraceable outcomes.

NHI Mgmt Group analysis

Capability without governance is the real security failure mode. Frontier models will keep improving, but model capability is not the limiting factor anymore. The limitation is whether security teams can force action-taking systems through a deterministic cage of policy, approval, and logging. In practice, this is the same governance problem that appears in NHI and privileged automation programmes, where access must be governed as a controlled runtime event, not assumed safe because the actor is software.

Agentic security tooling turns AppSec into a delegated privilege problem. Once a model can select tools and execute tests, the relevant question becomes who authorised the action, under what policy, and with what blast radius. That is an identity and privilege question as much as a testing question, because the system is effectively operating with machine identity and delegated authority. Practitioners should recognise this as a form of privileged workflow design, not just AI enablement.

Safety, reproducibility, and audit trails are now baseline control requirements. The article is right to frame these as structural, not optional, because you cannot prompt reliability into a stochastic system. For enterprise programmes, this aligns with least privilege, approval boundaries, and evidence retention expectations familiar from NIST CSF and NIST SP 800-53. The practical conclusion is simple: if an agentic workflow cannot be reproduced and explained, it is not ready for high-trust operational use.

AI governance debt: the longer organisations let agentic tools expand without control architecture, the more they accumulate hidden operational risk. That debt shows up as unclear accountability, weak auditability, and oversized trust in tool-driven automation. For AppSec, the lesson extends to any AI system that can act, because delegated actions without controls eventually become governance incidents.

Machine speed changes the review model. Human review was designed for discrete actions that could be inspected after the fact. Agentic systems compress decision and execution into a single runtime, which means governance has to shift left into the policy layer. The practitioner takeaway is to design for bounded execution first, then add capability, rather than allowing capability to outrun control design.

What this signals

AI-driven AppSec will force security teams to think less about model quality and more about control placement. The programme-level question is whether privileged automation can be constrained with the same discipline used for human or non-human access, because without that discipline the attack surface expands faster than review cycles can keep up.

AI governance debt: if teams add agentic features before defining policy boundaries, auditability, and approval points, they inherit hidden operational risk that compounds over time. That risk is especially relevant where tooling can reach secrets, targets, or production-like environments, because the consequences of a wrong action are immediate and hard to unwind.

Practitioners should expect future AI security tooling to be judged less by raw capability and more by whether it can prove safe execution. For identity-led programmes, the useful comparison is not AI versus no AI, but governed delegation versus uncontrolled delegation. That distinction maps directly to least privilege, session scoping, and evidence retention.

For practitioners

• Build a deterministic action layer Separate model reasoning from privileged execution, and require every sensitive action to pass through explicit policy enforcement, approval rules, and immutable logging.
• Scope tool access like privileged access Treat scanners, exploit chains, and data connectors as privileged resources, then assign only the minimum target, dataset, and command scope required for the task.
• Define human approval points for high-risk steps Place approval gates before target selection, exploit execution, data export, and any action that could affect a live system or expose sensitive information.
• Test for reproducibility and audit completeness Run repeated jobs against the same target set and verify that logs, prompts, tool calls, and outputs let a reviewer reconstruct what the system did.

Key takeaways

• Frontier AI is shifting AppSec from manual testing toward governed delegation, which makes policy enforcement the primary security control.
• The risk is not only model error, but unbounded action, weak auditability, and delegated access that behaves like privileged automation.
• Security teams should design deterministic control layers, scoped tool access, and human approval boundaries before expanding agentic workflows.

Key terms

• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions — including calling APIs, writing code, and orchestrating other agents — with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• Deterministic Enforcement: A policy model that produces a clear allow or block decision based on defined rules rather than probabilistic model judgment. For AI agents, deterministic enforcement is valuable because it can stop risky execution even when language is obfuscated, multi-turn, or designed to evade classifier-based guardrails.
• Human-in-the-Loop (HITL): A governance pattern requiring human approval before an AI agent takes high-impact, irreversible, or out-of-scope actions. HITL is a critical control for agentic AI identity governance.
• Audit Trail: An audit trail is a record of who accessed a system, what they did, and when they did it. For PHI environments, it provides the evidence needed to investigate incidents, support breach determinations, and demonstrate that access was attributable to a specific identity or workflow.

What's in the full article

PortSwigger's full post covers the operational detail this post intentionally leaves for the source:

• How its next-generation Burp AI is structured around an agentic core with policy enforcement and audit logging
• The specific ways the toolset uses deterministic components to make autonomous testing more reliable
• The private beta feedback loop that is shaping the public release before the Black Hat research presentation

👉 The full PortSwigger post covers the agentic architecture, practitioner role changes, and beta details behind the capability shift.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management in practical operational terms. It helps security practitioners build the control discipline needed when software gains delegated access.