Notifications
Clear all
Topic starter
02/08/2026 12:28 pm
TL;DR: Agentic AI introduces security risk when LLMs can plan, use tools, and persist memory, while Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027 and the article highlights real misuse patterns such as ransomware advice, fraud, and data extortion, according to Promptfoo. The governance challenge is that autonomy without constrained access turns AI systems into software with broader blast radius, not human-like judgment.
NHIMG editorial — based on content published by Promptfoo: LLM autonomy, agency, and the security risks of agentic AI
By the numbers:
- Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027.
- AI spending is to hit USD 375 billion this year alone, and USD 500 billion next year.
- A cybercriminal targeted 17 organizations as a part of a data extortion operation that sometimes exceeded USD 500,000 in the demanded ransoms.
Questions worth separating out
Q: How should security teams govern AI agents that can choose tools at runtime?
A: Security teams should govern runtime agent choice as an access event, not as a simple application action.
Q: Why do AI agents create more risk than traditional automation?
A: AI agents create more risk because they can interpret context, choose actions, and invoke tools autonomously.
Q: What breaks when AI agents are given broad standing access?
A: Broad standing access breaks governance because the agent can move from one task to another without a fresh authorization check.
Practitioner guidance
- Define agent permission boundaries before deployment Map every model and agent to a specific identity, then grant only the APIs, files, and workflows required for the task.
- Constrain tool use with step-up approval Require human confirmation before a model can call sensitive systems, move data, or trigger irreversible actions.
- Log and audit every agent action trail Capture prompts, tool calls, outputs, and delegated credentials in a tamper-evident audit trail so investigations can reconstruct what the system actually did.
What's in the full article
Promptfoo's full article covers the operational detail this post intentionally leaves for the source:
- The article expands on practical limits for memory, tool access, and recursive prompting that determine whether an AI system can act safely.
- It outlines concrete containment measures such as sandboxing, input filtering, output validation, and human confirmation before access is granted.
- It discusses real-world misuse patterns, including ransomware support, fraud, fake identity creation, and extortion workflows.
- It frames the accountability problem for autonomous behaviour and why designers, deployers, and users all need explicit responsibility boundaries.
👉 Read Promptfoo's analysis of autonomy, agency, and AI security risk →
AI agent autonomy and agency: are security controls keeping up?
Explore further
02/08/2026 1:27 pm
Autonomy without bounded identity is a governance failure, not a product feature. Once an AI system can select actions and invoke tools, it needs the same access discipline applied to other privileged software. The article is right to push back on anthropomorphic framing because the operational question is identity, permissions, and control scope. For practitioners, the implication is clear: if you cannot bound the agent, you cannot safely deploy it.
A question worth separating out:
Q: Who is accountable when an AI agent takes a harmful action in healthcare?
A: Accountability should remain with the human or team that deployed and authorised the agent, not with the model itself. The organisation needs named ownership, scope definitions, and logs that tie each action to an identity. Without that chain of responsibility, agentic behaviour becomes operationally opaque and difficult to defend in audits or investigations.
👉 Read our full editorial: AI agent autonomy and agency are widening security risk faster than controls
