Join our Newsletter — 33% off our NHI Course

Prompt injection and AI agent trust: what security teams need now

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 19129
Topic starter  

TL;DR: Prompt injection exploits the boundary between user input and system instructions in AI systems, giving attackers a low-skill path to manipulate models that can access data and tools, according to WorkOS's interview with Noam Schwartz. The governance problem is that production AI agents collapse old web security assumptions about instruction trust, making safety, access, and tool-use boundaries inseparable.

Editorial analysis by NHI Mgmt Group, based on content published by WorkOS: “AI is both weapon and target: Noam Schwartz on the new threat landscape”.

Key questions

Q: What breaks when prompt injection reaches a tool-using AI agent?

A: What breaks is the assumption that the model's output is low impact.

Q: Why do AI assistants with tool access increase prompt injection risk?

A: Tool access converts prompt injection from a wording issue into an execution issue.

Q: How should security teams reduce prompt injection risk in AI agents?

A: Security teams should reduce prompt injection risk by constraining what enters the context window, limiting tool permissions, and separating untrusted retrieval content from privileged instructions.

Practitioner guidance

  • Constrain agent tool scopes Map every tool an AI agent can invoke and remove any access that is not required for a single task or session.
  • Isolate untrusted prompt content Separate retrieved content, user input and system instructions so attacker-controlled text cannot freely override control intent.
  • Log tool decisions and prompt context Capture which input sources influenced each tool call, what data was accessed and what action was taken.

Bottom line: Prompt injection is dangerous because it converts untrusted text into effective control over an AI agent's behaviour and tool use.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 18837
 

Prompt injection is an instruction-boundary failure, not a content-moderation edge case. The relevant governance problem is that the model cannot reliably distinguish data from directives once both occupy the same runtime context. That means old application assumptions about trusted system prompts and untrusted user input no longer hold. Practitioners should treat this as a control-plane issue for AI agents, not a text-filtering problem.

A few things that frame the scale:

A question worth separating out:

Q: What is the difference between prompt injection and SQL injection from a defender's point of view?

A: SQL injection is prevented by a parser that enforces a hard boundary between query syntax and parameters. Prompt injection targets a language model that has no equivalent parser or privilege bit. The model’s instruction following is learned behavior, so the practical defense shifts from input separation to output containment, tool scoping, and execution policy at the application boundary.

👉 Read our full editorial: Prompt injection is the new SQL injection for AI agents



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.