Join our Newsletter — 33% off our NHI Course

LLM agent design patterns: what they still leave open in practice

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 16626
Topic starter  

TL;DR: Six LLM agent security patterns constrain control flow, but still leave a residual in most real assistants, according to ARMO’s analysis. Attackers can still shape arguments, rankings, recipients, placements and summaries even when they cannot redirect the tool choice, and that distinction matters because runtime abuse survives where output space remains open.

Editorial analysis by NHI Mgmt Group, based on content published by ARMO: “Design Patterns for Securing LLM Agents Against Prompt Injection: What Every Pattern Leaves Open”.

Key questions

Q: What breaks when LLM agents can still shape arguments inside approved actions?

A: The control flow may remain intact, but the payload becomes attacker-influenced.

Q: Why do LLM agent design patterns reduce but not eliminate prompt-injection risk?

A: They reduce risk because they stop untrusted text from steering the tool choice or sequence.

Q: How should security teams judge whether an agent output space is too open?

A: A useful test is whether the task can be reduced to a rigid schema without destroying the product value.

Practitioner guidance

  • Define the residual each agent design leaves open Document whether the open risk is argument shaping, ranking manipulation, output tampering or file placement, then map compensating controls to that specific residual.
  • Constrain assistant output to rigid schemas where possible Use structured fields for classification, routing and extraction tasks so the model cannot inject free text into high-risk responses.
  • Baseline approved actions at runtime Compare each agent’s actual tool use, arguments and sequence against its own normal behaviour so unusual but authorised calls stand out.

Bottom line: The article’s core point is that prompt-injection patterns mainly defend action selection, while the values carried by authorised actions can still be manipulated.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 5 months ago
Posts: 17057
 

Control-flow integrity is necessary, but it is not the same as agent security. The article’s central finding is that six design patterns can prevent injected text from choosing the action, yet they do not govern the values carried by the action itself. That distinction matters because enterprise risk often appears inside an authorised workflow rather than at the point of tool selection. Practitioners should read this as a boundary statement, not a green light.

A few things that frame the scale:

  • AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.

A question worth separating out:

Q: What should organisations do when inherited AI agents already use these patterns?

A: Assume the architecture decision is already made and focus on what remains observable at runtime. Inventory the approved actions, identify the values they carry, and baseline the workload so abnormal arguments or call sequences stand out. Inherited agents need behavioural controls because the design pattern alone does not close the residual.

👉 Read our full editorial: LLM agent design patterns stop control-flow hijack, not residual abuse



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.