Join our Newsletter — 33% off our NHI Course

LLM function-calling risks: are your tool boundaries strong enough?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 17435
Topic starter  

TL;DR: LLM applications that call external tools expand capability, but they also enlarge the attack surface through SSRF, prompt injection, excessive delegation, and credential leakage, according to Flatt Security. The practical lesson is that tool design, context separation, and least privilege must be treated as core security architecture, not post-launch hardening.

Editorial analysis by NHI Mgmt Group, based on content published by Flatt Security: “Securing LLM Function-Calling: Risks & Mitigations for AI Agents”.

Key questions

Q: What breaks when LLMs can call external tools without strict boundaries?

A: The main failure is that the model can convert untrusted text into real action.

Q: Why do AI toolchains increase risk when credentials and context are mixed?

A: Because the model can only act on what enters its context, and anything in that context may be influenced by attacker-controlled text.

Q: How can security teams tell whether LLM tool boundaries are too loose?

A: The clearest sign is when one tool can both retrieve untrusted content and take privileged action based on it.

Practitioner guidance

  • Boundary external tools by function Split retrieval, write, and administrative actions into separate tools with different permissions so the model cannot mix harmless and sensitive operations in one call.
  • Issue fine-grained credentials only Use scoped tokens for each tool and avoid broad personal access tokens or generic service credentials that can be reused across unrelated tasks.
  • Keep secrets out of model context Store and use credentials in trusted application logic rather than passing them through prompts, retrieved text, or other model-visible context.

Bottom line: LLM function-calling turns model output into delegated execution, so security must focus on tool permissions and trust boundaries rather than prompts alone.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 17367
 

Tool calling creates an identity problem, not just an LLM problem. Once the model can select actions that affect external systems, the security question becomes who or what is authorised to act on behalf of the user. That shifts the centre of gravity from prompt hygiene to delegated authority, scoped credentials, and action boundaries. Practitioners should assess AI toolchains as access paths with their own governance lifecycle, not as harmless automation.

A few things that frame the scale:

  • 62% of organisations are at least experimenting with AI agents, but only 23% have scaled one into a single business function, according to McKinsey's 2025 State of AI survey.

A question worth separating out:

Q: What should teams do when function-calling is needed for Git or web access?

A: Treat each integration as a separate delegated capability with its own scope, approval logic, and offboarding path. Git write access, URL fetching, and administrative actions should not share the same identity or token. That keeps a compromised or confused model from turning one task into broader system reach.

👉 Read our full editorial: LLM function-calling security needs least privilege and boundary separation



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.