Function-call injection

How function-call injection works

Function-call injection happens when a model's structured output is treated as an executable tool request rather than just text. The attack succeeds because the host application trusts the shape of the response, not because the prompt alone is persuasive.

This makes the boundary between generation and action the critical security seam. A harmless-looking response can become a command to retrieve data, call an API, modify state, or trigger another workflow if the application routes it into a tool executor without sufficient validation.

Why it is different from ordinary prompt injection

Ordinary prompt injection tries to steer the model's answer. Function-call injection tries to steer the application's behavior by exploiting the parser, dispatcher, or orchestration layer that turns model output into tool invocation. The real target is the control plane around the model, not just the model itself.

That distinction matters because the consequences are often operational, not merely linguistic. If the host accepts attacker-shaped arguments, the model can become a relay for unauthorized actions even when the generated wording looks syntactically valid and internally consistent.

Where the security boundary breaks

The failure usually sits in one of three places: weak schema validation, overbroad tool permissions, or unsafe assumptions about model honesty. Structured output can still be malicious if the application fails to constrain which tools may be called, which arguments are acceptable, and which actions require additional approval.

Systems that blend natural language with automation often need the same discipline you would apply to any privileged API surface. The output must be treated as untrusted input until it passes policy checks, authorization checks, and context checks appropriate to the action being requested.

For a broader view of the application-security baseline, the OWASP Top 10 remains the simplest reference point for understanding why untrusted input should never flow directly into privileged behavior.

Common abuse patterns and failure modes

Attackers often aim for tool calls that retrieve secrets, expose sensitive records, submit unwanted transactions, or chain into additional internal actions. In practice, the danger is magnified when the application assumes the model will only emit well-formed requests and does not separately verify intent, scope, or user authorization.

The other common failure is overconfidence in the model's own guardrails. A model can be socially or contextually manipulated into producing a valid function call that the host system should have rejected at the orchestration layer. The weakness is the trust relationship between generated structure and executed privilege.

For teams securing AI-enabled integrations, the OWASP API Security Top 10 is useful because many failures in function-call injection resemble API authorization and object-access defects once the request reaches the backend.

Risk and Threat Considerations

Function-call injection matters because it can convert a model output compromise into unauthorized execution, data exposure, or business-logic abuse. The risk is highest when the application grants the model access to sensitive tools, privileged workflows, or state-changing actions without a strong trust boundary.

Failure mechanism: An attacker shapes the model response so the host application interprets attacker-controlled output as a legitimate function call, then forwards it to a privileged tool or backend action.

Impact: This can produce unauthorized API calls, data theft, unwanted transaction execution, privilege abuse, or chained compromise across connected systems.

Practitioner Guidance

Why practitioners should care: Treat tool invocation as an authorization problem, not just a prompt-formatting problem. If a model can trigger actions, the application needs explicit policy enforcement around which functions are available, which inputs are allowed, and when human or programmatic approval is required.

Common misunderstanding: A syntactically valid function call is not the same thing as a safe one. The caller, the context, and the requested action still need independent validation before execution.

Practitioner takeaway: The safest pattern is to make the model propose action and make the host application decide whether the action is permitted.

Related resources from NHI Mgmt Group

• What breaks when prompt injection reaches a model that can call tools?
• How do security teams decide whether to block prompt injection at discovery time or at call time?
• What breaks when applications are not designed to resist code injection and function hooking?
• How should security teams reduce the risk of prompt injection in LLM applications that call third-party libraries?