How should security teams secure GenAI applications that use retrieval-augmented generation and plugins?

Security teams should start with visibility into every component in the GenAI stack, including models, vector databases, prompts, third-party libraries, and tools. Then they should apply scoped access, strong input validation, sandboxing for script execution, and human approval before risky actions. The core goal is to reduce prompt injection, data leakage, and unintended tool use before users can abuse the system.

Why RAG and Plugins Change the GenAI Security Boundary

Retrieval-augmented generation and plugins extend a GenAI application beyond the model itself. The security boundary now includes document stores, retrieval pipelines, prompt construction, external APIs, and any action the application can trigger on a user’s behalf. That creates a different risk profile from a standalone chatbot, because the system can be pushed to reveal retrieved content, follow hostile instructions, or execute tool actions that were never intended by the operator. The practical challenge is that each added component can become both a data source and an attack surface, so the team needs to govern trust at every handoff. NIST AI 600-1 GenAI Profile is useful here because it frames GenAI risk as a system-level governance problem, not just a model problem. In practice, many teams discover the weakest control only after a plugin has already been allowed to act on untrusted input.

How Retrieval, Prompting, and Tool Use Fail Together

RAG security depends on controlling what the model can see, what it can quote, and what it can do with that information. A retrieved passage may be useful for grounding the answer, but it should not be treated as trusted instruction. If the retrieval corpus contains hostile or low-trust content, the model may surface it into the prompt context and then follow it as if it were part of the task. That is why teams need content boundaries between indexed knowledge, system instructions, and user input. The same logic applies to plugins: the model may select the tool, but the tool should still enforce its own authorization, input checks, and output limits.

Good practice is to separate the problem into layers:

constrain retrieval to approved data sources and approved document scopes
sanitize or label retrieved text before it enters the model context
treat plugin calls as privileged actions with explicit policy checks
require human review for destructive, financial, or external-facing actions
log the retrieved material, prompt content, and tool decision for later review

This matters because a secure model can still be embedded in an insecure workflow. Even strong model behavior does not protect against a malformed retrieval object, an over-permissive connector, or a plugin that accepts commands without validating intent. NIST SP 800-53 Rev 5 Security and Privacy Controls is relevant where teams need a control baseline for access enforcement, logging, and separation of duties. The guidance breaks down when the application treats every retrieved snippet as equally trustworthy or lets tool selection proceed without independent authorization.

Where the Edge Cases Create the Most Trouble

Tighter isolation often increases product friction, requiring organisations to balance user experience against the risk of prompt injection and unsafe tool execution.

One edge case is internal retrieval. Teams sometimes assume internal documents are safe because they are not public, but insider misuse, poisoned content, or overbroad indexing can still turn internal knowledge into an attack vector. Another common variation is a plugin that only reads data at first. Even read-only tools can leak sensitive content if the model is allowed to summarise, transform, or repackage retrieved data for the wrong audience.

Another practical nuance is that not every plugin deserves the same trust model. A calendar lookup, ticket creation tool, and payment-related action should not share the same approval path. Guidance versus consensus is still evolving on how much autonomy to grant to agentic workflows, but there is broad agreement that higher-impact actions need stronger confirmation and narrower scopes. Teams should also remember that vector search can amplify exposure if embeddings or chunking make it easier to surface material out of context. Security tests should therefore include hostile prompts, poisoned retrieval items, and tool-calling abuse rather than only conventional web-app testing. The answer stops being reliable when the application cannot prove which content influenced a decision or which identity authorised the tool call.

Risk and Threat Considerations

GenAI applications with RAG and plugins create a composite risk of prompt injection, data exfiltration, and unauthorised action. The main exposure is not just model misuse, but trust transference across retrieval, generation, and execution boundaries.

Failure mechanism: An attacker places hostile instructions in retrieved content, manipulates a user prompt, or abuses a plugin path so the model incorporates untrusted text and then acts on it. If tool permissions are broad or output validation is weak, the system can leak sensitive material, call unintended functions, or move data to an external service.

Impact: Sensitive records can be disclosed, business actions can be triggered without proper approval, and the organisation may lose control over what the application retrieved, summarised, or executed.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

MITRE ATLAS and OWASP Agentic AI Top 10 address the attack and risk surface, while NIST AI RMF, NIST AI 600-1 and CIS Controls v8 set the governance and control requirements practitioners need to meet.

Framework	Control / Reference	Relevance
NIST AI RMF	GOVERN-GenAI — Generative AI Governance	GenAI app governance spans retrieval, tools, and model risk.
Recommendation — Establish governance for retrieval sources, tool approval, and system-level GenAI risk.
NIST AI 600-1	MAP-1 — Map the AI System	Maps the full GenAI stack and its dependencies before controls are applied.
Recommendation — Map every model, connector, retriever, and plugin before authorizing use.
CIS Controls v8	6 — Access Control Management	Scoped access and separation of duties are central to plugin safety.
	16 — Application Software Security	RAG and plugin pipelines need secure input handling and validation.
Recommendation — Restrict plugin and data-store access to the minimum required for each workflow. Validate retrieved content and tool inputs before they influence application behavior.
MITRE ATLAS	AML.TA0001 — Reconnaissance	Prompt injection and tool abuse are adversarial techniques against GenAI systems.
Recommendation — Hunt for prompt-injection and tool-abuse patterns in GenAI abuse testing.
OWASP Agentic AI Top 10	A2 — Tool and Action Safety	Plugin execution safety is central when models can trigger external actions.
Recommendation — Gate high-impact tool calls behind policy checks and human approval.

Practitioner Guidance

What to prioritise: Treat retrieval and tool use as the highest-risk parts of the stack, not the model alone. Security teams should decide which data sources are eligible for retrieval, which tools are allowed to execute, and which actions always require approval before deployment.

What to verify: Confirm that each plugin has its own authorization boundary, that retrieved content is traceable back to source, and that logs preserve the prompt, retrieved passages, and tool invocation context. If a team cannot explain why a tool was called, it does not yet have enough governance to trust the workflow.

Common mistake: Teams often harden the model interface while leaving the retrieval corpus, connector credentials, and downstream APIs over-permissive. That creates a false sense of safety because the model may behave correctly even as the application around it leaks data or executes risky actions.

Practitioner takeaway: The real control objective is to make every handoff in the GenAI workflow explicit, bounded, and reviewable, because most failures occur at the seams between retrieval, prompting, and execution rather than inside the model itself.