RAG vs Fine-Tuning: Security and Data Exposure Compared

When teams decide how to give a language model their enterprise data, they usually compare cost and answer quality, and only later find that retrieval-augmented generation (RAG) and fine-tuning expose that data in very different ways. RAG keeps the data in a store outside the model and fetches it at query time, so access control, deletion and audit can still work per document. Fine-tuning changes the model's weights, so whatever the training set contained can surface for anyone who can query the model, and removing it later means retraining. This guide compares the two from a security point of view: data exposure, access control, poisoning, deletion, the identities involved and how to monitor each. It is for AI platform teams, security architects and data owners deciding how to connect models to internal data. Platform examples are listed alphabetically. For deep coverage of each side see the RAG Security Guide and the Fine-Tuning and LoRA Security Guide.

Key takeaways

RAG can respect permissions; fine-tuning cannot. A retrieval layer can filter documents per user at query time, while a fine-tuned model gives every caller the same weights and so the same memorised data.
Fine-tuning data can leak verbatim. Research on extracting training data showed hundreds of verbatim sequences, including personal data, recovered from a language model, some from a single training document.
RAG moves the risk to retrieval. Over-shared source data, missing filters, poisoned documents and indirect prompt injection through retrieved content are the main RAG exposures.
Deletion is the sharpest difference. Removing a document from a RAG index takes effect on the next query; removing data a model has learned requires retraining or discarding the fine-tuned model.
Use fine-tuning for behaviour, RAG for facts. Fine-tune on curated, non-sensitive examples of style, format or task; keep sensitive or permissioned knowledge in a governed retrieval store.

What each approach does with your data

Retrieval-augmented generation (RAG): documents are split into chunks, converted to embeddings and stored in a vector index or search service. At query time the application retrieves the most relevant chunks and adds them to the prompt. The model itself is unchanged. OWASP describes RAG as a model adaptation technique that combines pre-trained language models with external knowledge sources. Microsoft describes its "on your data" feature in the same terms: the data stays in the data source the customer designates, and the service retrieves relevant data and augments the prompt.

Fine-tuning: a base model is trained further on a labelled dataset, either updating all weights or training a small adapter such as LoRA. The result is a new model or adapter file. The training data is no longer looked up; its patterns, and sometimes its exact content, are encoded in the weights. Managed services typically keep the fine-tuned model private to the customer: Microsoft states that fine-tuned models are exclusively available to the customer whose data was used to create them, are encrypted at rest when not deployed, and can be deleted by the customer at any time.

Many production systems use both: a fine-tuned model for tone, format or tool-calling behaviour, with RAG supplying current, permissioned facts.

Security comparison

Dimension	RAG	Fine-tuning
Where sensitive data lives	Source systems plus a copy in chunks and embeddings in the index	Training dataset, checkpoints, and encoded in the model weights
Per-user access control	Possible: filter retrieval by the caller's identity and groups	Not possible inside the model; control only who can call the model
Main confidentiality risk	Retrieval returns documents the user should not see; cross-tenant leaks in a shared index	Memorised training data extracted through prompts
Main integrity risk	Poisoned or attacker-written documents in the corpus; indirect prompt injection	Poisoned training examples and backdoors that trigger on specific inputs
Deleting a record (for example a data subject request)	Delete from source and index, effective on next query	Retrain without the record or retire the model
Freshness of permissions	As fresh as the ACL sync into the index	Fixed at training time
Audit trail	Which chunks were retrieved for which user can be logged	Cannot show which training example shaped an answer
Non-human identities involved	Connector or crawler accounts reading sources; the app identity querying the index	Training job identity; storage access for datasets and model artefacts
Artefact supply chain	Embedding model, index, connectors	Base model, adapters, training code and serialised model files

Data exposure in depth

Fine-tuning: memorisation and extraction

Carlini and colleagues, in "Extracting Training Data from Large Language Models", recovered hundreds of verbatim sequences from GPT-2, including names, phone numbers and email addresses, even where a sequence appeared in only one training document, and found that larger models were more vulnerable than smaller ones. NIST AI 600-1, the Generative AI Profile of the AI Risk Management Framework, describes this as data memorisation and notes that it may pose privacy risks even for data present in a small number of training samples. The practical rule is simple: anything in a fine-tuning set should be treated as potentially retrievable by every user of the resulting model. OWASP LLM02:2025 Sensitive Information Disclosure recommends sanitising data before it enters training and lists differential privacy as a mitigation. MITRE ATLAS AML.T0024 Exfiltration via AI Inference API covers the attacker side.

RAG: over-retrieval and leakage across contexts

OWASP LLM08:2025 Vector and Embedding Weaknesses lists unauthorised access and data leakage through misaligned access controls, cross-context leaks where several user groups or applications share one vector database, and embedding inversion, where text is partly recovered from vectors. RAG also tends to surface data that was technically accessible but never found: if a file share is open to everyone, a RAG assistant will cite it to everyone. Fix over-sharing in the source before indexing. See the Permission-Aware RAG Guide and OWASP LLM Top 10 LLM08 Vector and Embedding Weaknesses Guide.

Poisoning and injection

OWASP LLM04:2025 Data and Model Poisoning covers manipulation of pre-training, fine-tuning and embedding data, including backdoors that leave behaviour unchanged until a trigger appears. For fine-tuning, poisoned examples change the model itself (ATLAS AML.T0020 Training Data Poisoning, AML.T0018 Manipulate AI Model). For RAG, a poisoned or attacker-supplied document changes answers for as long as it stays in the index (AML.T0070 RAG Poisoning), and instructions hidden in retrieved content act as indirect prompt injection (AML.T0051 LLM Prompt Injection). The EchoLeak case showed the pattern: a crafted external email retrieved as context carried instructions that could send the user's data out through a link, and it was fixed by the vendor before any known exploitation. The lesson for RAG owners is to keep untrusted content apart from sensitive content in retrieval scope. See the AI Data Poisoning and Backdoors Guide.

Access control: how each can be enforced

RAG access control is only as good as the identity passed to the retrieval layer. Search and vector services offer document-level filtering, but the application must supply a verified identity:

Amazon Bedrock Knowledge Bases: retrieval accepts metadata filters with operators such as equals, in and listContains, combined with andAll and orAll. AWS documents these for relevance, so an access filter built on them must be applied by your application on every query from the caller's verified attributes.
Azure AI Search: offers built-in document-level ACL support and, where that cannot be used, a security filter pattern on a field such as group_ids. Microsoft notes that the principal in a security filter is just a string with no authentication behind it, and that the filter must be applied to every query.

Common patterns, in order of strength:

Separate indexes per audience or tenant: strongest isolation, simple to reason about, costs more to run.
Per-document ACLs synced from the source, filtered by the caller's identity: fine-grained, depends on sync freshness and on the app passing the right identity.
Post-retrieval permission check against the source system: most accurate, adds latency.
Index only data everyone in scope may see: simplest, suitable for public or all-staff content.

Fine-tuned models allow only the coarse control of who may call the model endpoint. If two groups must not see each other's data, they need separate fine-tuned models or no sensitive data in training at all. Never rely on the system prompt to withhold data: OWASP LLM07:2025 states that the system prompt should not be used as a security control.

Which to use: decision table

Requirement	Prefer	Reason
Users have different permissions on the source data	RAG with permission-aware retrieval	Only retrieval can apply per-user access at query time
Data subject to deletion or correction requests (personal data)	RAG	Delete from index; no retraining
Data changes often (policies, tickets, prices)	RAG	Index refresh instead of retraining
Need citations and an audit trail of sources	RAG	Retrieved chunks can be logged and shown
Teach a format, tone, classification task or tool-calling behaviour	Fine-tuning on non-sensitive examples	Behaviour, not facts; no permission boundary needed
Strict latency or offline deployment with no retrieval infrastructure	Fine-tuning, with a sanitised dataset	Accept that every user sees what the model learned
Data is secret even from most internal users (M&A, HR cases)	Neither by default; if needed, RAG with a separate index	Keep out of shared models entirely
Both behaviour and current facts needed	Fine-tune for behaviour, RAG for facts	Keeps sensitive knowledge in the governed store

Small teams: start with RAG over a narrow, all-staff corpus and a managed service; avoid fine-tuning on internal documents. Large organisations: set a data classification rule for what may enter a fine-tuning set, require a data owner sign-off, and run RAG indexes per classification or business unit with ACL sync.

Identities and pipeline security

Both approaches create non-human identities that often hold more access than any user. RAG connectors crawl SharePoint, wikis, ticketing and storage with broad read rights; fine-tuning jobs read datasets and write model artefacts. Give each its own identity, scoped to the sources it needs, with short-lived credentials from workload identity rather than stored keys. Keep secrets out of the corpus: ATLAS AML.T0082 RAG Credential Harvesting describes attackers using a RAG system to find credentials in indexed documents. For training pipelines, verify base models and adapters before loading, and confirm that fine-tuning has not weakened safety and security behaviour, as NIST AI 600-1 action MS-2.7-008 suggests. See the AI data access review guide.

Implementing safely

Classify the data first. Decide what may enter a RAG index and what may enter a training set. Default sensitive and permissioned data to RAG only.
Clean the sources. Fix over-shared folders and remove secrets before indexing or training; scan datasets for personal data and credentials.
Build access control into retrieval. Pass the caller's verified identity to the retrieval layer and apply filters on every query; test with users from different groups.
Pilot with red-team prompts. For RAG, try to retrieve documents across groups and plant test documents with injected instructions. For fine-tuning, probe for memorised records using canary strings inserted into the training set.
Log retrieval and model access. Record user, query, retrieved document IDs and response metadata.
Plan rollback. For RAG, keep the previous index version and be able to remove a source in minutes. For fine-tuning, keep the previous model version deployed-ready and be able to switch traffic back.

Detection and measurement

Retrieval logs: OWASP LLM08 recommends detailed immutable logs of retrieval activity. Alert on retrieval of high-sensitivity labels, unusual volumes per user and retrieval across tenant boundaries.
Corpus changes: monitor new or changed documents from external or low-trust sources and documents containing instruction-like text.
Canary detection: canary strings placed in training sets or restricted documents that appear in outputs show memorisation or a retrieval leak.
Pipeline identities: watch connector and training job identities for access outside their sources.

Metric	Target
RAG queries served with an identity-based filter applied	100% for permissioned corpora
Cross-group retrieval test cases passing	100% before each release
Fine-tuning datasets with recorded classification and owner sign-off	100%
Canary strings surfacing in model output	Zero
Time to remove a document from all RAG indexes after a deletion request	Within your documented SLA, for example 24 hours
Secrets found in indexed or training data	Zero after each scan

Common failure patterns

Fine-tuning on support tickets or email. Personal and customer data becomes part of a model every user can query.
One shared index for all users. Retrieval ignores source permissions, so the assistant answers from HR or finance documents for anyone.
Trusting a string as identity. The application passes a group name taken from user input into the filter instead of from a verified token.
Stale ACLs. A leaver or a revoked share still appears in the index because permission sync runs weekly.
Untrusted content in the corpus. External email, web pages or public tickets are indexed beside sensitive data, which enables indirect prompt injection.
No way to forget. A deletion request arrives for data that was used in fine-tuning and there is no retraining plan.

Framework mapping

Framework	Reference	Relevance
OWASP Top 10 for LLM Applications 2025	LLM02 Sensitive Information Disclosure; LLM04 Data and Model Poisoning; LLM08 Vector and Embedding Weaknesses	Memorisation, poisoning and retrieval access control
MITRE ATLAS	AML.T0070; AML.T0051; AML.T0020; AML.T0018; AML.T0024; AML.T0082	RAG poisoning, prompt injection, training poisoning, model manipulation, extraction and credential harvesting
NIST AI 600-1	Data Privacy risk; MS-2.5-005; MS-2.7-008	Memorisation risk, data provenance and safety checks after fine-tuning
NIST SP 800-53 Rev 5	AC-3 Access Enforcement; AC-4 Information Flow Enforcement; AC-6 Least Privilege; AU-6 Audit Record Review, Analysis, and Reporting	Retrieval filtering, separation of corpora, pipeline identity scope, retrieval log review
NIST CSF 2.0	PR.AA-05; PR.DS-01; ID.AM-07	Least privilege; protection of data at rest; inventory of data used for RAG and training

Practitioner checklist

Write a rule for which data classifications may enter a RAG index and which may enter a fine-tuning set.
Keep permissioned and personal data out of fine-tuning sets; use RAG for it.
Apply identity-based retrieval filters on every query, from a verified token, never from user input.
Separate indexes where tenants or groups must never see each other's data.
Scan sources and datasets for secrets and personal data before indexing or training.
Keep untrusted external content out of indexes that hold sensitive data.
Give RAG connectors and training jobs their own scoped, short-lived identities.
Log retrieval with user, query and document IDs, and review high-sensitivity hits.
Test with canary strings and cross-group queries before each release.
Document how a deletion request is handled for both indexes and fine-tuned models.

Worked scenarios

The scenarios below are illustrative examples, not real incidents.

RAG scenario: an HR policy assistant. An organisation builds an assistant that answers staff questions from its HR policy library, including some manager-only documents. Each chunk is indexed with the source document's access group, and the retrieval layer filters results by the caller's groups at query time. When a manager-only salary banding document is found to be mislabelled, the team corrects the label and re-indexes it; the change takes effect on the next query without touching the model.

Fine-tuning scenario: a support reply style model. A support team wants replies in a consistent tone and format. It fine-tunes a model on 2,000 curated past replies after removing customer names, account numbers and free-text notes, and keeps product facts in a separate retrieval store. A later review finds one training example still contained a customer email address. Because removing learned data needs retraining, the team rebuilds the dataset with an automated personal data scan, retrains the adapter and deletes the earlier fine-tuned model.

RAG vs Fine-Tuning: Data Exposure, Access Control and Security Trade-offs