Should organisations use synthetic data or real user data for RAG testing?

Why This Matters for Security Teams

RAG testing is only useful when it reflects the conditions that matter in production: the right documents, the right retrieval paths, and the right failure modes. synthetic data is valuable because it can be generated quickly, shaped to cover rare scenarios, and used before sensitive content is ready for wider access. Real user data matters because it reveals how the system behaves against genuine language, real document structures, and unexpected queries. That distinction is central to AI governance and model risk management, not just test design.

Security teams often overfit test suites to neat examples and then discover that retrieval quality drops when users ask ambiguous questions, use domain shorthand, or search across noisy repositories. The governance challenge is to ensure test data supports validation without exposing unnecessary personal or regulated information. For that reason, guidance from NIST Cybersecurity Framework 2.0 is useful as a control lens for data handling, resilience, and verification discipline, even though it is not RAG-specific.

In practice, many security teams encounter retrieval failures only after users hit live content paths that no synthetic test set ever exercised, rather than through intentional validation of production-like behaviour.

How It Works in Practice

The strongest pattern is to split testing into two layers. Synthetic data is used first to create breadth: edge-case queries, adversarial prompts, malformed inputs, rare entities, and intentionally ambiguous phrasing. Real user data is then used to validate realism: whether retrieved passages match actual user intent, whether ranking behaves as expected, and whether the system surfaces the correct source material under normal workload patterns. This mirrors how AI assurance is evolving in current guidance, where coverage and realism are treated as complementary rather than interchangeable.

Practical teams usually start by sanitising real queries and documents, then pairing them with curated synthetic variants. That approach reduces privacy exposure while preserving the signal needed for evaluation. The goal is not to copy production data into test environments indiscriminately, but to preserve salient characteristics such as terminology, document length, formatting quirks, and access boundaries. For retrieval-specific testing, teams should measure whether the right chunks are returned, whether citations are stable, and whether hallucinated or stale content is filtered out. The OWASP Top 10 for Large Language Model Applications is a useful reference for prompt injection, data leakage, and output handling concerns that often show up in RAG pipelines.

• Use synthetic data to probe edge cases, long-tail queries, and safety controls.
• Use real user data to validate retrieval relevance and operational realism.
• Separate test datasets by sensitivity, access scope, and retention rules.
• Check both retrieval quality and answer grounding, not just final response text.

Where possible, teams should also compare test results against logged production behaviour and incident samples, because that exposes drift between expected and observed retrieval outcomes. These controls tend to break down when the knowledge base changes rapidly because the test set becomes stale faster than the evaluation process can be refreshed.

Common Variations and Edge Cases

Tighter use of real user data often increases privacy, legal, and governance overhead, requiring organisations to balance stronger realism against data minimisation and approval burden. That tradeoff is especially visible in regulated environments, where test data can contain personal information, confidential customer content, or privileged operational records. In those cases, best practice is evolving toward de-identification, pseudonymisation, and controlled sampling rather than unrestricted reuse of production data.

There is no universal standard for how much synthetic versus real data is enough. For some teams, synthetic data can cover most functional checks, with a small set of real samples reserved for final validation. For others, especially those supporting highly domain-specific retrieval, synthetic data may miss document layout, naming conventions, or regional terminology that materially affects results. The deciding factor is usually whether the failure mode is semantic, operational, or compliance-related. If the question is “Does the system work in principle?”, synthetic data is often sufficient. If the question is “Will it work for actual users with actual documents?”, real data becomes essential.

For broader governance, the NIST AI Risk Management Framework helps teams document risk, map validation evidence, and define accountability around dataset selection. Where RAG feeds regulated workflows, the choice of test data can also intersect with auditability and retention requirements under the EU AI Act and operational control expectations in CISA Secure by Design.

Related resources from NHI Mgmt Group

• When should organisations use document-level permissions in RAG?
• How should organisations move from reactive data security to a real data protection strategy?
• What should organisations do when access reviews do not match real data exposure?
• How do organisations reduce exposure for data in use?