Treat RAG as a governed workflow, not a single model call. Define approved data sources, log retrieval and tool activity, scope every connector credential, and require evidence that the system can be monitored for privacy leakage, hallucination, and unauthorized access before it is used in sensitive processes.
What governance should cover before a RAG system reaches production?
RAG governance should start with the workflow, not the model. The hard questions are which sources are allowed, who can retrieve from them, what the system is permitted to reveal, and how much of the retrieval and connector activity is observable once the application is live. If those basics are undefined, the system can look accurate while still being unsafe.
That is why the retrieval layer deserves the same control discipline as the generation layer. If the team cannot explain what content was fetched, why it was fetched, and under what entitlement, then the application is not yet ready for sensitive use. For permission-aware retrieval concerns, the most useful starting point is Permission-Aware RAG Guide.
Connector governance matters as much as prompt governance. Every external system, vector store, indexer, or API integration introduces a credential, a trust boundary, and a failure mode, so pre-production review should verify ownership, rotation expectations, and blast radius before the first user query is allowed through.
Which controls matter most for source control, retrieval, and connectors?
The first control is source approval. A team should know which repositories, document stores, ticket systems, and knowledge bases may be indexed or queried, and which categories are excluded because they are too sensitive, too stale, or too broad for the intended use case. “Search everything” is not a governance strategy.
The second control is retrieval authorization. A RAG system should not be allowed to surface material that the requesting user or calling process could not legitimately access outside the model. That includes document-level permissions, connector scopes, and any cross-tenant or cross-environment boundaries that could let one user infer another user’s information.
The third control is connector containment. Treat each connector credential as an operational dependency with explicit scope, expiry, and monitoring. If the credential can write, modify, or trigger downstream actions, the review should classify it as higher risk than read-only access and require a tighter approval path.
For teams building AI assistants and agentic workflows around connectors, Low-Code Agent Platform Security Guide is useful because the same patterns recur: maker credentials, connector policies, sharing limits, and the need to watch what the system can do, not just what it says.
Pre-production governance should also include logging design. The application needs enough telemetry to reconstruct retrieval decisions, connector calls, and sensitive response paths without logging the full secret material or creating a new data exposure problem. If the logs cannot support review, incident response, or privacy investigation, the governance model is incomplete.
What should teams prove before sensitive use is approved?
Before production approval, teams should be able to show that the system has been tested for the failure modes that make RAG unsafe in practice: over-sharing from retrieval, prompt and context leakage, hallucinated answers presented as facts, and unauthorized access through a connector or index. The point is not to prove perfection. The point is to prove that the system has boundaries, that the boundaries are measurable, and that the team knows how they fail.
Teams should also verify that the monitoring model matches the intended use case. A RAG system used for internal knowledge search can tolerate different controls than one used in customer support, regulated decisioning, or any workflow that could expose personal or confidential information. The higher the consequence of a bad answer, the stronger the evidence needed before go-live.
Pre-production review should include a rollback or disablement path. If retrieval begins to leak data, if a connector is abused, or if answer quality degrades in a way that changes business risk, the team needs a fast way to suspend retrieval, revoke the relevant credential, or narrow the approved data set without waiting for a full rebuild.
Risk and Threat Considerations
RAG systems fail most often at the seams, not in the language model itself. The main risks are data overexposure, credential abuse, and false confidence in outputs that look grounded because they cite retrieved content. Once a connector or index is over-broad, the model can surface information that was never meant for that requester, and that error is hard to detect after the fact.
Failure mechanism: Weak source approval, excessive connector scope, or missing retrieval checks lets the application pull sensitive material into context and then reproduce it in a seemingly legitimate answer. Over time, the same weakness can create recurring leakage rather than a one-off incident.
Impact: The result can be privacy exposure, policy violations, unauthorized disclosure, and business decisions made on a response that appeared trustworthy but was not properly bounded.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST AI RMF, NIST SP 800-53 Rev 5 and OWASP ASVS set the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | Govern | RAG pre-production governance needs lifecycle oversight for data, risk, monitoring, and accountability. |
| Recommendation — Establish governance, testing, and monitoring requirements before allowing RAG into sensitive workflows. | ||
| NIST SP 800-53 Rev 5 | AU-2 — Event Logging | RAG governance depends on logging retrieval and connector activity for review and incident response. |
| AC-6 — Least Privilege | Connector credentials and retrieval paths should be scoped to the minimum access needed. | |
| IA-5 — Authenticator Management | Connector and tool credentials used by RAG require lifecycle control, rotation, and scope management. | |
| Recommendation — Log retrieval, connector, and access events needed to reconstruct RAG behavior. Constrain RAG connectors and retrieval permissions to least privilege. Manage and rotate RAG connector credentials with tight scope and expiry. | ||
| ISO/IEC 27001:2022 | A.5.15 — Access control | RAG governance needs defined access rules for approved sources and retrieval paths. |
| A.5.23 — Information security for use of cloud services | Many RAG stacks rely on cloud-hosted models, indexes, or connectors with shared responsibility. | |
| Recommendation — Define and enforce access rules for RAG data sources and connector paths. Review cloud-hosted RAG dependencies and assign security responsibilities clearly. | ||
| OWASP ASVS | V8 — Authorization | RAG applications must prevent responses from exposing information outside the requester’s entitlement. |
| V16 — Security Logging and Error Handling | RAG monitoring needs logs that support detection of leakage, misuse, and failure states. | |
| Recommendation — Enforce authorization checks on retrieved data and generated responses. Instrument logging and error handling to detect unsafe retrieval and disclosure. | ||
Practitioner Guidance
What to prioritise: Start with the highest-risk retrieval paths, especially any connector that can reach sensitive, regulated, or shared repositories. If the system cannot show whose permissions govern a fetch, treat that path as blocked until the control model is explicit.
What to verify: Confirm that retrieval logs, connector audit trails, and alerting are sufficient to answer three questions after any issue: what was fetched, who or what was authorized to fetch it, and whether the response exposed data beyond the intended audience.
Practitioner takeaway: The safest production threshold for RAG is not “the model is accurate enough,” it is “the workflow is bounded enough that mistakes are observable, attributable, and reversible before they become a data incident.”
Related resources from NHI Mgmt Group
- How should security teams test and harden agentic AI applications before they go into production?
- How should security teams reduce privilege escalation risk in LLM applications before they go into production?
- How should security teams govern semiautonomous AI agents before they go live?
- How should teams secure AI-generated applications before they reach production?