Notifications
Clear all
Topic starter
20/08/2026 5:35 pm
TL;DR: RAG applications can be built quickly, but they only become production-ready when teams add observability for retrievers, prompt chains, tool access, privacy leakage, and hallucination monitoring across the full LLM workflow, according to Fiddler. The governance issue is no longer model choice alone, but whether AI systems can be monitored, explained, and controlled as they move into business-critical use.
NHIMG editorial — based on content published by Fiddler: Building RAG-based AI Applications with DataStax and Fiddler
Questions worth separating out
Q: How should security teams govern access in RAG systems?
A: Security teams should govern RAG access at the retrieval layer, not only at authentication.
Q: Why do RAG applications create extra security risk for enterprise AI?
A: RAG applications create extra risk because they combine model output with live access to documents, connectors, and sometimes tools.
Q: What do organisations get wrong about AI observability?
A: They often confuse technical telemetry with governance evidence.
Practitioner guidance
- Inventory every AI connector and credential path Map the service accounts, API keys, and database connectors used by each RAG workflow, then classify which datasets and tools each one can reach.
- Log retrieval and tool usage as governed events Capture which documents were retrieved, which prompts were used, which tool calls were made, and which identity performed each action.
- Limit the blast radius of data access Scope each RAG workflow to the minimum datasets, namespaces, and APIs needed for its use case, and separate internal from external applications where access sensitivity differs.
What's in the full article
Fiddler's full blog post covers the implementation detail this post intentionally leaves for the source:
- How the RAG stack is assembled across vector database, retriever, orchestration, and observability layers
- Which metrics the Fiddler platform uses for hallucinations, toxicity, faithfulness, and data drift
- How the documentation chatbot was instrumented with prompt history and embeddings
- What the dashboard views show during different load scenarios and cohort comparisons
👉 Read Fiddler's blog on building RAG-based AI applications with DataStax →
RAG observability and control gaps in production AI apps?
Explore further
20/08/2026 6:21 pm
RAG observability is becoming an identity governance problem. Once AI systems retrieve private data and use tools, the control question shifts from model quality to who or what is allowed to access which information at runtime. That means service accounts, API keys, and workload identity now sit directly inside the AI governance conversation. Practitioners should treat RAG telemetry, access scope, and auditability as part of the same control plane.
A question worth separating out:
Q: How can organisations reduce the blast radius of compromised AI or SaaS integrations?
A: Organisations should reduce blast radius by limiting scopes, shortening token lifetimes, segmenting high-risk systems, and requiring reapproval for sensitive actions. They should also inventory every integration that can reach identity platforms, cloud control planes, or production data. The goal is to ensure a stolen grant cannot move far or persist long.
👉 Read our full editorial: RAG observability is now part of AI governance, not optional tooling
