Notifications
Clear all
Topic starter
20/08/2026 6:06 pm
TL;DR: Production RAG still fails most often at orchestration, retrieval quality, and answer verification, with Arize’s article showing how ADK routing and Arize AX monitoring target those gaps in enterprise knowledge systems. The practical lesson is that reliable RAG depends on continuous evaluation of groundedness, citation quality, and context use, not prompt tuning alone.
NHIMG editorial — based on content published by Arize: Mastering Production RAG with Google ADK and Arize AX for Enterprise Knowledge Systems
By the numbers:
- Only 5.7% of organisations have full visibility into their service accounts.
- 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools.
- 90% of IT leaders say properly managing NHIs is essential for a successful zero-trust implementation.
Questions worth separating out
Q: How should security teams govern production RAG systems with private data sources?
A: Treat production RAG as a governed access path, not just an application feature.
Q: Why do RAG systems still hallucinate when retrieval is enabled?
A: Retrieval does not eliminate hallucination when the retrieved context is irrelevant, stale, or only loosely connected to the question.
Q: What breaks when RAG observability is missing?
A: Without RAG observability, teams can see that a service is healthy while missing the real failure in retrieval, reranking, or context assembly.
Practitioner guidance
- Separate orchestration from authorization Assign distinct identities and access scopes to retrieval services, embedding pipelines, and generation workloads so a failure in one layer cannot expose every data source.
- Instrument citation and groundedness checks Track whether each answer can be traced to retrieved evidence, and alert when citations no longer support the response or when the model departs from the source context.
- Curate the corpus as a governed asset Review chunking, metadata, document freshness, and source prioritisation together so retrieval quality does not depend on ad hoc content hygiene.
What's in the full article
Arize's full blog covers the operational detail this post intentionally leaves at the governance level:
- Step-by-step tracing setup for ADK instrumention and production observability inside Arize AX
- Concrete evaluator examples for hallucination, groundedness, relevancy, and citation verification
- Implementation details for automatic online evaluation and regression dataset curation
- Embeddings analysis workflows that reveal knowledge-base drift and retrieval gaps
👉 Read Arize's analysis of production RAG orchestration, observability and evaluation →
Production RAG observability and evaluation: what teams are missing?
Explore further
20/08/2026 6:41 pm
Production RAG creates an identity and access problem hidden inside an AI quality problem. The article shows that RAG reliability depends on more than retrieval accuracy, because the system must decide which knowledge sources can be queried, by whom, and under what scope. That is where IAM, service-account governance, and workload identity become part of AI assurance. Practitioners should treat RAG access paths as governed identities, not invisible implementation detail.
A question worth separating out:
Q: How do teams know if RAG quality is actually improving?
A: Teams know quality is improving when live scores for groundedness, faithfulness, and answer relevance trend upward across real traffic, not just benchmark sets. They should also check whether long-tail cohorts improve, because aggregate gains can hide regressions in specific query classes.
👉 Read our full editorial: Production RAG needs observability, evaluation and routing controls
