Notifications
Clear all
Topic starter
19/08/2026 3:30 pm
TL;DR: Offline LLM evaluation is not enough on its own because quality, safety, and operational metrics must work together, with reference-based metrics suited to structured tasks, reference-free methods better for open-ended use cases, and RAG-specific scoring needed for grounded answers, according to Fiddler. The real governance gap is continuous monitoring, not one-time model testing, because production behavior changes after launch.
NHIMG editorial — based on content published by Fiddler: Your LLM Application Passed Eval But It's Still Failing in Production
By the numbers:
- Enterprises can incur approximately $260K annually at 500K traces per day when relying on external LLM evaluation calls.
Questions worth separating out
Q: How should teams evaluate LLM features before using them in production workflows?
A: Teams should use a layered evaluation model.
Q: Why do RAG systems need separate faithfulness and relevance checks?
A: Because a response can be grounded in source documents and still fail to answer the user’s question, or answer the question while introducing unsupported claims.
Q: What do security teams get wrong about AI agent and NHI monitoring?
A: They often treat monitoring as a logging problem instead of an identity governance problem.
Practitioner guidance
- Define separate quality, safety, and runtime KPIs Track reference-based quality, reference-free safety, and operational latency or cost as distinct controls, then review them together before production release.
- Instrument RAG pipelines at retrieval and generation layers Measure faithfulness, answer relevance, and contextual precision independently so you can see whether failures start in retrieval ranking, prompt construction, or answer synthesis.
- Trace tool use as part of identity governance For agentic workflows, log every tool call, parameter set, and downstream action against the initiating identity or service account so review teams can reconstruct delegation paths.
What's in the full article
Fiddler's full blog post covers the operational detail this post intentionally leaves for the source:
- metric-by-metric explanations for BLEU, ROUGE, BERTScore, SelfCheckGPT, and LLM-as-a-Judge
- practical guidance on RAG faithfulness, answer relevance, and contextual precision scoring
- how the in-environment evaluation models reduce per-query cost and latency in production
- examples of guardrail policy enforcement across Azure OpenAI, Amazon Bedrock, LangGraph, and Google Gemini
👉 Read Fiddler’s full guide to LLM performance metrics and production monitoring →
LLM evals passed, but why do production systems still fail?
Explore further
19/08/2026 4:13 pm
LLM evaluation has become a governance control, not a model-tuning exercise. The article shows that quality metrics, safety metrics, and operational metrics all have to work together if production behaviour is going to stay within policy. That is the same pattern identity teams see in access governance, where authentication, authorisation, and monitoring each cover a different failure class. Practitioners should treat evaluation as part of control design, not as a post-development checklist.
A question worth separating out:
Q: How often should organisations refresh LLM evaluation datasets?
A: At minimum, refresh them quarterly and sooner if user behaviour, tools, retrieval sources, or policy rules change. Production systems evolve quickly, and stale test data can hide new failure modes. Regular refreshes keep evaluation aligned with the actual operating environment instead of an outdated benchmark set.
👉 Read our full editorial: LLM evaluation gaps show why production monitoring still fails
