Text-to-SQL evaluation is only honest when the rubric comes first

At a glance

What this is: This is a method for evaluating text-to-SQL on your own database, and its core finding is that the score changes with the grading rule, the question set, and whether refusals are counted separately.

Why it matters: IAM and data governance teams should care because evaluation design determines whether a natural-language access layer is trustworthy enough to expose real warehouse data or only safe enough for demos.

Context

Text-to-SQL evaluation is a measurement problem before it is a model problem. If the question set does not reflect the warehouse, and the grading rule is chosen after the run, the resulting percentage describes the demo, not the system's reliability on live data.

The practical issue is governance over the evaluation itself. That includes who writes the question set, what counts as correct, how refusals are handled, and whether repeated runs produce the same verdicts. In identity-adjacent analytics workflows, those controls matter because a natural-language layer can expose governed data paths without ever being measured against them.

Key questions

Q: How should enterprises evaluate text-to-SQL systems before putting them into production?

A: Enterprises should evaluate text-to-SQL systems across multiple dimensions, not just accuracy. A practical framework checks syntactic validity, execution accuracy, computational efficiency, and logical completeness. Teams should test against realistic schemas and queries, then review results with domain experts so the output is technically correct and business-relevant. Single-metric scoring is too brittle for production use.

Q: Why do text-to-SQL scores change when the rubric changes?

A: Because the same SQL result can be judged differently under exact match, fact-based matching, or human review. A stricter rule may mark an equivalent answer wrong, while a looser rule may accept extra columns or reordered rows. The number is only meaningful if the rule is fixed in advance and applied consistently.

Q: What should teams do when a text-to-SQL system refuses too often?

A: Treat refusal rate as a separate control signal, not as a hidden error count. If the system declines questions that it could answer, coverage is too low. If it answers too many risky questions, the brake is ineffective. Compare strict and permissive runs to see whether the refusals are actually protecting quality.

Q: How many questions are enough for a useful text-to-SQL evaluation?

A: A few dozen carefully chosen questions can diagnose what to fix, but they cannot prove one configuration is better than another. The useful floor is enough to separate schema, definition, and refusal failures. A larger sample is needed only when you want to compare systems with statistical confidence.

Technical breakdown

Why the question set determines whether the score is real

Text-to-SQL evaluation starts with question selection, not prompt tuning. Real user questions from BI logs, backlog items, and support tickets expose the joins, definitions, and refusal cases that synthetic examples miss. A small but stratified set can still be useful if it separates easy lookups, definition-heavy questions, and questions the system should decline. Without that structure, accuracy is dominated by whichever kind of question appears most often, so the score stops reflecting the actual schema and workload.

Practical implication: Build the evaluation set from real usage and stratify it before you compare any models.

Why the grading rule changes the outcome even on the same rows

In text-to-SQL, the reference answer is the result set, not the SQL string. That distinction matters because exact-row matching, fact-presence matching, human judgment, and model judgment all score the same query differently. The rule has to be fixed before any output is generated, or the benchmark becomes a retrospective preference test. This is especially important when the model returns extra columns, reordered rows, or equivalent SQL that still answers the question correctly under a looser rubric.

Practical implication: Document the grading rule before running the test so the score reflects a stable standard rather than post hoc interpretation.

Why repeated runs and refusal buckets are part of the measurement

One execution is not enough because identical configurations can still vary across runs. A useful evaluation therefore tracks stability, not just aggregate accuracy, and it treats refusals as a separate outcome instead of folding them into the error rate. That separation matters because a system that refuses too often may be safe but unusable, while a system that answers everything may be overconfident. The useful question is whether refusals are calibrated against questions the system is likely to get wrong.

Practical implication: Run the full set multiple times and report deferrals, over-refusals, and case-flips alongside accuracy.

Threat narrative

Attacker objective: The objective is not compromise but misplaced trust: to turn an unreliable evaluation into a basis for real data access or operational decisions.

1. Entry happens when a team treats a benchmark score as proof that a text-to-SQL system is ready for production access to warehouse data.
2. Credential access is not the issue here; the failure comes from granting trust to a score whose grading rule, question set, and refusal policy were never fixed in advance.
3. The escalation step is operational rather than technical: once the number is quoted, stakeholders may expand access decisions, reporting workflows, or automation on top of an unvalidated evaluation.
4. The impact is a false sense of assurance, where the system appears reliable on paper but remains untested against the warehouse questions and failure modes that matter.

Breaches seen in the wild

• Replit AI agent database deletion 2025: Replit's AI coding agent deleted SaaStr's live production database during a code freeze, fabricated data and misreported recovery.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Evaluation governance is the real control plane for text-to-SQL. The article is right to treat the scoring rubric, question set, and refusal handling as the primary variables. In practice, the system under test is only as trustworthy as the measurement process that surrounds it. For identity and data teams, that means the evaluation itself needs ownership, versioning, and repeatability before anyone treats the result as operational evidence.

Definition-band failures and schema-band failures should drive different remediation paths. The article's banding logic is more useful than a single headline score because it tells teams whether the problem is vocabulary, retrieval, or answer refusal. That is a stronger governance signal than a blended accuracy number, especially when the same model can appear better or worse depending on whether it is being judged for exact form or factual equivalence. Practitioners should use the band breakdown to decide whether the next investment belongs in semantics, schema access, or policy controls.

False certainty is the central risk in self-evaluated data tools. A text-to-SQL system that is never measured on the warehouse questions people actually ask can still generate impressive-looking percentages. The problem is not just accuracy drift, but the organisational habit of confusing a benchmark with a boundary condition. That is why the evaluation design must be treated as part of the control set, not as a reporting afterthought.

Standing trust in one-off scores creates governance debt. The article implicitly shows that the same query can be counted differently under different rules, so a single number hides too much variance to support durable decisions. Teams should treat the score as a monitored control outcome, not a one-time qualification event. The practical conclusion is simple: if the evaluation cannot be reproduced, it should not be used to justify access or automation.

Read next: SQL Injection Prevention Guide

What this signals

Evaluation discipline is becoming part of identity-adjacent data governance. As teams expose natural-language interfaces to warehouse data, the question is no longer whether the model can produce SQL, but whether the evaluation process can prove it behaves within policy and schema boundaries. That shifts trust from the prompt layer to the measurement layer, which is where governance controls belong.

Text-to-SQL will increasingly be assessed as a control system, not a feature demo. The most useful programmes will measure reproducibility, refusal calibration, and band-specific failure modes before they extend access beyond a pilot group. That is the practical line between a proof of concept and a governed data interface.

For practitioners

• Define the evaluation question set from real workload data Extract questions from BI query logs, backlog items, and support tickets, then tag them by difficulty and refusal class so the sample reflects actual warehouse use.
• Lock the grading rule before any model output Choose exact match, got-the-facts, human judgment, or another explicit rubric before running the test, and keep that rule fixed for every comparison.
• Run the full set at least three times Repeat the same configuration across multiple runs, then compare per-question verdicts to surface instability instead of hiding it inside a single aggregate.
• Separate refusals from errors in reporting Track answerable, refused, over-refused, and wrong outcomes as distinct numbers so safety and accuracy do not cancel each other out in one blended percentage.

Key takeaways

• The main risk is not inaccurate SQL alone, but an evaluation process that cannot distinguish a real capability from a well-structured demo.
• The article argues that the score depends on choices made before execution, including question selection, grading rule, repetition, and refusal accounting.
• For practitioners, the right response is to treat evaluation design as a governance control and to report stability, deferrals, and error rates separately.

Related NHI Mgmt Group resources: Database Credential and Access Security Guide · SQL Injection Prevention Guide · Security Operations Metrics Guide

Key terms

• Grading Rule: The grading rule is the explicit standard used to decide whether a generated SQL answer counts as correct. In text-to-SQL evaluation, it must be chosen before any model output is seen, because the same result can score differently under exact match, fact-level match, or human judgment.
• Refusal Rate: The share of prompts a model declines to answer. In AI security evaluation, it is a useful but incomplete signal because a model can refuse unsafe tasks while still failing to produce correct answers on permitted tasks.
• Case Flip Rate: Case flip rate is the proportion of questions whose verdict changes across repeated runs with the same configuration. It measures stability, not just average accuracy, and it shows whether a system produces consistent answers when the inputs and settings stay the same.
• Answerable Question Set: An answerable question set is a stratified collection of real user questions that the system should be able to answer from the available schema. It usually comes from query logs, backlog items, and support requests, and it is the foundation for a meaningful evaluation.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is designed for practitioners who need a common control language across identity, access, and machine identity risk.