Why can RLVR improve scores without improving reasoning?

Why RLVR Can Lift Pass@1 Without Improving Reasoning

RLVR can improve measured performance by making the model more likely to emit an already-known correct path, without changing whether it can actually discover new solution paths. If the base model already contains the needed reasoning, training can raise first-try success by shifting probability mass toward the right answer, which looks like better reasoning only if you read pass@1 too literally.

What Changes, and What Does Not

The key distinction is between search efficiency and capability. A model can become better at selecting one of several latent solutions, or at avoiding obviously wrong branches, while its underlying reasoning depth remains unchanged. In practice, that means the model may look better on a single-sample benchmark even if its internal ability to generalise, plan, or derive novel solutions is the same.

This is why evaluation setup matters. pass@1 is sensitive to sampling behaviour, decoding temperature, and prompt format, so a gain there can come from better calibration, better retrieval of a stored pattern, or better alignment with the benchmark style. To claim improved reasoning, you need evidence that the model is succeeding more often across harder variants, not just on the first sampled attempt.

How to Interpret RLVR Results

RLVR should be read as a training signal that can reshape which answers the model prefers, not automatically as proof that the model learned a stronger reasoning algorithm. If the reward primarily favours answers that match known solution templates, the policy can become more decisive without becoming more exploratory. That distinction is important when comparing pass@1 against pass@k, consistency across seeds, or performance on out-of-distribution tasks.

It also helps explain why a model may improve on benchmarks with narrow answer spaces faster than on tasks that require genuinely new inference. When the correct reasoning chain is already present in the base model, RLVR can make that chain more accessible under the benchmark’s scoring regime. When the chain is absent, however, the same method usually has far less room to help.

Risk and Threat Considerations

Benchmark gains can be misleading when teams infer reasoning progress from a metric that is partly a sampling artefact. The risk is overestimating model robustness, then deploying a system that still fails when prompts are slightly changed, the task format shifts, or the model is asked to explain or generalise beyond the benchmark pattern.

Failure mechanism: RLVR can overweight high-probability solution templates and suppress exploration, so first-attempt answers improve even when the model has not learned a more reliable reasoning process.

Impact: Teams may ship models that look better in evaluation but remain brittle in production, especially on edge cases, longer-horizon tasks, or questions that require compositional reasoning rather than template recall.

Practitioner Guidance

What to verify: Check whether the gain survives changes in decoding, prompt wording, and seed, and compare pass@1 against pass@k before treating the result as a reasoning improvement.

What to measure: Track consistency across multiple samples and harder variants, because a real reasoning gain should usually improve more than one sampling regime.

Common mistake: Treating a better single-shot score as evidence of deeper cognition when it may just reflect better selection among already-available answers.

Practitioner takeaway: If RLVR raises pass@1 without broadening success across variants and samples, the safer interpretation is better answer selection, not stronger reasoning.

Related resources from NHI Mgmt Group

• Why should AI governance teams care about RLVR if the model scores improve?
• How should teams design multi-agent fine-tuning to improve reasoning without collapsing into one pattern?
• Why does reinforcement learning improve reasoning in AI models without replacing the need for supervised fine-tuning?
• Why does a CMDB improve IT operations without automatically improving identity control?