Usually fix prompts and workflow structure first, because oversized prompts, retrieval bloat, and retry loops often account for a large share of cost. Switch models when tracing shows the prompt is already tight and the remaining spend comes from a model that is unnecessarily strong for the task.
Fix the workflow first when the spend is driven by structure, not model choice
The cheapest model is not always the cheapest run. Cost often hides in prompt bloat, repeated retrieval, long chat histories, and retry loops, so the first question is whether the workload is wasting tokens before the model even starts reasoning. If the prompt is doing too much work, a smaller model may only mask the real inefficiency.
That is why prompt tightening, context trimming, and workflow simplification usually come before a model swap. When the same task can be expressed with fewer instructions and less retrieved text, you cut cost without changing the task contract or introducing quality drift.
Good teams treat token usage as a workflow signal, not just a vendor bill. If one step repeatedly expands the context window or triggers retries, the issue is usually in orchestration, retrieval, or output formatting, not in raw model capability.
When a cheaper model is the right lever
A model change makes sense when the prompt is already lean and the remaining cost comes from over-capacity. If the task is narrow, repetitive, and well-bounded, moving to a smaller model can preserve acceptable quality while materially reducing spend and latency.
The practical test is whether quality stays stable after the prompt has been simplified and the workflow has been measured under realistic load. If the task still uses an expensive model for classification, extraction, or routine rewriting, that is a strong signal to downshift.
Cheaper models are also a better fit when the output is easy to verify downstream. If the response can be checked by rules, schemas, or human review, you can accept a more economical model choice with less risk than in open-ended reasoning tasks.
Use measurement to separate prompt waste from model waste
The decision should be driven by traces, not intuition. Compare input tokens, output tokens, retries, retrieval payload size, and success rate by task type so you can see where the cost is accumulating and whether quality falls when the model is reduced.
Once you have that breakdown, the pattern usually becomes clear: oversized prompts and noisy retrieval point to a prompt and workflow fix, while stable prompts with consistently high spend point to model over-selection. That sequence avoids premature optimization and keeps the change aligned to the real bottleneck.
For teams operating at scale, this is also where governance matters. A model downgrade that saves money but increases retries or manual correction can erase the expected gain, so cost per successful task is a better decision metric than raw per-call price.
Practitioner Guidance
What to prioritize: Start by shrinking the context you send, then remove repeated retrieval and retry behavior before comparing model prices. The fastest savings usually come from eliminating unnecessary tokens, not from negotiating a lower per-token rate.
Decision rule: If a task still performs well after prompt trimming and workflow cleanup, test a cheaper model. If quality only holds when the prompt is bloated, fix the structure first and treat model choice as secondary.
What to measure: Track cost per successful completion, not just cost per call. That exposes cases where a cheaper model creates hidden rework that wipes out the savings.
Practitioner takeaway: Optimize the shape of the work before optimizing the price of the engine, because a lean workflow makes model choice easier, safer, and more defensible.