Cost per completed task

What cost per completed task actually measures

Cost per completed task is a workload efficiency metric, but it is also a reliability metric in disguise. It captures the total spend required to reach a successful outcome, so failed attempts, retries, tool calls, orchestration steps, and human intervention all matter. That makes it more decision-useful than a simple per-token or per-request figure.

The key idea is that the unit of value is a finished task, not raw model activity. Two systems can have the same token cost and very different business cost if one repeatedly fails, consumes more context, or needs heavier coordination to finish the job. For operators, that difference is often where the real savings or overruns appear.

Why this metric is more operationally honest than token price

Token price is easy to compare, but it can mislead because it measures inputs rather than outcomes. A cheaper model that needs extra retries or longer prompts may end up costing more per completed task than a more capable model that finishes cleanly on the first pass. The metric therefore rewards end-to-end effectiveness, not just model thrift.

This matters most in production workflows where completion quality depends on context length, prompt design, retrieval calls, guardrails, tool execution, or human review. If any of those elements are required to get to a successful result, they belong in the cost base. That is why the metric is useful for budgeting, vendor comparison, and workload design.

In practice, it helps teams separate apparent model savings from true operating efficiency. A system that is inexpensive per call but expensive to finish is not actually cheap at scale.

How retries, context, and orchestration drive the real number

Retries are often the largest hidden driver because they multiply all other costs. Each failed attempt can consume additional tokens, extra tool execution, more elapsed time, and sometimes more operator attention. Context use also matters because larger prompts, retrieved evidence, and longer conversation state increase cost even when the model call itself looks modest.

Orchestration overhead includes the glue around the model, such as routing, validation, policy checks, workflow engines, and exception handling. These steps are essential in many environments, but they are still part of the cost of completion. When teams ignore them, they understate the operational burden of production AI.

The metric is therefore most accurate when measured against a full workflow, not a single inference event. It should reflect the actual path from task start to task done, including the work needed to recover from failure or uncertainty.

How to use it in AI operations and business cases

Cost per completed task is most useful when comparing alternative systems, prompt strategies, or workflow designs. It lets teams ask which option gets the same job done with fewer failed attempts, less context waste, and less coordination effort. That is especially important when evaluating whether to add retrieval, stricter validation, or higher-capability models.

It also improves governance conversations because it ties spend to delivered value. A team can justify a higher per-call model if it completes tasks more reliably and reduces total workflow cost. Conversely, a cheaper model may be rejected if its failure rate inflates the true cost of completion.

For budgeting, this metric is strongest when paired with task classes, since simple tasks and complex tasks behave very differently. That allows cost to be tracked against actual workload mix rather than averaged into a number that hides variance.

What practitioners should watch when they measure it

Measuring cost per completed task requires a consistent definition of success. If one team counts a partial result as success and another counts only fully verified completion, the metric stops being comparable. The same is true if retries, human intervention, or downstream review are treated inconsistently across workflows.

The most useful implementations treat completion as an end-to-end operational outcome, not a model output event. That means the metric should include the surrounding system costs that make the task actually usable, otherwise it can create false confidence in “cheap” automation.

When it is measured this way, the metric becomes a practical way to manage efficiency, reliability, and scale together.

Related resources from NHI Mgmt Group

• Should organisations optimise for benchmark scores or cost per completed task?
• What is the difference between cost per attempt and cost per accepted task in coding-agent evaluation?
• Why does cost per successful task matter more than token price for agents?
• When should teams prioritise safer model behaviour over lower per-task cost?