LLM Cost per Successful Task: Include Retries and Review

LLM cost per successful task is the cost of delivering a defined workload divided by the number of distinct tasks that meet your acceptance criteria. Include paid work on rejected outputs and retries, along with tools, infrastructure and any required human review. A low token rate alone does not tell you whether a feature is economical to deliver.

Imagine an invoice-extraction feature. One invoice succeeds immediately, another needs a retry, and a third never produces an acceptable result. The third invoice still belongs in the workload's costs. Counting only the final successful API calls hides the effort needed to get there.

This guide gives you a calculation method, a worked fictional batch and downloadable worksheets. The examples are accounting fixtures, not vendor benchmarks or measured customer results.

Start with a logical task and a clear acceptance rule

A task is the unit the user needs completed: extract one invoice, classify one support ticket or produce one usable product description. An attempt is one execution trying to complete that task. Several model calls and tool calls can belong to one attempt; several attempts can belong to one task.

Give each logical task a stable task_id. Give every retry a new attempt_id and preserve its parent task. These IDs let you add costs without treating a retry as another successful customer outcome.

For invoice extraction, an acceptance rule might require valid fields, totals matching the source document and human approval for specified exceptions. Store the rule's version. Schema validity alone cannot establish factual accuracy, and not every task requires manual review.

An output repaired by a person can count as an accepted assisted task if that is the service you promise. Record the repair time and report assisted versus autonomous acceptance separately. If your metric promises autonomous delivery, the same intervention should fail that rule. Decide this before measuring, rather than changing the denominator after seeing results.

Count accepted tasks once

If two attempts for the same invoice both pass, count the invoice once. Record the selected final outcome and account for both attempts' costs. A late retry result must not inflate the denominator.

Use a consistent cohort, such as tasks first created during one week, with a declared period for late completion and late billing reconciliation. Mixing this week's costs with successes from a different task cohort can produce a misleading ratio. For a calendar-month finance view, define the allocation method explicitly and keep it separate from the task-cohort view.

Keep zero acceptance visible

When no tasks are accepted, cost per successful task is undefined, because its denominator is zero. Report the workload's spend, task count and zero acceptance alongside that result. A dashboard showing zero cost would imply an efficiency the system has not achieved.

Choose your cost boundary before adding numbers

There are two useful views:

• Incremental delivery cost: additional expenses caused by this workload, including required variable review effort.
• Fully allocated delivery cost: the incremental costs plus a documented share of fixed delivery expenses, such as reserved capacity or platform subscriptions.

Label the view you use. Neither is company profit or customer acquisition cost. If you are considering a serving change, compare both: a dedicated server can introduce committed costs that a marginal-only calculation overlooks.

Normalize provider usage rather than guessing from text

Start with reported usage and invoice records. Counting visible response words can miss billed work. For example, OpenAI's token-counting documentation explains that reported output totals can include non-visible generated tokens. Do not add an already included reasoning-token subtotal to the output total again.

Cache accounting is provider- and model-dependent. A provider may report cached input as a subset of total input; only under that documented definition should you subtract cached tokens to derive ordinary input. Other billing categories can include cache writes or modality-specific usage. OpenAI's current caching guide illustrates why a universal two-rate caching formula is insufficient.

Normalize mutually exclusive billable categories, preserve their original usage fields, and store the rate's currency, unit, applicable tier and effective date. A calculation using rates per million tokens needs token_count / 1,000,000 × rate; do not accidentally apply a per-thousand rate.

Billed worker time can exceed generation time

Infrastructure charges follow the service's billing lifecycle. In Runpod's Serverless pricing documentation, billing covers the period from worker start to full stop, rounded to seconds. That includes handler initialization, execution and a running worker's idle-timeout wait. Image pulls and cached-model downloads normally happen while scaled down and are not billed. A worker shown as scaled-down Idle is different from a running worker waiting for another job.

This documented pattern is a reason to collect billed duration, not an assumption that every provider charges the same way. Allocate shared worker intervals once: summing overlapping wall-clock durations for concurrent requests can overstate the infrastructure bill.

Runpod is one documented example, not a default. If you price a serverless GPU path, start with the RunPod listing on Vantaige and confirm RunPod GPU pricing before fixing an hourly assumption.

Record retries, rejection and uncertain billing

A timeout tells you that your application did not receive a timely result. It does not tell you whether the provider performed billable work. Preserve the provider request ID and reconcile the charge. Use pending_reconciliation when usage is missing; avoid silently replacing unknown cost with zero.

A useful attempt ledger records:

• Task, attempt and provider request IDs, with environment and model/configuration versions.
• Start and finish timestamps, terminal status and failure category.
• Reported billable usage, rate reference and tool or infrastructure allocations.
• Review minutes, reviewer rate basis and any assisted repair.
• Acceptance decision, rule version and who or what made the decision.
• Billing state: estimated, reconciled, disputed or still unknown.

Keep prompts and customer documents out of a cost ledger unless they are actually needed. IDs, usage counters and bounded failure labels are often sufficient for this accounting job.

Two documented implementation patterns

Attach quality decisions to execution records. Langfuse's evaluation concepts describe scores attached to traces, observations, sessions or dataset runs. You can use this documented capability to associate a boolean acceptance decision and its rule version with the task's execution record. The mapping from your task_id to trace structure is an application design choice; it does not happen automatically.

Compare versions and investigate the affected workload. Langfuse's metrics documentation describes cost, latency and quality dimensions including feature, model and prompt version. Group your own task records by these dimensions to investigate a change, then reconcile the resulting cost estimates against billing. Telemetry and an automated quality score do not independently establish invoice accuracy or customer value.

These documented methods illustrate how to structure tracing; configure your chosen observability tool to expose SDK retries and confirm that usage totals capture all attempts.

Work through one fictional batch

The supplied batch contains six logical tasks, ten attempts and four accepted tasks. Nine attempts technically complete. One times out; its illustrative cost has already been reconciled within this fictional fixture. The two unaccepted tasks remain in the numerator.

Every rate below is an arbitrary teaching input. None is a current vendor quote:

This fixture has no separate cache-write category, no allocated fixed costs, and an explicit assumed zero allocation for storage/transfer. Those are example boundaries, not claims about any provider's free tier. Add the categories your actual workload incurs.

The calculation is:

human_cost = (review_minutes / 60) * hourly_rate
workload_cost = model + tools + infrastructure + review + allocations
accepted_tasks = count(distinct task_id with an accepted final outcome)
cost_per_success = workload_cost / accepted_tasks

For this batch, USD 4.175150 / 4 = USD 1.0438 per accepted task, rounded to four decimals. Task acceptance is 4/6, or 66.7%; it is not 4/10. Cost per execution attempt is about USD 0.4175, a different metric.

Human review is the largest component in this invented batch because of the inputs chosen. Your workload can have a completely different cost profile. There is no measured comparison here between models, providers or businesses.

Download and adapt the worksheets

• Successful-task cost worksheet: line items, units, formulas and the worked batch.
• Synthetic attempt log: ten rows showing retries and accepted/rejected outcomes.
• Scenario worksheet: low/base/high assumptions and separate single-variable changes.

The CSV files contain labelled data and formula descriptions, not an executable spreadsheet model. Import them into your preferred spreadsheet and create formulas for your own columns. Preserve enough precision for low token costs; round only presentation totals.

First replace the fictional rates. Then populate a real task cohort, deduplicate accepted IDs, add missing cost categories and mark unreconciled records. Keep the original synthetic example in a separate tab so it cannot be mistaken for production evidence.

Use scenarios to expose assumptions

The scenario sheet starts with 1,000 logical tasks in each case. It varies executed attempts, acceptance and review effort. The labels are planning assumptions, not claims about typical application performance.

Changing several assumptions together does not show which change caused the difference. Use a single-variable calculation for that question. For example, holding the base batch's other costs constant while increasing review sampling from 20% to 40% adds USD 270 and takes the synthetic unit cost to about USD 0.7087.

That result describes this arithmetic only. In a real application, changing review coverage might also change discovered failures, acceptance and repair effort. Measure those relationships rather than assuming independence.

Turn the result into an engineering decision

Investigate the largest measured, actionable component while preserving quality requirements.

• If rejected outputs drive rework, inspect failure categories and improve the relevant retrieval, schema, prompt or workflow. Re-evaluate acceptance after the change.
• If tool charges dominate, test whether repeated lookups can be reused within valid freshness and permission boundaries.
• If review effort dominates, distinguish necessary review from preventable repair. Removing review is not a saving if unusable outputs then reach customers.
• If committed infrastructure or idle usage dominates, compare capacity settings and serving choices against actual demand and latency requirements.

A cheaper model is a candidate to evaluate, not a cost reduction already achieved. Likewise, dedicated hardware has no universal volume threshold and does not automatically guarantee queue priority or lower cost. Include operations, capacity, model support and the relevant service terms in the comparison.

Use Vantaige's model inference and serving inventory when the next question is where to run the workload. Use the LLM gateways inventory when you need to compare routing or usage-control options. Verify a shortlisted product's actual controls before relying on them.

Reconcile before using the number to set prices

Match the provider's billing period, time zone, account, currency and usage categories; do not assume all invoices use the same boundaries. Document taxes, credit drawdowns, discounts and fixed allocations. Keep development experiments and production delivery distinguishable.

Reconcile discrepancies at the finest level the provider exposes. If invoices only provide aggregate charges, document the remaining uncertainty instead of claiming exact task-level attribution.

Recalculate after a material rate, model, prompt, retry-policy, acceptance-rule or workload change. Track both cost and accepted-task count: a lower ratio with a less demanding acceptance rule is not evidence of a better product.

Start with one real feature and one bounded cohort. Download the worksheet, replace its example inputs, and report the acceptance rule, missing charges and cost boundary alongside the result.

Frequently asked questions

Do failed requests always cost money?

Not necessarily. A rejected output can incur charges even when it has no value, while a request rejected before processing might have none. A timeout can leave billing uncertain. Record the actual or estimated charge and reconcile it under the provider's terms.

Is cost per successful task the same as gross margin?

No. Gross margin requires revenue and the relevant accounting cost boundary. This guide calculates delivery cost for a declared accepted unit of work. It does not include every expense of running a company or acquiring a customer.

Should I include launch-time evaluation work?

Track it separately from ongoing production delivery unless your declared cost view allocates it. Recurring production review belongs in the delivery model when it is required to provide the service. Make the distinction visible rather than silently dropping labor.

What should I publish when no tasks succeed?

Report an undefined unit cost, the total spend, zero accepted tasks and any unresolved billing. Investigate the failure before treating the batch as a pricing baseline.