A retry is cheap when the model is free and the request is tiny. In production, a retry often repeats the entire prompt, retrieval context, tool trace, and output generation. Stack retries inside an SDK, a gateway, an agent loop, and a queue worker, and one user request can become 243 backend calls in the worst case.

That 243 figure is simple math: three attempts at each of five nested layers. It is not LiteLLM's default behavior, and it is not a claim that every outage produces 243 calls. It is a design warning. If you cannot draw the retry boundary for every layer in your system, you do not have a cost ceiling. You have a hope.

Nested retry layers multiply one LLM request

The useful fix is a small gateway policy. Put LiteLLM in front of the providers, give each workflow a budget tag, keep retries bounded, and make the database requirement explicit. The configuration is less glamorous than adding another agent, but it is the part that keeps a provider outage from becoming an invoice you discover the next morning.

A retry policy you can copy

Start by deciding which failures deserve another attempt. A connection reset, a provider 429, or a temporary 5xx can be worth retrying. A malformed request, an invalid API key, a context window error that will happen again, or a tool call with a duplicate side effect usually is not.

LiteLLM's reliability documentation shows num_retries=2. Treat that as a ceiling for one model attempt, not as permission for every wrapper around the call to retry twice again. The simplest policy is one retry owner. If the gateway owns retries, the worker should stop retrying after the gateway returns a final error. If the worker owns retries, turn gateway retries off for that route. Pick one layer.

A minimal SDK call looks like this:

from litellm import completion

response = completion(
    model="openai/gpt-5.6-luna",
    messages=[{"role": "user", "content": prompt}],
    num_retries=2,
)

For a proxy, use the router and fallbacks deliberately. A fallback is not free reliability. It is another billable model call, and the prompt may be sent again. Set a maximum number of fallback targets, keep the chain short, and log the model that finally answered. If the request changes state outside the model, do not automatically retry it unless the operation has an idempotency key.

The practical rule is blunt: one retry layer, at most two retries for transient provider failures, and no automatic retry for deterministic errors. Add a request deadline as well. LiteLLM's routing docs describe timeout and cooldown controls; use them so a failed provider does not hold a worker indefinitely while the queue continues adding work.

The budget that actually stops the bill

Retries control call count. Budgets control money. You need both.

LiteLLM's tag budget documentation uses an engineering example with a $500 maximum and a 30d reset window. A tag can represent a project, customer, department, or workflow. Attach the tag to the virtual key so clients inherit the limit instead of trusting every caller to send the right metadata.

A practical starting point is a separate tag for each workload that has a different owner or failure profile:

# Illustrative values based on LiteLLM's tag budget fields
name: document-ingest
max_budget: 50.0
budget_duration: 1d

The official API example uses a $500 engineering budget over 30d. For a small background workflow, $50 over 1d is easier to reason about. For a customer-facing product, set the number from your gross margin and traffic rather than copying either example. The setting is a guardrail, not a pricing strategy.

When a tag crosses its limit, LiteLLM documents a budget_exceeded response that includes the current cost and maximum budget. Your application should treat that response as a normal circuit-breaker event: stop the job, record the workflow ID, and alert the owner. Do not catch it and immediately retry with another key. That bypasses the only control that worked.

There is a trap here that deserves its own paragraph. LiteLLM's proxy docs state that budgets require a connected database. On a DB-less deployment with 0 connected databases, global spend cannot be read, so the global budget check is skipped and requests continue. A config file containing max_budget is not proof that spend is capped. Test the failure path with a deliberately tiny budget, confirm that the request receives budget_exceeded, and inspect the proxy logs before connecting a real provider key.

A small deployment recipe

The order matters more than the YAML.

First, run LiteLLM with a supported database and keep the database on the same private network as the proxy. A local SQLite experiment can prove that the client works, but do not treat a DB-less proxy as a cost-control system. The official docs name Postgres-compatible services such as Supabase and Neon as examples.

Second, create a virtual key for one workflow and attach one tag. Do not reuse a master key in application code. A workflow-level key gives you a place to set a budget, model permissions, rate limits, and an owner without changing every caller.

Third, set the retry policy at the gateway. Use num_retries=2 only for transient failures, then cap fallback targets. If the workflow calls five model steps, the retry decision must still be made per step. A single outer retry around the entire agent loop can repeat all five steps and any external tools inside them.

Fourth, send a small test request until the tag crosses its cap. The official tag example shows a $500 limit and a response reporting a current cost of $505.50 with budget_exceeded. In your test, use a tiny amount and a disposable provider key. The acceptance test is not "the proxy started." It is "the proxy refused a request after the budget was exceeded."

Fifth, kill the provider connection and watch what happens. The system should stop after the configured retries and fallback chain. Check that the queue does not redeliver the same job forever. Check that your alert includes the workflow tag, request ID, provider error, retry count, and final cost.

For cost review, calculate spend per completed task rather than spend per request. A retry can make the second number look normal while the first gets worse. Track at least: successful tasks, failed tasks, total model calls, retry calls, fallback calls, input tokens, output tokens, and dollars. If retries exceed 10% of calls for a workflow, pause and inspect the failure rather than raising the retry limit.

This fits beside existing automation. If you are already deciding whether to self-host workflow infrastructure, the self-hosting break-even math gives the infrastructure side of the decision. If the workload includes MCP tools, a cheap home for MCP workflows covers the hosting side. LiteLLM is the missing boundary between those workflows and the provider bill.

What this setup does not solve

A budget tag does not detect a bad prompt before it spends money. A retry cap does not make a tool call safe to repeat. A database does not tell you whether a completed answer was useful. You still need idempotency for side effects, per-request token limits, queue backpressure, and a human review path for jobs that fail repeatedly.

The 243-call example is useful because it changes the question. Do not ask whether retries are enabled. Ask how many distinct components can retry, what each one costs, and where the system stops. Then make that stop visible in a test.

Sources