Routing
What is model fallback?
Model fallback is what happens when the first model fails — down, rate-limited, or refusing. How fallback chains work and why silent fallback is dangerous.
All explainers · Last reviewed:
Why it is not optional
A single-provider workflow inherits that provider’s availability exactly. Rate limits in particular are not rare events — they are the normal consequence of a busy hour — and a workflow with no fallback simply stops.
Fallback turns a hard failure into a degraded answer. Whether that is an improvement depends entirely on whether you are told it happened.
Choosing the order
The intuitive order is "next best model", but that is often wrong. If the first choice failed because the request was too long, a smaller model will fail too. If it refused on content grounds, a similar model will refuse similarly.
A more useful order changes something structural: a different provider entirely, or a local model with different rules, rather than a sibling that will fail the same way.
The dangerous kind
Silent fallback is a system that quietly answers with a different model and tells you nothing. You get a worse answer, attributed in your mind to the model you chose, and you draw a wrong conclusion about that model.
When the fallback crosses a privacy boundary it is worse than a wrong conclusion. Falling back from a local model to a cloud provider sends data somewhere the user specifically chose to avoid. A fallback chain that can leave local execution should be a chain the user opted into explicitly.
How ClawAI does it
Routing modes define their own chains, and the local-only mode keeps its chain on local providers rather than reaching for a cloud model when the local one is busy. Every message records the model that actually answered, so a fallback is visible after the fact rather than inferred from a change in tone.
Questions people ask
- Is fallback the same as a retry?
- A retry sends the same request to the same model, which helps with a transient error. Fallback changes the model, which helps when the first one cannot serve the request at all. Most robust systems do both, in that order.
- Should fallback ever cross from local to cloud?
- Only if the user asked for that. Local execution is usually chosen for a reason that a fallback cannot honour, so the safe default is to fail and say so rather than to succeed elsewhere.
- How many models should a chain have?
- Two or three is usually enough. Long chains mostly add latency, because every failed attempt is paid for in time before the next one starts.
Try it rather than read about it
ClawAI’s routing modes carry their own fallback chains, and local-only keeps its chain local rather than silently reaching a cloud provider.