Orchestration
What is AI consensus?
Consensus asks several models the same question and treats their agreement as a signal. What agreement does and does not tell you, and when the cost is justified.
All explainers · Last reviewed:
What agreement actually tells you
Agreement is evidence, not proof. Models trained on overlapping data share biases and can be confidently wrong in the same direction. Three models agreeing on a false fact is a common outcome, not a rare one.
The signal is stronger when the models are genuinely different — different vendors, different training, different sizes. Consensus across three variants of the same family is close to worthless.
Disagreement is the more useful output
The practical value of consensus is usually the negative case. When models diverge, you have located a question that needs a person — and locating those cheaply is worth more than a marginal confidence boost on the questions that were easy anyway.
This reframes when to use it. Consensus is not a quality upgrade applied to everything; it is a triage tool applied where being wrong is costly.
The cost
Running three models costs roughly three times the tokens and takes as long as the slowest. On a routine question that is pure waste. On a contract clause, a migration plan, or a medical summary you intend to act on, it is cheap.
When not to use it
Do not use consensus for questions with a checkable answer. If code either compiles or does not, run it — that is a stronger signal than three models agreeing about it. Consensus is for judgement questions where no cheap external check exists.
Questions people ask
- How many models do I need?
- Three is the usual choice, because two can only agree or disagree while three shows you the shape of a disagreement. More than three rarely changes the decision and multiplies the bill.
- Does consensus prevent hallucination?
- No. It catches hallucinations that are specific to one model, and misses the ones several models share. It is a filter, not a guarantee.
- Is this the same as best-of-N?
- No. Consensus compares answers from different models to see whether they agree. Best-of-N generates several candidates and picks one. Consensus measures agreement; best-of-N selects quality.
Try it rather than read about it
Consensus is one of ClawAI’s 9 orchestration modes, and each run records every model it used and what that run cost.