AI Gateway adds confidence-based decision fallbacks
Mirrored from Vercel — AI for archival readability. Support the source by reading on the original site.
AI Gateway can now escalate a decision request to a fallback model when the primary model's answer trips a confidence condition you set.
Conditions can be combined, so an escalation can depend on more than one signal. The plain model names you already list in models keep catching outright errors, so existing fallbacks are unaffected. Confidence conditions cover Choice and Score questions. Boolean questions escalate on a probability range instead.
Set the fallback by adding one conditional object to providerOptions.gateway.models. Requests without it keep their existing behavior.
{ "gateway": { "models": [Escalating an uncertain answer to a fallback model with a confidence condition
Leave out question and the condition checks every question of the matching type, so { "confidenceBelow": 0.6 } escalates when any Choice or Score answer falls below 0.6.
Because a triggered fallback runs a second decision, it bills both stages.
Decision fallbacks are in beta. Learn more in the decision fallbacks documentation.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.