feat(opencode): add LLM provider fallback chain - #26292

Issue for this PR

Closes #7602

Type of change

• Bug fix
• New feature
• Refactor / code improvement
• Documentation

What does this PR do?

Adds a configurable fallback chain so that when a provider returns a transient error (rate limit, overload, 5xx), OpenCode automatically retries on the next model in the chain instead of failing the session.

{
  "model": "anthropic/claude-sonnet-4-20250514",
  "fallbacks": ["openai/gpt-4.1", "deepseek/deepseek-v4"],
  "cooldown_seconds": 300
}

fallbacks can be set at the top level or per-agent. cooldown_seconds defaults to 300 — after a retryable failure, that provider/model is skipped for the cooldown duration so you don't wait on retries to an overloaded provider.

Why built-in instead of a proxy: cheaper providers are unreliable, and routing through LiteLLM degrades tool-call quality. When a provider gets overloaded, falling through immediately is faster than retrying the same one.

Design

Data flow

User sends message
       │
       ▼
  LLM.run(input)
       │
       ▼
  pickStart(primary, fallbacks, cooldown)
       │
       ├─ primary available → call primary
       │
       └─ primary on cooldown → find first available fallback
                                 │
                                 └─ all on cooldown → pick soonest expiry
       │
       ▼
  streamText() → Effect Stream
       │
       ├── success → Stream ends normally. Clear cooldown on the winning provider.
       │
       └── retryable error → CooldownManager.put(failed, duration)
                              │
                              ▼
              chainFallback → try next in chain
                              │
                              ├── success → prepend notice event, continue stream
                              │
                              └── error → chain next fallback (or fail)

Cooldown durations

On success, the winning provider's cooldown is cleared so it's immediately available next request.

Key decisions

1. Cooldown over session state — We track what's failed, not what's succeeded. No "sticky" fallback — once a provider recovers, traffic routes back naturally.

2. Stream-level error detection — Providers can return HTTP 200 with an error in the stream body. We wrap fullStream to throw on { type: "error" } chunks, triggering fallback just like a connection error.

3. Quota limits trigger fallback (6h cooldown) — A quota-limited provider is unavailable, not the session. Fallback keeps the session alive. The 6h cooldown prevents hammering a capped provider.

4. No dedup in the chain — The same model can appear twice. After falling through once, the primary may be worth retrying with fresh context. Cooldown handles this naturally — if primary is still on cooldown, it's skipped; if it's cleared, trying it again is valid.

5. Model attribution updates on fallback — When a fallback succeeds, usedFallback propagates back so events, logs, and billing reflect the actual provider that handled the request.

Config contract

// Top-level (applies to all agents)
{
  "model": "anthropic/claude-sonnet-4-20250514",
  "fallbacks": ["openai/gpt-4.1", "deepseek/deepseek-v4"],
  "cooldown_seconds": 300
}

// Per-agent (overrides top-level)
{
  "agents": {
    "code": {
      "model": "anthropic/claude-sonnet-4-20250514",
      "fallbacks": ["openai/gpt-4.1"]
    }
  }
}

New fields: fallbacks (array of provider/model strings), cooldown_seconds (positive int, default 300).

Not in scope

• Per-provider cooldown configuration (use cooldown_seconds globally for now)
• TTFT timeout thresholds (per-provider config, separate feature)
• Admin API to inspect/clear cooldown state (runtime only, no persistence)
• Persisted cooldown across restarts (cooldowns reset on process restart)

How did you verify your code works?

• Unit tests for CooldownManager (put/get/clear/expiry) and config validation (fallbacks array and cooldown_seconds)
• Integration test: stream error with fallbacks triggers fallback, stream error without fallbacks halts with error
• Running this in production for 1 week across daily work without issues
• bun typecheck passes for all 12 packages in the monorepo

Screenshots / recordings

N/A — no UI changes visible in screenshots (toast is a runtime notification)

Checklist

• I have tested my changes locally
• I have not included unrelated changes in this PR

Comparison with related PRs

Reviewed #24369, #26192, #24013, #18443, and the closed #13189:

• vs feat(processor): add model fallback chain when retries are exhausted #24369: We have cooldown (they have none), stream error detection, and user notification. They have a resolveFallbackChain utility — minor convenience we can add later.
• vs fix(session): add fallback retry handling and harden pre-push bun path #26192: They identified the model attribution gap (events/logs/billing showing original provider instead of fallback) — now fixed in our PR. Their dedup would prevent trying the same model twice in a chain, but this can be intentional (try primary, fall through, then retry primary with fresh context). Our cooldown solves this differently.
• vs fix(opencode): stop retrying non-transient rate limits #24013: We take the opposite approach — quota-limited providers should trigger fallback (the next provider may not be quota-limited), but we put them on a 6h cooldown instead of the default 5min. This way the session doesn't fail just because one provider hit its monthly cap.
• vs fix(retry): retry transient 429 responses even when provider marks non-retryable #18443: They handle 429+isRetryable=false from provider proxies. Our code already handles this via 5xx override, but the explicit 429 path would be a good future improvement.
• vs feat: add model fallback support with TTFT-based timeout #13189 (closed, stale): They had TTFT timeout and session state tracking. Our cooldown naturally routes away from failed providers toward working ones (simpler), and TTFT is better handled per-provider in config.

Key differences in our approach:

• Cooldown is superior to session state: remembers what's failed, not what's succeeded — no "sticky" fallback
• Stream error detection via first-chunk peek catches 200-with-error responses
• retry-after header parsing respects provider-suggested backoff
• Quota limits trigger fallback with 6h cooldown: the provider is unavailable, not the session
• No dedup by design: deliberate — allows retrying primary after falling through