Image API fallback routing should classify errors — transient infrastructure failures retry; policy and validation failures never change providers.
“A luminous midnight garden growing around a black portal, mint and violet botanical forms, surreal editorial illustration, deep negative space”
FLUX Dev · premium · 960 × 1472Classify the failure before choosing the next action
Timeouts, rate limits, and service errors can be temporary. A malformed payload, unsupported dimension, insufficient balance, or policy rejection usually requires operator or client action. Treating all failures as retryable wastes money and can create inconsistent policy behavior.
Provider adapters translate vendor responses into a small internal error model with code, status, and retryable flag. The router trusts that classification, not string matching in the HTTP route.
| Class | Examples | Router behavior |
|---|---|---|
| Transient | Timeout, 429, selected 5xx | Retry once, then fallback |
| Request | Invalid size, unsupported model | Stop and return actionable error |
| Policy | Prompt or account restriction | Stop; never try another provider |
| Configuration | Missing key or disabled route | Skip in auto mode; error when pinned |
Keep fast, premium, standard, and verified 18+ routes separate
A premium request should not quietly fall back to a cheap fast model, and a standard request should never move into an adult-permissive configuration. Define candidate lists per lane. A provider may appear in more than one list only when the exact model and safety configuration have been reviewed for that lane.
The adult list defaults to Runware and local only, and adult mode is disabled unless the key is approved and required output moderation is configured. Leonardo and Together remain standard fallbacks but are not assumed eligible for the adult lane.
- Independent fast and premium orders
- Explicit provider capability flags
- Jurisdiction and account checks before routing
- No cross-lane fallback after an error
Stop sending every request into a known incident
A circuit breaker counts retryable failures for each provider. After a threshold, it opens for a cooldown period and the router advances immediately. A later probe closes the circuit after success. This protects latency and reduces pressure on a degraded service.
When multiple workers or regions are active, keep circuit state in shared infrastructure rather than process memory.
for provider in eligible_providers(request.lane):
if circuit.is_open(provider):
continue
try:
return await provider.generate(request)
except ProviderError as error:
if not error.retryable:
raise public_error(error)
circuit.record_failure(provider)
raise ProvidersExhausted(attempts)Bind retries to a durable operation identity
The router can avoid duplicate calls inside one request, but only the API boundary can protect against a client that times out after a successful generation. An Idempotency-Key is stored with a hash of the normalized request and the complete response. The same key and payload returns the original result; the same key with a different payload returns 409.
Use shared database or Redis persistence so idempotency survives worker changes and deploys. Set a retention window long enough for expected retries and purge expired records on a schedule.
- Normalize dimensions before hashing the request
- Scope the key to the authenticated owner
- Never log the raw key
- Return the original request ID on replay
Expose enough metadata to prove the router helped
Track attempts, time spent per provider, circuit state, success provider, cost, and fallback rate. A fallback that succeeds after 70 seconds may protect availability but still violate the user experience. A provider that is cheap per call may be expensive per accepted image.
Set alerts for provider-specific error rate, fallback share, moderation capacity, and cost drift. Publish customer-facing status separately from internal provider detail so incidents remain transparent without exposing credentials or contractual information.
Questions about image API fallback routing
Should a 400 response ever fall back?
Generally no. It usually indicates a request or policy issue. Provider adapters may classify rare vendor-specific cases differently, but that must be explicit.
How many retries should I use?
Start with zero or one retry inside a strict time budget. More retries can worsen incidents and increase cost.
What happens when all providers fail?
Return a stable providers_exhausted error with a request ID and sanitized attempt markers. The client can retry later using the same idempotency key.
Can adult mode use the standard fallback list?
No. It has a separate allowlist and stronger authorization and moderation requirements.