Routing and fallbacks

Default routing

Each canonical model is backed by one or more provider endpoints and credentials. ElevenRouter continuously learns which credential can reach which model (listing sync every few minutes plus live feedback), tracks health with an exponentially weighted error rate and latency per endpoint, and applies circuit breakers and cooldowns. Requests go to the healthiest eligible endpoint; if it fails before any output was sent, the next one is tried automatically.

Credentials are also checked against what their provider reports about them: expired, exhausted or revoked credentials are skipped before a request is sent, and a credential whose remaining balance cannot cover the request's worst-case cost is skipped too. Among credentials that can all serve a model, the one expiring soonest is used first. The public /models list exposes the outcome as endpoints_available and availability (available, degraded, unavailable). When nothing can serve a model you receive a model_unavailable error; supply models fallbacks to keep serving.

Provider preferences

{
  "model": "anthropic/claude-opus-4.6",
  "messages": [...],
  "provider": {
    "order": ["anthropic"],
    "allow_fallbacks": true,
    "ignore": [],
    "only": [],
    "sort": "latency"
  }
}

Providers are identified by the model vendor's slug (anthropic, openai, google, …) — the same prefix as the model id — or, for your own attached keys, the provider slug shown on the BYOK page. This matters mostly with models fallbacks across vendors and with your own keys.

• order: providers to try first, in this order.
• allow_fallbacks (default true): when false, only providers in order are used.
• only / ignore: restrict or exclude providers.
• sort: price, latency or throughput to override the default health-weighted ordering. Platform endpoints all charge the official model price, so price only changes the order once cheaper endpoints (for example your own provider keys) are attached to a model.
• max_price: { "prompt": 3, "completion": 15, "request": 0 } in USD per million tokens / per request. Endpoints whose official price is above a cap are skipped (your own keys are compared at their BYOK fee price), so combined with models fallbacks you can say "never pay more than this". When the cap rules out every endpoint the request fails with a 404 no_endpoints_within_max_price that quotes the cap and the official price.
• require_parameters (default false): by default, parameters a model or wire protocol cannot honour (for example logprobs on Claude) are dropped and listed under pipeline.dropped_parameters in the route receipt. Set it to true to route only to endpoints that support every parameter you sent; with no such endpoint the request moves to the next models fallback or fails with a 404 no_endpoints_supporting_parameters naming the parameters. Likewise only / ignore / order that leave no endpoint fail with no_endpoints_matching_provider_preferences. These are distinct from model_unavailable (503), which means the platform itself cannot serve the model right now.
• preferred_max_latency / preferred_min_throughput: a number (applies to the p50) or { "p90": 800 }-style object against the endpoint's measured time-to-first-token (ms) and output tokens/s over its last 200 successful requests. Endpoints that miss the bar are demoted, never dropped, so the request still succeeds when nothing meets it.
• quantizations: accepted for compatibility and ignored — the vendors we route to do not expose quantization.

Model variants

Shorthands appended to any model id:

• model:nitro — prioritise throughput (tokens per second), e.g. openai/gpt-5.6-luna:nitro.
• model:floor — prioritise the lowest price.
• model:thinking — turn reasoning on at medium effort unless the request sets reasoning itself. Fails with variant_unsupported on models that cannot reason.
• :online and :extended are reserved and currently return variant_unavailable.

An explicit provider.sort always wins over the suffix. Responses and the activity log report the canonical model id; the suffixed id you sent is kept as the requested model. Variants also work on /messages, /responses, /completions, /embeddings and GET /models/:id.

Latest aliases

~vendor/family-latest always resolves to the newest live model of a family, so you can track a vendor's current generation without changing code: ~anthropic/claude-sonnet-latest, ~openai/gpt-terra-latest. The response's model is the concrete model that answered and is billed at that model's official price. GET /models exposes each model's family and, on the newest member, its latest_alias. Retired models never resolve; the alias moves to the next member automatically.

Deprecation and retirement

Models carry deprecated_at, retired_at and successor in the catalog. While deprecated, responses include x-er-model-deprecated (the retirement date) and x-er-model-successor; a Deprecations alert rule notifies you when a model you use is scheduled for retirement. After retirement, requests are forwarded to the successor — the response says so with x-er-model-forwarded-from and pipeline.forwarded_from — unless your organization turns forwarding off, in which case you receive 404 model_retired naming the replacement.

Response cache

Send X-ER-Cache: true to serve exact repeats of a request from a cache scoped to your API key. Hits are free, skip the vendor entirely and are recorded in Logs with a cache badge. The key covers the model and every parameter that changes the answer; it ignores stream, usage, metadata, session_id, user and routing preferences, so a streaming request can be served from a cached JSON answer (replayed as one chunk).

X-ER-Cache: true            # opt in for this request
X-ER-Cache-TTL: 3600        # seconds, 60 … 604800 (default 3600)
X-ER-Cache-Clear: true      # ignore what is stored and overwrite it

# response
X-ER-Cache: HIT | MISS | BYPASS
X-ER-Cache-Age: 42          # seconds since the answer was stored (HIT)
X-ER-Cache-TTL: 3600
X-ER-Cache-Source-Id: gen-… # the generation that produced the cached answer (HIT)

Only complete, non-streamed answers are stored; insured (empty) answers never are. An organization can switch the cache off under Settings, after which requests get BYPASS. Deleting a key deletes its cache.

Message transforms

"transforms": ["middle-out"] makes an oversized conversation fit the model's context window: messages are removed from the middle (system messages, the opening turn and the two latest turns are always kept; tool calls and their results go together), then the longest remaining message is trimmed in the middle. The receipt's pipeline.transforms reports what was removed and the prompt size before and after.

Structured outputs and response healing

response_format with json_schema is forwarded to models that support structured outputs; on models that only support JSON mode it is downgraded to json_object (reported as a dropped structured_outputs parameter) unless you set require_parameters. Add "plugins": [{ "id": "response-healing" }] to repair almost-JSON (code fences, prose around the object, trailing commas, single quotes, unterminated strings) and validate the result against your schema; the outcome is in pipeline.healing with repaired, valid and the validation errors. Healing applies to non-streamed responses.

Rate limits

Every response carries x-ratelimit-limit and x-ratelimit-remaining for your key; exceeding the limit returns 429 rate_limit_exceeded with retry-after in seconds. Vendor-side throttling is absorbed by failover; when every endpoint for a model is throttled you receive 429 upstream_rate_limited with the vendor's retry-after hint when it sent one. Mid-stream failures arrive as an error event inside the SSE stream (see Streaming).

Prompt caching

Place cache breakpoints with cache_control on text content parts, on a whole message, or on Anthropic-style system blocks. Breakpoints are forwarded to providers that support them (Anthropic-native endpoints receive them as native cache_control blocks; OpenAI-compatible aggregators receive them unchanged). Cached reads and writes are billed at the model's official cache rates and show up as prompt_tokens_details.cached_tokens / cache_write_tokens.

{
  "model": "anthropic/claude-sonnet-4.5",
  "messages": [
    { "role": "system", "content": [
      { "type": "text", "text": "<large reference document>", "cache_control": { "type": "ephemeral", "ttl": "1h" } }
    ] },
    { "role": "user", "content": "Summarise section 3." }
  ]
}

Model fallbacks

Supply models with alternatives. If the primary model has no available endpoint, the next model is used and the response's model field tells you which one answered.

{ "model": "anthropic/claude-sonnet-5", "models": ["openai/gpt-5.6-terra", "deepseek/deepseek-v4-pro"], "messages": [...] }

Retries and idempotency

ElevenRouter retries transient upstream failures (rate limits, 5xx, timeouts, access denials) on other endpoints as long as nothing has been streamed to you yet. Client-side errors (invalid parameters, context length) are returned immediately without retry. Every attempt is recorded and visible in the dashboard's activity view.