---
title: "Provider abstraction, model routing, and fallbacks"
chapter: "04"
---

# Provider abstraction, model routing, and fallbacks

Provider portability is valuable at stable concepts—messages, embeddings,
tools, and vector stores—but model behavior is not interchangeable.

## Route by role

Define configuration roles instead of scattering model IDs:

```yaml
ai:
  roles:
    complex-reasoning: openai:gpt-5.6-sol
    interaction: google:gemini-3.6-flash
    high-volume: google:gemini-3.5-flash-lite
    private-draft: ollama:qwen3
    embedding: google:gemini-embedding-2
```

Every role needs measured quality, p50/p95 latency, cost, context needs,
regional availability, tool/JSON support, and lifecycle status.

## Fallback semantics

A fallback is safe only if it preserves:

- data residency and retention rules;
- prompt and output contracts;
- tool schemas and authorization;
- embedding dimensions and vector-space identity;
- multimodal formats;
- acceptable quality and latency.

Do not catch every provider error and silently send confidential data to
another vendor. Circuit breakers protect capacity; they do not define product
correctness.

## Capability registry

Store model/provider capabilities in configuration: reasoning, streaming,
structured output, image/audio input, tools, parallel tools, context limits,
fine-tuning, lifecycle date, and approved data classes. Test the registry
against live provider smoke calls.

## Shadow and canary

Shadow requests can compare a candidate without changing user output, but they
double data exposure and cost. Redact, sample, and obtain policy approval.
Canary a new model by role and tenant; compare the same evaluation slice.

## Feynman check

A provider adapter makes plugs look similar. It does not make the electricity,
price, or appliance behavior identical.