Provider abstraction, model routing, and fallbacks

Provider portability is valuable at stable concepts—messages, embeddings, tools, and vector stores—but model behavior is not interchangeable.

Route by role

Define configuration roles instead of scattering model IDs:

ai:
  roles:
    complex-reasoning: openai:gpt-5.6-sol
    interaction: google:gemini-3.6-flash
    high-volume: google:gemini-3.5-flash-lite
    private-draft: ollama:qwen3
    embedding: google:gemini-embedding-2

Every role needs measured quality, p50/p95 latency, cost, context needs, regional availability, tool/JSON support, and lifecycle status.

Fallback semantics

A fallback is safe only if it preserves:

data residency and retention rules;
prompt and output contracts;
tool schemas and authorization;
embedding dimensions and vector-space identity;
multimodal formats;
acceptable quality and latency.

Do not catch every provider error and silently send confidential data to another vendor. Circuit breakers protect capacity; they do not define product correctness.

Capability registry

Store model/provider capabilities in configuration: reasoning, streaming, structured output, image/audio input, tools, parallel tools, context limits, fine-tuning, lifecycle date, and approved data classes. Test the registry against live provider smoke calls.

Shadow and canary

Shadow requests can compare a candidate without changing user output, but they double data exposure and cost. Redact, sample, and obtain policy approval. Canary a new model by role and tenant; compare the same evaluation slice.

Feynman check

A provider adapter makes plugs look similar. It does not make the electricity, price, or appliance behavior identical.