Model Providers

Provider model#

Four layers have distinct responsibilities:

Pricing metadata is not routing truth. Adding a price entry does not make a model routable; a target still needs a configured provider key and an eligible binding.

Target access boundaries#

BYOK is the portable default: the application authenticates to Provon with its project API key; the Gateway authenticates upstream with the provider key it stores server-side.

When a deployment serves a managed catalog, those models appear as read-only entries in Providers. Callers never set an upstream key, base URL, or vendor preference for them, and the application sees only the public model names. Self-hosted deployments do not add a managed catalog by changing pricing metadata.

Provider types#

The runtime provider catalog is the source of truth. Query it instead of maintaining a static provider list in application code:

export PROVON_API_URL="https://api.provon.dev/v1"

curl "$PROVON_API_URL/gateway/providers" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Connect a provider#

In the Workbench:

1. Open Providers and choose a provider.
2. Under Provider keys, create a key with a descriptive name.
3. Add the upstream API key. You can add more than one credential later for load balancing or rotation.
4. Keep the official endpoint, or enter an allowed override where the provider supports one.
5. Optionally add header overrides that the Gateway sends only to this provider key.
6. Choose All or Specific model mapping mode.
7. Keep the provider key enabled.
8. Send a test request and verify its trace.

Success checks#

• provon gateway status lists the provider key as enabled, or the Workbench key row shows enabled state.
• A test request returns 2xx and a trace shows the selected provider key and upstream model id.
• A key you disabled disappears from eligible targets in capability discovery.

Header overrides are useful for provider-specific metadata such as routing tags, but they are privileged configuration: treat them as capable of carrying secrets. The application never receives the upstream credential or these overrides; it calls the Gateway with a project API key that has gateway:invoke.

Custom providers#

A custom provider has a project-scoped routing id and uses a built-in protocol driver. The openai-compatible driver covers OpenAI-compatible HTTP endpoints; user configuration never supplies executable adapter code.

Choose one endpoint mode:

Configure Responses API independently for each custom provider:

The compatibility mode supports text, image input, function calling, structured output, usage metadata, and streaming. It intentionally rejects stateful fields such as previous_response_id, conversation, store: true, and background: true.

For example, an exact endpoint configured as https://models.example/custom/chat receives:

{
  "model": "acme-kimi/kimi-2.7-code",
  "messages": [{ "role": "user", "content": "Hello" }]
}

The acme-kimi prefix selects the project-defined custom provider; the upstream receives kimi-2.7-code.

Provider key priority#

A project can configure multiple provider keys for the same provider. Drag keys to set priority. Matching requests use the first eligible key after capability, policy, health, and load-balancing decisions.

Use separate provider keys for:

• regions or provider deployments;
• independent upstream accounts;
• isolated model allowlists;
• distinct timeout or header requirements;
• ordered recovery targets.

Do not duplicate a key only to rename the same secret. Use one target per operational boundary.

Multiple credentials and load balancing#

A single provider key can hold more than one upstream credential. The runtime selects the active credential using credential priority and the key's load-balance policy. This lets you spread load across several API keys, rotate a secret without downtime, or isolate keys with different rate-limit headroom.

See Routing and reliability for the load-balance strategies and configuration example.

Model mappings#

Provider keys support two modes:

An explicit mapping contains:

Example:

AI Gateway model: support-model
Upstream model:   gpt-4o-mini
Endpoints:        chat-completions, responses
Features:         streaming, tool-calling, structured-outputs

The application can then send the provider-qualified public id:

{
  "model": "openai/support-model"
}

Plain "support-model" requests require a matching model policy or registry inference. See Routing and reliability.

Capability discovery#

Do not assume endpoint parity between providers or between two provider keys for the same provider.

curl "$PROVON_API_URL/gateway/capability-matrix?includeDisabled=false" \
  -H "Authorization: Bearer $PROVON_API_KEY"

The matrix includes:

• recognized endpoint families;
• provider-native, translated, bridged, and unsupported status;
• provider feature declarations;
• enabled provider keys and credentials;
• model mappings and their endpoint and feature declarations.

Filter model selection for the exact request shape:

curl \
  "$PROVON_API_URL/gateway/model-selection?endpoint=responses&features=streaming,tool-calling" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Custom OpenAI-compatible targets#

Use the openai-compatible provider for a user-owned endpoint that implements supported OpenAI request and response shapes.

Configure:

• a public HTTP(S) base URL; use HTTPS for production traffic;
• a provider key secret;
• explicit model mappings when the target should expose only named logical models;
• accurate endpoint and feature declarations for mapped models;
• an attempt timeout.

The URL validator rejects credentials in URLs, query strings, and fragments; for custom-base-url providers it also rejects loopback and internal hostnames and private or reserved IP literals. A deployment may additionally inject DNS resolution policy so that public names resolving to any non-public address are rejected too. Official provider adapters keep their fixed official endpoint unless their manifest explicitly allows an override.

Do not point a public custom provider key at a loopback or internal service.

Advanced provider key settings#

Provider keys can define:

• per-attempt timeout;
• header overrides sent only to that target;
• retry policy;
• fallback status and network-error policy;
• circuit-breaker policy;
• priority and credential ordering.

The Workbench exposes the commonly used timeout and header controls; provider key management APIs carry the reliability policy fields used by runtime adapters.

Recovery remains capability-aware: an alternate key or model that cannot satisfy the request is not a valid fallback.

Trace evidence#

When Trace Capture selects a request, the trace can show:

• selected provider, provider key, and credential fingerprint;
• requested, Gateway, and upstream model ids;
• translated or bridged operation shape;
• every provider attempt and fallback index;
• latency and time to first chunk;
• token, cache, reasoning, and cost fields;
• normalized provider errors and guardrail outcomes.

See Gateway tracing.

Security#

• Store upstream credentials only in project-scoped provider keys.
• Give calling applications Provon project API keys.
• Use Specific mappings for strict model allowlists.
• Rotate upstream and project keys independently.
• Restrict custom endpoints to intended public hosts.
• Treat header overrides as secrets-capable configuration.
• Do not expose model servers directly when the Gateway is the control point.