Gateway API

Base URLs#

For hosted Provon:

For the local Node runtime:

Use the Gateway URL as the SDK base URL for model traffic. Use the API URL for discovery, Provider Key management, and other control-plane routes. In a one-origin self-hosted deployment, /gateway/v1 keeps model traffic distinct from Provon's control API and OTLP routes.

export PROVON_GATEWAY_URL="https://gateway.provon.dev/v1"
export PROVON_API_URL="https://api.provon.dev/v1"

Authentication#

Send a Provon project API key as a Bearer token:

Authorization: Bearer <PROVON_API_KEY>

Relevant API-key capabilities:

Workbench sessions use project permissions instead of these API-key capabilities.

Provider credentials are not valid Gateway client credentials. They remain attached to project-scoped Provider Keys.

First Request#

curl "$PROVON_GATEWAY_URL/chat/completions" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-otel-gen-ai-conversation-id: conversation-123" \
  -d '{
    "model": "openai/gpt-5-mini",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

The request and response follow the selected Gateway endpoint shape. Provider adapters may translate that shape to a native upstream protocol and translate the result back.

Model Values#

Provider-qualified values are the safest integration default because provider intent is explicit. Plain names are useful when the project owns a stable logical model name. provon/auto is useful when target selection should consider request fit, health, latency, and estimated cost.

See Routing and reliability.

Request Context#

Gateway context headers become OpenTelemetry attributes on the root and attempt spans. They do not change provider routing unless a separately configured policy uses the same value.

Values are trimmed and limited to 512 characters. Use stable pseudonymous user IDs when raw identity is not required. Never send secrets in context headers.

Every header above is caller context: it says who is calling, not what the inference call did. Caller context is recorded on every span of the request — the synthesized trace root, the SERVER span for the request, and the outbound CLIENT span — so a span can always be traced back to the conversation, agent, workflow and user it belongs to. Some of these attributes are named gen_ai.* because that is what the OTel GenAI conventions call them; the name is not what decides the role.

What the gateway measures itself — model, token usage, cost, response, message content — is inference telemetry and stays on the outbound CLIENT span, reported as gen_ai.* there and mirrored as gateway.* on the SERVER span.

For multi-turn diagnostics, x-otel-gen-ai-conversation-id is the most important field: it also sets the trace boundary, so every request carrying the same value joins one trace.

Trace Boundary#

trace_id resolves in this order:

1. x-otel-trace-id — explicit trace boundary.
2. x-otel-gen-ai-conversation-id — conversation-scoped trace boundary.
3. the request id — one trace per request (default when neither header is sent).

A trace is a span tree, so a trace covering several requests needs a span that parents them. For the first two cases Gateway synthesizes that root itself, marks it provon.trace.root=true, and parents each request's SERVER span to it, giving one connected tree for the whole conversation. Its span id is derived from the trace id, so a stateless Gateway computes the same value on every request and the repeated writes converge onto a single row. The third case adds no such span, because a single-request trace is already rooted by its own SERVER span.

Endpoint Discovery#

Gateway recognizes these endpoint families:

chat-completions      responses             messages
embeddings            rerank                ocr
image-generations     image-edits           image-variations
audio-transcriptions  audio-speech          moderations
batches               files                 fine-tuning
vector-stores         realtime              videos
video-extensions      video-edits           search
classify              pipeline              a2a
mcp-tools

Recognition does not imply that every provider supports every family. Query the project capability matrix:

curl "$PROVON_API_URL/gateway/capability-matrix?includeDisabled=false" \
  -H "Authorization: Bearer $PROVON_API_KEY"

The response distinguishes:

• native: the provider exposes the endpoint shape directly;
• translated: Provon adapts the Gateway shape to a provider-native endpoint;
• bridged: the public endpoint is implemented through another provider endpoint family;
• unsupported: the provider cannot serve that endpoint;
• available or disabled: project Provider Key availability.

Inspect eligible model targets for one endpoint and feature set:

curl \
  "$PROVON_API_URL/gateway/model-selection?endpoint=chat-completions&features=streaming,tool-calling" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Supported feature filters are:

streaming              tool-calling          parallel-tool-calls
json-mode              structured-outputs    reasoning
vision-input           audio-input            audio-output
prompt-caching         cache-read             cache-write
reasoning-tokens       usage-metadata         native-passthrough
custom-base-url

List the provider catalog:

curl "$PROVON_API_URL/gateway/providers" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Control API#

Project API keys with workspace:read can inspect Gateway configuration:

Keys with gateway:manage can write Provider Keys and mappings:

Signed-in Workbench routes also expose project-qualified variants under /v1/projects/:projectId/gateway/*. Prefer the unqualified paths for project API-key clients because the project is already derived from the key.

Custom Providers use a project-scoped provider ID and a static driverId. An exact endpoint URL is never extended with a protocol path:

{
  "provider": "internal-kimi",
  "providerKind": "custom",
  "driverId": "openai-compatible",
  "displayName": "Internal Kimi",
  "endpointConfig": {
    "mode": "exact-url",
    "endpointUrls": {
      "chat-completions": "https://models.example/custom/chat"
    },
    "endpointBridges": {
      "responses": "chat-completions"
    }
  },
  "authScheme": { "kind": "bearer" },
  "apiKey": "..."
}

endpointBridges.responses = "chat-completions" exposes /responses for an upstream that only implements Chat Completions. Provon performs stateless request, response, and SSE translation. Omit the bridge and configure a native responses endpoint URL when the upstream implements the Responses API directly.

Common Inference Paths#

Append these operation paths to the Gateway base URL:

This table lists common inference paths. The endpoint-family list above is the capability discovery taxonomy, not a guarantee that every family has a hosted /v1/* alias. Less common or provider-native paths include alternate forms such as /reranking, /realtime/calls, /realtime/transcription_sessions, /videos/*, /classification, /inference, /a2a, /mcp, and /mcp/tools. Use the capability matrix and the selected deployment's Gateway base URL before integrating one of those paths.

Streaming#

For OpenAI-compatible streaming endpoints, send the normal request field:

{
  "model": "openai/gpt-5-mini",
  "stream": true,
  "messages": [{ "role": "user", "content": "Write one sentence." }]
}

The candidate target must declare streaming. Provon streams the provider-compatible response while collecting final attempt, usage, guardrail, and trace evidence where available.

Request IDs#

Successful and failed inference responses include x-provon-request-id after request context is established. Log this value with the application request and use it to correlate:

• the client failure;
• the Gateway root trace ID;
• upstream attempt spans;
• usage-policy and guardrail evidence.

Clients may send x-provon-request-id, but generated IDs are safer unless the caller guarantees uniqueness.

Error Shape#

Gateway-owned errors use a JSON envelope:

{
  "error": {
    "code": "GATEWAY_PAUSED",
    "message": "gateway is paused for this project",
    "details": {}
  }
}

Usage-policy rejections return 429 with Retry-After, RateLimit-Limit, RateLimit-Remaining, and RateLimit-Reset. Guardrail blocks return 409 with PROVON_GATEWAY_GUARD_TRIGGERED and guardrail headers.

Provon Cloud managed model requests return 402 with INSUFFICIENT_GATEWAY_CREDITS when the organization cannot reserve the estimated request cost. A paused project Gateway returns 503 with GATEWAY_PAUSED. Model resolution with no eligible target returns 404 with UPSTREAM_NOT_CONFIGURED.

Upstream provider errors remain provider-compatible when possible and gain normalized classification in the Gateway trace.

Transport Limits#

Runtime defaults are:

• 4 MiB for JSON request bodies;
• 25 MiB for other supported request bodies;
• no compressed Gateway request body;
• a 60-second overall Gateway request deadline;
• 2 MiB maximum response body capture for telemetry.

Deployments can override these limits. A provider may impose stricter limits.