Integrate SDKs With The Gateway

Complete the Gateway quickstart first so the provider, project key, and trace path are known to work.

The Gateway does not require a Provon-specific inference SDK. The stable contract is the Gateway base URL, Bearer authentication, the request model, and the optional evidence-context headers.

Shared configuration#

export PROVON_GATEWAY_URL="https://gateway.provon.dev/v1"
export PROVON_API_KEY="your_project_api_key"

For local development, set PROVON_GATEWAY_URL to http://127.0.0.1:3000/gateway/v1.

PROVON_API_KEY is a Provon project API key with gateway:invoke. Do not put an upstream provider key in the application.

Clients that read the standard OpenAI environment variables can often be redirected without code changes:

export OPENAI_BASE_URL="$PROVON_GATEWAY_URL"
export OPENAI_API_KEY="$PROVON_API_KEY"

Use a provider-qualified model such as openai/gpt-4o-mini during migration. Introduce plain model names or provon/auto only after routing policy has been tested.

Direct HTTP#

Any HTTP client can call the Gateway:

const response = await fetch(`${process.env.PROVON_GATEWAY_URL}/chat/completions`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.PROVON_API_KEY}`,
    'Content-Type': 'application/json',
    'x-otel-gen-ai-conversation-id': 'conversation-123',
    'x-otel-gen-ai-agent-id': 'support-agent',
  },
  body: JSON.stringify({
    model: 'openai/gpt-4o-mini',
    messages: [{ role: 'user', content: 'Summarize this ticket.' }],
  }),
});

if (!response.ok) throw new Error(await response.text());
console.log(await response.json());

Log the x-provon-request-id response header with the application request:

console.log(response.headers.get('x-provon-request-id'));

OpenAI Python SDK#

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PROVON_API_KEY"],
    base_url=os.environ["PROVON_GATEWAY_URL"],
    default_headers={
        "x-otel-gen-ai-agent-id": "support-agent",
        "x-otel-gen-ai-agent-name": "Support Agent",
    },
)

response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Summarize this ticket."}],
    extra_headers={
        "x-otel-gen-ai-conversation-id": "conversation-123",
        "x-otel-user-id": "user-456",
    },
)

print(response.choices[0].message.content)

The same client can use the Responses API when the selected target supports it natively or has the stateless Responses compatibility bridge enabled:

response = client.responses.create(
    model="openai/gpt-4o-mini",
    input="Summarize this ticket.",
    extra_headers={
        "x-otel-gen-ai-conversation-id": "conversation-123",
        "x-otel-user-id": "user-456",
    },
)

print(response.output_text)

OpenAI JavaScript / TypeScript SDK#

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.PROVON_API_KEY,
  baseURL: process.env.PROVON_GATEWAY_URL,
  defaultHeaders: {
    'x-otel-gen-ai-agent-id': 'support-agent',
    'x-otel-gen-ai-agent-name': 'Support Agent',
  },
});

const response = await client.chat.completions.create(
  {
    model: 'openai/gpt-4o-mini',
    messages: [{ role: 'user', content: 'Summarize this ticket.' }],
  },
  {
    headers: {
      'x-otel-gen-ai-conversation-id': 'conversation-123',
      'x-otel-user-id': 'user-456',
    },
  },
);

console.log(response.choices[0]?.message.content);

Streaming uses the SDK's normal stream option. The Gateway preserves the provider-compatible stream and records final usage and attempt evidence when the provider exposes it.

Native Messages payloads#

Where the selected target declares the native Messages endpoint, the Gateway also forwards /messages payloads under the Gateway base URL. Use Bearer authentication even when the upstream provider normally uses x-api-key:

curl "$PROVON_GATEWAY_URL/messages" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -H "x-otel-gen-ai-conversation-id: conversation-123" \
  -d '{
    "model": "anthropic/claude-haiku",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Summarize this ticket."}]
  }'

Do not assume endpoint parity across providers. Query the capability matrix for the exact target before relying on a native payload shape.

Frameworks and agent runtimes#

For LangChain, LlamaIndex, the Vercel AI SDK, or another framework:

1. Select its OpenAI or OpenAI-compatible provider adapter.
2. Set the base URL to PROVON_GATEWAY_URL.
3. Set the API key to PROVON_API_KEY.
4. Use a model id available in the capability matrix.
5. Add the context headers through the framework's default-header or per-request option.

Framework configuration names change across releases. The invariant contract is:

Base URL:      https://gateway.provon.dev/v1
Authorization: Bearer <Provon project API key>
Model:         <provider>/<model>, plain policy model, or provon/auto

Do not add a second /v1 when the configured Gateway URL already ends in /v1. The client must be able to send Authorization: Bearer <Provon project API key>; a client that hard-codes provider-specific authentication without a header override is not drop-in compatible.

Diagnostic context#

Stable context makes individual model calls useful to conversation diagnostics:

Use the same conversation id for every turn in one user goal. Use non-secret, pseudonymous values; these headers become retained trace attributes.

Set request-scoped values such as conversation, session, and user ids on each request. Do not place one user's identifiers in process-wide default headers. Agent name and version can be client-wide when one client instance represents one deployed agent.

Integration checklist#

1. Connect one provider key and keep Request traces set to All.
2. Verify the application endpoint is https://gateway.provon.dev/v1 or the equivalent self-hosted Gateway base URL.
3. Replace the provider key with a Provon project key.
4. Start with a provider-qualified model.
5. Send stable conversation and agent identifiers.
6. Test streaming, tools, structured output, and multimodal inputs separately.
7. Verify the root and provider-attempt spans in Traces.
8. Add limits, guardrails, and recovery policy before increasing traffic.

For rollout and rollback gates, continue with Run The Gateway In Production.