Skip to content

Gateway Quickstart

By the end of this guide, one model request will have passed through the AI Gateway and the same request will be visible as trace evidence in the Workbench.

View as Markdown Open the plain-text version of this page.

Prerequisites#

  • A Provon project in the hosted Workbench or a local/self-hosted deployment
  • An upstream provider API key for the bring-your-own-key (BYOK) path
  • curl

1. Open Provon#

Open https://app.provon.dev and create an account, organization, and project when prompted.

To run a local development deployment instead:

bash
pnpm install
pnpm dev

Then open http://127.0.0.1:3000.

2. Create the application key#

In the project:

  1. Open API keys.
  2. Select Create API key.
  3. Keep gateway:invoke enabled. Keep workspace:read too if this key will call Gateway discovery endpoints.
  4. Copy the secret when shown; it cannot be retrieved later.

Export the hosted example values:

bash
export PROVON_API_KEY="your_project_api_key"
export PROVON_GATEWAY_URL="https://gateway.provon.dev/v1"

For local development, use http://127.0.0.1:3000/gateway/v1 for PROVON_GATEWAY_URL.

This key authenticates the application to Provon. It is not an OpenAI, Anthropic, or other upstream provider key.

3. Connect a model target#

The portable default is BYOK: you supply an upstream provider key and the Gateway holds it server-side. A deployment operator may additionally expose managed models as a read-only model catalog; callers never configure upstream keys or vendor targets for those. This guide uses the BYOK path, which works on both the hosted edge (Cloudflare Worker) and the self-hosted Node runtime.

In the Workbench:

  1. Open Providers and select the upstream provider.
  2. Under Provider keys, select Add key.
  3. Store the upstream provider API key.
  4. Leave Models set to All, or add an explicit model mapping.
  5. Keep the Provider Key enabled.
  6. Set Request traces to All while validating the integration.

With an explicit mapping, the AI Gateway model is the model ID sent by the application and the Upstream model is the ID sent to the provider. Endpoint and feature declarations restrict when that mapping is eligible.

The application never receives the upstream credential. See Model providers for target access, provider keys, and model mappings, and Gateway CLI for the terminal equivalent.

4. Send the first request#

Use a provider-qualified model for the first request so routing intent is unambiguous:

bash
curl -i "$PROVON_GATEWAY_URL/chat/completions" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-otel-gen-ai-conversation-id: quickstart-conversation-1" \
  -H "x-otel-gen-ai-agent-id: quickstart-agent" \
  -H "x-otel-gen-ai-agent-name: Quickstart Agent" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "Explain why an agent repair needs production evidence."
      }
    ]
  }'

Replace openai/gpt-4o-mini with a model available through the provider key you configured.

Do not use a plain model name or provon/auto for this first check. Those forms add routing policy to a transport test.

5. Verify the response#

A successful model path returns:

  • an HTTP 2xx response in the provider-compatible shape;
  • an x-provon-request-id response header;
  • model output in the response body.

Record x-provon-request-id. The response proves that inference worked; it does not prove that the evidence path worked.

If this step fails, start with Gateway troubleshooting before changing routing or guardrail settings.

6. Verify the trace#

Open Traces in the same project and locate the request by its timestamp or x-provon-request-id.

Trace delivery can be asynchronous in split-runtime deployments. The Gateway accepts the request and forwards captured evidence to the telemetry runtime; it is normal for the trace to appear a few seconds after the response. Wait 10–30 seconds and refresh before troubleshooting.

Verify:

  • the root operation is POST /v1/chat/completions;
  • the selected provider and model are correct;
  • an upstream attempt appears as a child span;
  • latency, status, token usage, and cost appear when available;
  • the conversation is quickstart-conversation-1;
  • the agent is quickstart-agent.

If the response succeeded but no trace appears, confirm that Providers → Request traces is set to All. See Gateway tracing for capture modes and Gateway troubleshooting for the verification sequence.

You are ready#

The Gateway path is working when both checks pass:

  1. The client receives the expected provider-compatible response and x-provon-request-id.
  2. The same request appears in Traces with its provider attempt and conversation context.

Keep the project key and upstream provider key separate. Use stable, non-secret identifiers for conversation, user, session, and agent context because these values may be retained with telemetry.