Gateway Realtime

Realtime traffic goes through the same authentication and model-resolution layer as HTTP inference: the client must present gateway:invoke, the project Gateway must not be paused or have ingestion suspended, and the requested model resolves to an eligible Provider Key. Because the two sockets are then bridged byte-for-byte, per-event guardrails, usage-limit accounting, and trace telemetry are limited to the session setup and upstream attempt rather than to every streamed event (see Limitations).

When To Use It#

Use Gateway Realtime when:

• the application already uses OpenAI-compatible realtime clients;
• you want voice or multimodal sessions to pass through Provon routing and evidence capture;
• multiple provider keys or regions need the same application-level model ID.

Do not use Gateway Realtime for plain request/response chat; use /chat/completions instead.

Supported Paths#

Append these operation paths to the Gateway base URL:

Hosted base URL:

wss://gateway.provon.dev/v1/realtime?model=openai/gpt-4o-realtime-preview

Local development base URL:

ws://127.0.0.1:3000/gateway/v1/realtime?model=openai/gpt-4o-realtime-preview

The /v1 prefix is optional for the WebSocket path; /gateway/v1/realtime and /v1/realtime are both accepted.

Prerequisites#

1. The deployment supports terminating WebSocket upgrades. If a runtime returns REALTIME_WEBSOCKET_UNAVAILABLE, use a runtime listed as Realtime-capable in the deployment capability matrix; do not expect a plain HTTP-only runtime to proxy upgrades.
2. The target provider key has a mapping that declares the realtime endpoint family and the model you request.
3. The project API key used by the client has gateway:invoke.

Query eligible realtime models:

export PROVON_API_URL="https://api.provon.dev/v1"

curl \
  "$PROVON_API_URL/gateway/model-selection?endpoint=realtime&features=audio-input,audio-output" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Connect From A Browser#

const ws = new WebSocket(
  'wss://gateway.provon.dev/v1/realtime?model=openai/gpt-4o-realtime-preview',
  [],
  {
    headers: {
      Authorization: `Bearer ${PROVON_API_KEY}`,
      'x-otel-gen-ai-conversation-id': 'realtime-session-123',
      'x-otel-user-id': 'user-456',
    },
  },
);

ws.addEventListener('open', () => {
  ws.send(
    JSON.stringify({
      type: 'session.update',
      session: { model: 'gpt-4o-realtime-preview' },
    }),
  );
});

ws.addEventListener('message', (event) => {
  const message = JSON.parse(event.data);
  console.log(message);
});

The model query parameter can be provider-qualified (openai/gpt-4o-realtime-preview) or a plain model ID if a project Model Policy resolves it. Provon strips the provider prefix before forwarding to OpenAI-compatible upstreams and adjusts the path for Azure targets.

Client Secret#

OpenAI's client SDKs often expect an ephemeral secret from /realtime/client_secrets. The Gateway normalizes the request body, resolves the model, and forwards it to the upstream provider:

curl -X POST "$PROVON_GATEWAY_URL/realtime/client_secrets" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-realtime-preview",
    "session": {
      "model": "gpt-4o-realtime-preview",
      "modalities": ["audio", "text"]
    }
  }'

This is an inference call against the Gateway host (gateway.provon.dev/v1), not the management/API host.

Calls And Transcription Sessions#

/realtime/calls and /realtime/transcription_sessions follow the same normalization rules as /realtime/client_secrets. Send the OpenAI-compatible body; Provon resolves the nested model fields and routes to the selected target.

Fallback And Recovery#

Realtime WebSocket connections support ordered fallback across multiple eligible Provider Keys. If the first upstream WebSocket cannot be established, the Gateway tries the next prepared connection. Once the bridge is active, recovery is no longer possible without reconnecting the client.

Configure fallback in the Provider Key's fallbackPolicy as described in Routing and reliability.

Error Codes#

Limitations#

• Realtime uses raw WebSocket passthrough. The Gateway does not inspect or rewrite individual realtime events after the bridge is established.
• Request/response guardrails that act on the full body run only on the setup handshake, not on each streamed event.
• Trace capture records the session lifecycle and upstream attempt, not per-event telemetry.
• The upstream WebSocket handshake is bounded by the provider key's attempt timeout; client messages sent before the upstream socket opens are buffered and replayed once it connects.
• Once the bridge is active, a failed upstream cannot be retried without the client reconnecting.

Verify a Realtime session#

1. The model query parameter resolves in capability discovery (endpoint=realtime) before the client connects.
2. On connect, the handshake returns an upgrade and the trace shows one upstream attempt for the session; a missing mapping surfaces as UPSTREAM_NOT_CONFIGURED at setup time.
3. Send a minimal session.update and confirm the assistant response event type matches the model family before enabling voice input or tools.
4. After closing the client connection, confirm the session span closes with a normal close reason rather than an upstream failure.