Tracing Quickstart

Send one diagnosis-ready agent trace to Provon over standard OTLP/HTTP, inspect it, and run the built-in diagnostic Rules with the CLI.

View as Markdown Open the plain-text version of this page.

This guide uses a direct OTLP JSON request so the transport and evidence are visible. For application SDK setup, continue with OpenTelemetry setup. To capture model calls through Provon instead, use the Gateway quickstart.

Prerequisites#

A running Provon deployment.
A project API key with telemetry:ingest and telemetry:read.
curl, openssl, and a POSIX-compatible shell.
The provon CLI for the inspect and diagnose steps.

Configure the project API key:

export PROVON_API_KEY="your_project_api_key"

Hosted ingest defaults to https://otel.provon.dev/v1. For local development, set PROVON_OTEL_URL to http://127.0.0.1:3000/v1.

Choose a setup path#

Your situation	Path in this guide	Full docs
Prove the transport with a minimal request	Direct OTLP curl (Step 1)	OpenTelemetry
Existing app calling models through the Gateway	Gateway request (Step 1)	Multi-agent tracing
Existing Claude Code / Codex / Pi session history	Agent transcript sync (Step 1)	Agent transcripts

Steps#

Send a trace

Pick one capture path below. Each produces project-scoped trace evidence; the rest of this guide uses the trace ID printed by the direct OTLP request.

The example creates one bounded agent run with a child tool call. It uses stable operation names, explicit participant identity, readable input and output, and a conversation ID.

TRACE_ID="$(openssl rand -hex 16)"
CONVERSATION_ID="quickstart-$TRACE_ID"
ROOT_SPAN_ID="$(openssl rand -hex 8)"
TOOL_SPAN_ID="$(openssl rand -hex 8)"
START_NS="$(($(date +%s) * 1000000000))"

curl "${PROVON_OTEL_URL:-https://otel.provon.dev/v1}/traces" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @- <<JSON
{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "quickstart-support-agent" }
          },
          {
            "key": "deployment.environment.name",
            "value": { "stringValue": "development" }
          }
        ]
      },
      "scopeSpans": [
        {
          "scope": { "name": "provon.quickstart" },
          "spans": [
            {
              "traceId": "$TRACE_ID",
              "spanId": "$ROOT_SPAN_ID",
              "name": "invoke-support-agent",
              "kind": 1,
              "startTimeUnixNano": "$START_NS",
              "endTimeUnixNano": "$((START_NS + 3000000))",
              "status": { "code": 1 },
              "attributes": [
                {
                  "key": "gen_ai.operation.name",
                  "value": { "stringValue": "invoke_agent" }
                },
                {
                  "key": "gen_ai.conversation.id",
                  "value": { "stringValue": "$CONVERSATION_ID" }
                },
                {
                  "key": "gen_ai.agent.id",
                  "value": { "stringValue": "support-agent" }
                },
                {
                  "key": "gen_ai.input.messages",
                  "value": {
                    "stringValue": "[{\"role\":\"user\",\"content\":\"Check whether order 123 can be refunded.\"}]"
                  }
                },
                {
                  "key": "gen_ai.output.messages",
                  "value": {
                    "stringValue": "[{\"role\":\"assistant\",\"content\":\"Order 123 is eligible for refund review.\"}]"
                  }
                }
              ]
            },
            {
              "traceId": "$TRACE_ID",
              "spanId": "$TOOL_SPAN_ID",
              "parentSpanId": "$ROOT_SPAN_ID",
              "name": "lookup-refund-policy",
              "kind": 1,
              "startTimeUnixNano": "$((START_NS + 1000000))",
              "endTimeUnixNano": "$((START_NS + 2000000))",
              "status": { "code": 1 },
              "attributes": [
                {
                  "key": "gen_ai.operation.name",
                  "value": { "stringValue": "execute_tool" }
                },
                {
                  "key": "gen_ai.conversation.id",
                  "value": { "stringValue": "$CONVERSATION_ID" }
                },
                {
                  "key": "gen_ai.agent.id",
                  "value": { "stringValue": "support-agent" }
                },
                {
                  "key": "gen_ai.tool.name",
                  "value": { "stringValue": "lookup_order" }
                },
                {
                  "key": "gen_ai.tool.call.id",
                  "value": { "stringValue": "call-quickstart-1" }
                },
                {
                  "key": "gen_ai.tool.call.arguments",
                  "value": { "stringValue": "{\"order_id\":\"123\"}" }
                },
                {
                  "key": "gen_ai.tool.call.result",
                  "value": {
                    "stringValue": "{\"status\":\"eligible_for_review\"}"
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}
JSON

printf 'trace id: %s\n' "$TRACE_ID"
printf 'conversation id: %s\n' "$CONVERSATION_ID"

A successful request returns asynchronous acceptance:

{
  "partialSuccess": {},
  "queued": true,
  "jobId": "ing_demo_abc123"
}

This confirms staging and queue admission, not query readiness. See the OTLP/HTTP API for the complete delivery contract.

Point an OpenAI-compatible client at the Gateway origin and add a stable conversation header:

export PROVON_GATEWAY_URL="https://gateway.provon.dev/v1"

curl "$PROVON_GATEWAY_URL/chat/completions" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-otel-gen-ai-conversation-id: quickstart-gateway-1" \
  -d '{
    "model": "openai/gpt-5-mini",
    "messages": [{ "role": "user", "content": "Refund order 123 and explain the policy." }]
  }'

Replace the model with one enabled for your provider. The Gateway records request, response, provider attempt, latency, tokens, and errors per the project's trace-capture settings. Guardrail outcomes appear only on calls actually proxied through the Gateway; see AI Gateway quickstart.

Project an existing Claude Code session without a live connection. First preview, then upload:

provon auth token set --api-key "$PROVON_API_KEY"
provon ingest sync ~/.claude/projects --once --dry-run --print-sample
provon ingest sync ~/.claude/projects --once

Use ~/.codex/sessions for Codex and ~/.pi/agent/sessions for Pi. Live hook status for each tool is listed in install hooks; backfill semantics are in Agent transcripts.

2
Verify the evidence
Open Traces and search for the printed trace ID. The direct-OTLP example should contain:
text
```
invoke-support-agent
`-- lookup-refund-policy
```
Verify:
- service.name is quickstart-support-agent;
- the tool span is a child of the agent span;
- both spans use the printed conversation ID;
- the root input and output are readable;
- the tool arguments and result are retained.
If the trace is not visible yet, wait for background ingest and summary materialization. An accepted payload that never appears is covered in Troubleshooting: OTLP succeeds but no trace appears.

Note
Telemetry ingestion succeeds when the trace is visible. A healthy, problem-free trace does not need to produce a Finding.
3
Inspect it from the CLI
CLI authentication reads the same project key:
bash
```
provon auth token set --api-key "$PROVON_API_KEY"
```
Search and summarize the trace:
bash
```
provon traces search "$TRACE_ID" --search-type trace-id --range 1h
provon traces summarize "$TRACE_ID"
```
traces summarize reports telemetry facts such as duration, spans, errors, tokens, cost, and slow operations. It does not run diagnostic Rules or create Findings.
4
Diagnose the same evidence with the CLI
Export a trace bundle, then diagnose it locally without uploading or persisting a Finding:
bash
```
provon traces export "$TRACE_ID" --out ./quickstart-trace
provon diagnose ./quickstart-trace --mode rule-signals
```
The healthy example should normally produce no Finding. That is a successful result: traces are evidence, while Findings are supported problems. Drop --mode rule-signals (use the default reviewed-diagnosis profile) when the calling agent can perform the hash-bound review round trip. See Trace diagnosis with CLI.

Limitations#

A hand-built curl trace demonstrates the transport; it lacks real tool failures and multi-turn history, so it will not exercise most Rules.
provon traces summarize is a read summary; diagnosis and Finding publication require Rules and a recorded conversation settled long enough to run.
Transcript sync projects persisted files; it cannot recover sessions that were never written to disk, and live hook support varies by tool (see the status table above).