Skip to content

Get started with Provon

Provon turns real agent traces into actionable fixes, reusable Knowledge, reviewed Datasets, and model improvements. Every output remains grounded in the production behavior that m

View as Markdown Open the plain-text version of this page.

Pick the path that matches your goal:

Goal Start here
Understand what Provon does and where it fits What Provon does
Install or access Provon Install Provon
Complete the first trace-to-Finding flow Quickstart
Choose Gateway, OTLP, or transcript sync Choose a capture path
Follow the Workbench onboarding flow Workbench onboarding
Fix a recurring agent failure Repair loop
Improve and redeploy a model from traces Model improvement loop
Self-host Provon Self-hosting
Use the CLI or HTTP APIs CLI · API reference

What Provon does#

flowchart LR
  agent["Agent"] --> capture["Capture traces"]
  capture --> understand["Understand behavior"]
  understand --> findings["Findings"]
  understand --> knowledge["Knowledge"]
  understand --> datasets["Datasets"]
  findings --> repair["Repair handoff"]
  knowledge --> improve["Improve prompts and workflows"]
  datasets --> train["Evaluate or fine-tune"]
  train --> deploy["Deploy through the Gateway"]
  deploy --> agent

Core loop#

Stage Provon capability Result
Capture Gateway traces, OTLP/HTTP ingest, and transcript projection Project-scoped evidence of what the agent actually did
Understand Trace search, conversation reconstruction, dashboards, and Rules Reviewable behavior, performance, cost, and failure context
Improve Findings, Knowledge extraction, and Dataset collection Evidence-backed inputs for repair, evaluation, and training
Act Issue-system handoff, local model runtime, and fine-tuning A repaired harness or a new model checkpoint
Verify New production traces tied to the same observable behavior Evidence that the change worked or regressed

Provon does not silently modify an application or accept generated conclusions as truth. Users retain control of repair, review, training, and deployment decisions.

Evidence capture#

AI Gateway#

The OpenAI-compatible Gateway routes requests to cloud, custom, and local/private model targets; applies retries, fallback, load balancing, affinity, circuit breaking, and routing policies; enforces request, token, and cost controls; runs data, tool-use, and context guardrails; and captures request, response, attempt, token, cost, latency, error, and guardrail evidence.

Use Gateway capture when Provon should own the model-call path. See the Gateway overview and Gateway quickstart.

OpenTelemetry#

Provon accepts standard OTLP/HTTP traces, logs, and metrics in protobuf or JSON. It normalizes GenAI semantics and projects telemetry into trace, conversation, user, cost, latency, and diagnostic views.

Use OTLP when the agent already has instrumentation or must keep calling its provider directly. See OpenTelemetry setup.

Agent transcripts#

The CLI can project persisted Claude Code, Codex, and Pi transcripts into canonical OpenTelemetry spans and upload them through the same evidence path. See Agent transcript sync.

Diagnostics and Findings#

Provon reconstructs an ordered conversation trajectory, derives facts, detects deterministic signals, and uses model adjudication only where semantic judgment is required. A Finding combines a review state, severity, confidence, problem statement, cause assessment, remediation hint, and trace evidence. Users can confirm, dismiss, or hand a Finding off to an issue tracker.

See Findings and Trace diagnosis with CLI.

Supporting capabilities#

  • Observability views: trace lists, search, waterfalls, span detail, conversation summaries, token/cost/latency projections, configurable dashboards, and retention policies.
  • Playground: replay a production trace, compare prompt and model variants, and record rerun references.
  • Self-hosted models and fine-tuning: import open-weight models, train LoRA or full fine-tuning jobs from chat Datasets, register checkpoints, and deploy them behind self/ Gateway targets. Requires the Node runtime; not available in the Cloudflare runtime.
  • Knowledge extraction: define project goals and extract structured knowledge items from conversations, with confidence scores and evidence references.
  • Datasets: collect reviewed Examples under one explicit improvement objective and export them for evaluation, preference learning, or supervised fine-tuning.
  • Connectors: encrypted credentials for issue-system handoff and notifications.

Product boundary#

Provon is not an agent framework, a replacement for OpenTelemetry, a coding agent that edits repositories, a general-purpose ticketing platform, or a guarantee that every trace problem should become a Finding. Its responsibility is narrower: preserve production evidence, publish supported problems, and make agent improvement reviewable from the original behavior through the resulting repair or model.