Findings Quickstart

Prerequisites#

• A Provon project and project API key with telemetry:ingest and telemetry:read; see Install Provon or the hosted Workbench.
• At least one diagnostic Rule enabled on the Findings page.
• A GitHub OAuth app or personal token only if you hand a Finding off to GitHub.

1. 1
   Record a conversation

   Choose one capture path. The run should include a clear user goal, at least one tool or model step, and — if you want a Finding — a failure, skipped verification, or unsupported claim. Healthy conversations often produce no Finding, which is expected.

   AI GatewayOpenTelemetryAgent transcript

   bashCopy

   export PROVON_API_KEY="your_project_api_key"
   export PROVON_GATEWAY_URL="https://gateway.provon.dev/v1"
   
   curl "$PROVON_GATEWAY_URL/chat/completions" \
     -H "Authorization: Bearer $PROVON_API_KEY" \
     -H "Content-Type: application/json" \
     -H "x-otel-gen-ai-conversation-id: quickstart-conversation-1" \
     -H "x-otel-gen-ai-agent-id: quickstart-agent" \
     -d '{
       "model": "openai/gpt-5-mini",
       "messages": [
         {"role": "user", "content": "Find the failed deployment and fix it."}
       ]
     }'

   Guardrail facts appear only on calls actually proxied through the Gateway.

2. 2
   Verify the trace

   Open Traces in the Workbench and select the new trace. Confirm:

   • gen_ai.conversation.id is stable across the run;
   • the root span states the user goal;
   • model, tool, and final-answer spans appear in causal order;
   • a failure or terminal state is visible when one was expected.

   Use provon traces summarize <traceId> for a quick CLI read; it summarizes telemetry only and does not run Rules. See Diagnosis-ready tracing if any evidence above is missing.

3. 3
   Enable diagnosis

   Open Findings:

   1. Open the Diagnostic rules popover and enable at least one Rule; automatic diagnosis runs while any Rule remains enabled.
   2. For historical traces, run an enabled Rule through the historical scan dialog: pick Last 24 hours (the dialog also offers 7 days, 30 days, 90 days, all time, and a custom range).

   Automatic diagnosis queues after a conversation has been inactive for 10 minutes. The Findings queue updates when a candidate clears thresholds; below-threshold items appear in the review tier rather than as Findings.

4. 4
   Review the candidate

   Expand the Finding or review item and check:

   • Severity matches the impact.
   • Diagnostic confidence supports acting on the evidence.
   • Cause assessment points to a failure mechanism rather than a symptom.
   • Evidence links to the trace and span that support the claim.
   • Remediation hint gives a narrow repair direction.

   If it is not actionable, use Dismiss with a reason. If it is actionable, use Confirm.

5. 5
   Hand off to GitHub

   1. On the Findings page, open the issue tracker menu and choose GitHub.
   2. Connect with OAuth or paste a token.
   3. Set the default owner and repository, then save the connector.
   4. Expand the Finding and choose Create repair issue.

   Provon creates an issue with a focused repair goal, the Finding summary and cause assessment, evidence links, a remediation hint, the provon-repair label, and hidden correlation metadata. The Finding becomes confirmed and tracks the work item.

6. 6
   Verify the outcome

   After the fix ships:

   1. Run the same agent task again.
   2. If the failure is gone, no new occurrence is published; a merged pull request with successful checks on the linked issue resolves the Finding automatically.
   3. If a new occurrence appears, a resolved Finding returns to Regressed.
   4. For GitLab, Linear, and Jira, completion enters awaiting_verification until you verify the repair; the tracker being closed is not by itself resolved.

Limitations#

• A hand-built hello-world request or a healthy session rarely satisfies Rule evidence floors and will not produce a Finding.
• Backfill ranges and the conversation budget cap how much history one scan covers; see Rules.
• Local provon diagnose output never enters this queue; persistence happens server-side over ingested evidence.