Findings Troubleshooting

An empty Findings queue is not a health signal: no incoming traces, disabled Rules, and a clean conversation all produce an empty queue. A closed tracker item is also not a resolution until the sync rules below agree.

No Findings appear#

1. Every Rule is disabled#

Automatic diagnosis runs while at least one Rule is enabled. Open Findings, open the Diagnostic rules popover, and confirm a Rule is switched on; disabling every Rule turns automatic diagnosis off.

2. The trace is not diagnosable#

A trace can be valid OpenTelemetry and still too weak for diagnosis. Provon needs:

• a stable gen_ai.conversation.id;
• the user goal and constraints;
• ordered model, tool, retrieval, and final-answer events;
• tool inputs, results, failures, and retries;
• a terminal answer or explicit non-completion.

Open the trace in Traces and compare it with the checklist in Diagnosis-ready tracing. Thin records (metadata only, redacted tool bodies) stay visible but do not satisfy Rule evidence floors.

3. The diagnosis has not run yet#

Automatic diagnosis queues after 10 minutes of conversation inactivity. For existing traces, run an enabled Rule through the historical scan dialog over a recent window (presets: last 24 hours, 7 days, 30 days, 90 days, all time, plus a custom range; default 30 days; a budget caps conversations per run).

4. Keywords filtered the conversation#

If a Rule has keywords, it only evaluates conversations containing one of those keywords (case-insensitive over captured text). Remove or broaden keywords to test.

5. Thresholds filtered the candidate#

Temporarily lower minSignalScore and minDiagnosticConfidence to see whether a candidate was close. If it appears, tune back up and decide whether a signal override fits instead. Below-threshold candidates surface as the non-scored review tier (leads), not as Findings.

6. The conversation is healthy#

Healthy conversations do not produce Findings. That is the expected result, not a defect. Review-only types (goal-misunderstood, retrieval-insufficient, input-underspecified, capability-unavailable) also never publish under the local deterministic profile; see Rules.

7. Traces exist but nothing queued#

• Confirm the traces actually persisted (not just an accepted 200); check the time range filter on the queue and the trace list.
• Confirm traces carry a conversation identity; ungrouped traces diagnose one trace only.
• A rejected or failed Rule Run retains audit counters but publishes nothing; check the run status below.

Too many false positives#

Raise the confidence bar#

Increase minDiagnosticConfidence. This is the fastest way to suppress low-confidence candidates.

Disable noisy categories#

Use signalOverrides to disable categories that are not actionable in your domain:

{
  "configuration": {
    "signalOverrides": {
      "repetition-no-progress": { "enabled": false },
      "incoherent-answer": { "enabled": false }
    }
  }
}

Lower severity, not publication#

If a category is useful but rarely urgent, override its severity to low or info instead of disabling it entirely.

Use the deterministic baseline for screening#

Run provon diagnose --mode rule-signals to skip model review and publish only mechanically confirmed signals. It is a local/offline-CI mode: recall on judgment-heavy categories drops, but output becomes predictable. Production Rules always run the reviewed-diagnosis profile and stay on the candidates_only default; there is no server-side no-model switch.

Handoff and work item errors#

HANDOFF_PROVIDER_UNAVAILABLE (503)#

Open the issue tracker menu on the Findings page, confirm the provider is enabled and reachable, and set the default owner/repo, project path, team, or project/issue type. Check that the connector token was not expired or revoked. Retry after the connector is healthy.

HANDOFF_ALREADY_EXISTS (409)#

A Finding can have only one current repair handoff. Retry or abandon a failed handoff, or use Replace when a primary work item already exists.

HANDOFF_NOT_RETRYABLE (409)#

Retry only works when the current FindingHandoff is failed. If a FindingWorkItem already exists, use Sync; otherwise use Replace to start a fresh handoff.

ACTIVE_WORK_ITEM (409)#

You cannot dismiss a Finding while its work item is todo, doing, or unknown. Finish the external item, sync, and then manage review state.

WORK_ITEM_NOT_DONE (409)#

Manual resolution (resolutionStatus: resolved) is blocked until the external work item reaches done. Sync the item first; GitHub can auto-resolve under the check rules in repair handoff.

Sync returns failed or error#

Check the work item message. Common causes:

• the OAuth token expired or was revoked;
• the external issue was deleted;
• the connector default destination changed;
• the external provider is unreachable or rate-limited.

Reconnect the connector, Sync again, or use Replace to create a tracked item under a working destination.

The sync vocabulary has two layers with different enums; do not treat the attempt outcome as the stored field:

Regression#

A resolved Finding becomes regressed only when a new occurrence is observed after resolvedAt. Historical backfills cannot regress a Finding. For unexpected regression, verify:

• the new trace was captured after the fix deployed;
• the conversation ID matches the cluster;
• the symptom is the same failure mechanism rather than a new, unrelated issue;
• a reopened or failing external item was synced back into Provon.

Rule run statuses#

A diagnosis Run is projected as completed, skipped, or failed from its conversation_analysis_run span; there is no separate Runs store.

See Diagnostic rules for the gate vocabulary and the Findings API for run and funnel endpoints.