Evidence and Verdicts

The evidence layer reconstructs semantic evidence from attributed completion chunks — entirely client-side. This page explains what the scores are made of, so you can decide what to do with them.

Honest by design

Attribution grounding measures provenance, not truth: it tells you how strongly generated words trace back to conversation context. A claim can be perfectly grounded in a source that is itself wrong, and a correct claim can be ungrounded because the model knew it from training. That is why verdicts without a verifier never say "supported" — only grounded, partially_grounded, or ungrounded.

How claims are segmented

Visible text is accumulated until a claim becomes stable: at a newline, at sentence-ending punctuation (., !, ? followed by whitespace), or at the max_claim_chars limit (default 500, split at the last space). Each claim records its exact character span in the assistant text (output_span), so you can highlight it in your UI.

The grounding score

Each claim's grounding_score in [0, 1] combines four signals:

grounding = (0.65 · coverage + 0.25 · source_share + 0.10 · context_overlap)
            · (0.75 + 0.25 · source_quality)

• attribution_coverage — the term-weighted share of the claim that has any positive input attribution at all. The dominant signal: unattributed stretches of text drag it down.
• source share — input-attribution mass relative to total mass. Text that mostly draws on the model's own earlier output scores low.
• context_overlap — the share of the claim's important terms (numbers, negations, capitalized and long words) that literally appear in the attributed source texts. A weak lexical cross-check on top of attribution.
• source quality — an attribution-weighted average over the claim's sources: tool results 1.0, system messages 0.9, user messages 0.75, assistant messages 0.5, tool descriptions 0.25. Quality scales the score by at most ±25%.

Two further fields do not enter the score but are useful signals: instruction_grounding (how much of the input mass comes from system and user messages) and sources (the ranked source list with the exact attributed words).

Verdicts

Verdict	Produced when
contradicted	A verifier reported contradicts with score ≥ 0.5.
supported	A verifier reported entails with score ≥ 0.5.
grounded	Grounding score ≥ 0.65.
partially_grounded	Grounding score ≥ 0.3.
ungrounded	Grounding score below 0.3.

Verifier verdicts take precedence over attribution thresholds; claims judged with a verifier report method: "attribution_and_verifier". The unknown verdict value is reserved for host code that post-processes evidence — the SDK itself never emits it.

Output-kind detection

The output_kind event fires once, as soon as the kind of the first output is observed — tool_call on the first tool-call chunk, text on the first non-whitespace text. Its definitive flag is true because the kind was directly observed rather than guessed. A response may still continue with the other kind afterwards; the final ResponseEvidence.output_kind reports text, tool_call, or mixed over the whole response.

Tool-call evidence

Completed tool calls are assessed on three independent axes:

1. Structure — JSON parsing and schema validation against your tool definitions (types, enums, required arguments, additional properties), reported as machine-readable schema_issues, plus completeness (the share of required arguments present; 0 for unparseable or unknown calls).
2. Grounding — parameters links each argument leaf to its context sources (share of input mass plus a lexical match bonus), and intent_grounding measures how much the call as a whole traces back to system, user, and tool messages.
3. Policy — your local ToolPolicy (risk level, approval requirement, side effect, sensitive parameters) produces risk_reasons and drives the final decision: block on schema violations, review when policy demands approval, otherwise allow.

The final report

The closing response_evidence event aggregates everything: overall_grounding averages the claim grounding scores (or the tool-call intent groundings when the response contains no claims), and requires_review is set when any claim is contradicted, ungrounded, or unknown, or any tool call decided other than allow.