Escalate Unsupported Claims to an LLM Judge
This recipe extends Hallucination detection. Attribution detects the suspicious claims while the answer streams: it measures which input each generated word came from. An LLM judge then gives a second opinion on those claims only. It either confirms the hallucination, and the claim is withheld, or rejects it, and the claim is shown. The judge's job is to prevent false alarms, not to find hallucinations.
The examples on this page are covered by the SDK's unit tests against a stubbed stream and judge.
Why not just judge every claim?
A judge without attribution can check claims too. What it cannot do:
| Attribution (invera) | LLM judge alone | |
|---|---|---|
| Which chunk and which words a claim comes from | yes, per word: citations, highlighting, audit trail | no, only a verdict |
| When it is available | while the answer streams | after the answer, one model call per claim |
| What it measures | what the model actually used to produce each word | what a second model believes about the claim |
| Reproducible | yes: greedy decoding, no sampling | depends on the judge model; it can hallucinate too |
| Cost per claim | none beyond the attributed stream | one LLM call |
Attribution alone has one weakness: it is strict. A correct claim that mostly repeats the question draws little attribution from the chunks and looks like model knowledge. Withholding it is a false alarm. The judge resolves exactly these cases, and it is only called for claims attribution flags.
When a claim goes to the judge
A claim is escalated in two cases:
- Attribution would not show it. Its action from the first recipe is
flagorwithhold(grounding belowSHOW_AT). - It is grounded, but hardly in the chunks. Grounding counts attribution
to any message, including the question and the system prompt. A claim
that the model takes from its own knowledge often scores high anyway,
because it repeats the words of the question. The chunk share catches
this: the part of the claim's attribution score that comes from the chunk
messages. A claim below
MIN_CHUNK_SHARE(0.5) is escalated.
Every other claim is shown without a judge call (see How many claims reach the judge for measured numbers).
| Judge reply | Meaning | Claim action |
|---|---|---|
SUPPORTED | hallucination rejected | show |
CONTRADICTED | hallucination confirmed: the chunks say otherwise | withhold |
NOT_FOUND | hallucination confirmed: the chunks do not say this | withhold |
| anything else | unclear | keep the action from attribution |
The answer gets the strictest action of its claims, as in the first recipe.
The judge
The judge receives the question, the claim, and all retrieved chunks, not only the cited ones: an unsupported claim often cites no chunk at all. The question matters as well. Claims such as "Yes, that is correct." can only be judged as part of the answer to it.
inveraJudge / invera_judge uses the model behind your invera Engine, so
the example needs no second provider. The judge is a plain function
(question, claim, chunks) → { judgement, raw }. To use another LLM, write a
function that sends the same prompt (JUDGE_PROMPT) to that LLM and passes its
reply to parseJudgement / parse_judgement.
The complete example
The example imports the first recipe, so put both files in the same
directory. Set INVERA_BASE_URL (and INVERA_API_KEY if your deployment
needs a key), then run npx tsx llm-judge-escalation.ts or
python llm_judge_escalation.py.
- TypeScript
- Python
// LLM judge escalation: ask a judge only about the claims attribution does not support.
import { InveraClient, type ClaimEvidence } from "@layerwise-ai/invera-sdk";
import { pathToFileURL } from "node:url";
// The first recipe, in the same directory: hallucination-detection.ts
import { SEVERITY, SYSTEM_PROMPT, answerWithEvidence, claimAction, type Action } from "./hallucination-detection.js";
// A claim that draws less than this share of its attribution from the retrieved chunks is
// escalated even when its grounding is high: it then rests on the question or the instructions,
// which is how model knowledge looks. Calibrate it on your own data, like SHOW_AT.
export const MIN_CHUNK_SHARE = 0.5;
export type Judgement = "supported" | "contradicted" | "not_found";
/** The judge's answer for one claim; `judgement` is null when it was not understood. */
export interface JudgeResult {
judgement: Judgement | null;
raw: string;
}
/** Judges one claim of the answer to `question` against the retrieved chunks. */
export type Judge = (question: string, claim: string, chunks: readonly string[]) => Promise<JudgeResult>;
export const JUDGE_PROMPT =
"You check whether a claim from an answer is supported by numbered context passages. " +
"Read the claim as part of the answer to the question; it may refer to the question. " +
"Reply with exactly one word: SUPPORTED if the passages state the claim, " +
"CONTRADICTED if the passages state something that conflicts with it, " +
"NOT_FOUND if the passages do not mention it.";
/** One claim with the action from attribution and, if it was escalated, the judge's view. */
export interface JudgedClaim {
claim: ClaimEvidence;
attributionAction: Action;
judge: JudgeResult | null;
action: Action;
/** True if the judge confirmed a hallucination, false if it rejected one, null if not judged. */
hallucination: boolean | null;
}
/** The streamed answer, the final action, and every claim with its decision. */
export interface EscalatedAnswer {
action: Action;
text: string;
claims: JudgedClaim[];
}
/** Read the first word of the judge's reply as a judgement. */
export function parseJudgement(text: string): Judgement | null {
const word = (text.trim().split(/\s+/)[0] ?? "").replace(/^[.:*`'"]+|[.:*`'"]+$/g, "").toUpperCase();
const judgements: Record<string, Judgement> = {
SUPPORTED: "supported",
CONTRADICTED: "contradicted",
NOT_FOUND: "not_found",
};
return judgements[word] ?? null;
}
/** Use the model behind the invera Engine as the judge. Any other LLM works the same way. */
export function inveraJudge(client: Pick<InveraClient, "chat">): Judge {
return async (question, claim, chunks) => {
const passages = chunks.map((chunk, index) => `[${index + 1}] ${chunk}`).join("\n");
let reply = "";
for await (const chunk of client.chat.completions.stream({
messages: [
{ role: "system", content: JUDGE_PROMPT },
{ role: "user", content: `Context passages:\n${passages}\n\nQuestion: ${question}\nClaim: ${claim}` },
],
maxTokens: 8,
topKAttributions: 1,
})) {
if (chunk.type === "text") reply += chunk.content ?? "";
}
return { judgement: parseJudgement(reply), raw: reply };
};
}
/** Return the share of the claim's attribution that comes from the retrieved chunks. */
export function chunkShare(claim: ClaimEvidence, chunkCount: number): number {
let total = 0;
let fromChunks = 0;
for (const source of claim.sources) {
total += source.attributionScore;
if (source.sourceType === "message" && source.sourceId !== null && source.sourceId >= 1 && source.sourceId <= chunkCount) {
fromChunks += source.attributionScore;
}
}
return total > 0 ? fromChunks / total : 0;
}
/** Escalate claims attribution would not show, and shown claims that barely use the chunks. */
export function needsJudge(claim: ClaimEvidence, chunkCount: number): boolean {
return claimAction(claim) !== "show" || chunkShare(claim, chunkCount) < MIN_CHUNK_SHARE;
}
/** Supported claims are shown; confirmed hallucinations are withheld; unclear ones stay. */
export function judgedAction(attributionAction: Action, result: JudgeResult): Action {
if (result.judgement === "supported") return "show";
if (result.judgement === "contradicted" || result.judgement === "not_found") return "withhold";
return attributionAction;
}
/** Answer with evidence, then let the judge decide only the claims attribution cannot. */
export async function answerWithEscalation(
client: Pick<InveraClient, "chat">,
question: string,
chunks: readonly string[],
judge: Judge,
systemPrompt: string = SYSTEM_PROMPT,
): Promise<EscalatedAnswer> {
const answer = await answerWithEvidence(client, question, chunks, undefined, systemPrompt);
const claims: JudgedClaim[] = [];
for (const claim of answer.claims) {
const action = claimAction(claim);
if (!needsJudge(claim, chunks.length)) {
claims.push({ claim, attributionAction: action, judge: null, action, hallucination: null });
continue;
}
// An unsupported claim may cite no chunk at all, so the judge sees every chunk.
const result = await judge(question, claim.text, chunks);
claims.push({
claim,
attributionAction: action,
judge: result,
action: judgedAction(action, result),
hallucination: result.judgement === null ? null : result.judgement !== "supported",
});
}
const final = claims.reduce<Action>(
(worst, item) => (SEVERITY[item.action] > SEVERITY[worst] ? item.action : worst),
claims.length > 0 ? "show" : "withhold",
);
return { action: final, text: answer.text, claims };
}
export async function main(): Promise<void> {
const client = new InveraClient({
baseUrl: process.env.INVERA_BASE_URL!,
// Only when the Engine deployment requires an API key.
apiKey: process.env.INVERA_API_KEY,
});
const chunks = [ // in your application: the top results of your retriever
"The Vellmar ferry line opened in 1931 and connects Ostrand with the island of Kessel.",
"The Kessel lighthouse was designed by the engineer Mira Talbot and first lit in 1874.",
"Kessel island has about 2,300 residents, most of whom work in fishing.",
];
// A prompt that lets the model add its own knowledge, so that the demo produces claims the
// chunks do not support. The first question gets one; the second only repeats the context.
const prompt = "Answer the question in two sentences. Use the context passages and your own knowledge.";
const judge = inveraJudge(client);
for (const question of [
"Who was the first person to walk on the Moon, and how many people live on Kessel island?",
"Is it true that the engineer Mira Talbot designed the Kessel lighthouse? Answer yes or no and repeat the claim.",
]) {
const answer = await answerWithEscalation(client, question, chunks, judge, prompt);
console.log(`${question}\n[${answer.action}] ${answer.text}`);
for (const item of answer.claims) {
const judged = item.judge ? `judge=${JSON.stringify(item.judge.raw.trim())}` : "not escalated";
console.log(
` ${item.attributionAction} -> ${item.action.padEnd(8)} ` +
`grounding=${item.claim.groundingScore.toFixed(2)} ` +
`chunks=${chunkShare(item.claim, chunks.length).toFixed(2)} ${judged} ${JSON.stringify(item.claim.text)}`,
);
}
}
}
// Run with Node.js (for example `npx tsx llm-judge-escalation.ts`).
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) await main();
"""LLM judge escalation: ask a judge only about the claims attribution does not support."""
from __future__ import annotations
import os
from collections.abc import Callable, Sequence
from dataclasses import dataclass
from typing import Literal
from invera_sdk import ChatMessage, ClaimEvidence, CompletionRequest, InveraClient
# The first recipe, in the same directory: hallucination_detection.py
from hallucination_detection import (
SEVERITY,
SYSTEM_PROMPT,
Action,
answer_with_evidence,
claim_action,
)
# A claim that draws less than this share of its attribution from the retrieved chunks is
# escalated even when its grounding is high: it then rests on the question or the instructions,
# which is how model knowledge looks. Calibrate it on your own data, like SHOW_AT.
MIN_CHUNK_SHARE = 0.5
Judgement = Literal["supported", "contradicted", "not_found"]
Judge = Callable[[str, str, Sequence[str]], "JudgeResult"]
"""Judges one claim: ``judge(question, claim, chunks)``."""
JUDGE_PROMPT = (
"You check whether a claim from an answer is supported by numbered context passages. "
"Read the claim as part of the answer to the question; it may refer to the question. "
"Reply with exactly one word: SUPPORTED if the passages state the claim, "
"CONTRADICTED if the passages state something that conflicts with it, "
"NOT_FOUND if the passages do not mention it."
)
@dataclass(frozen=True)
class JudgeResult:
"""The judge's answer for one claim; ``judgement`` is None when it was not understood."""
judgement: Judgement | None
raw: str
@dataclass(frozen=True)
class JudgedClaim:
"""One claim with the action from attribution and, if it was escalated, the judge's view."""
claim: ClaimEvidence
attribution_action: Action
judge: JudgeResult | None
action: Action
@property
def hallucination(self) -> bool | None:
"""True if the judge confirmed it, False if the judge rejected it, None if not judged."""
if self.judge is None or self.judge.judgement is None:
return None
return self.judge.judgement != "supported"
@dataclass(frozen=True)
class EscalatedAnswer:
"""The streamed answer, the final action, and every claim with its decision."""
action: Action
text: str
claims: tuple[JudgedClaim, ...]
def parse_judgement(text: str) -> Judgement | None:
"""Read the first word of the judge's reply as a judgement."""
words = text.strip().split()
word = words[0].strip(".:*`'\"").upper() if words else ""
return {
"SUPPORTED": "supported",
"CONTRADICTED": "contradicted",
"NOT_FOUND": "not_found",
}.get(word)
def invera_judge(client: InveraClient) -> Judge:
"""Use the model behind the invera Engine as the judge. Any other LLM works the same way."""
def judge(question: str, claim: str, chunks: Sequence[str]) -> JudgeResult:
passages = "\n".join(
f"[{number}] {chunk}" for number, chunk in enumerate(chunks, start=1)
)
request = CompletionRequest(
[
ChatMessage("system", JUDGE_PROMPT),
ChatMessage("user", f"Context passages:\n{passages}\n\nQuestion: {question}\nClaim: {claim}"),
],
max_tokens=8,
top_k_attributions=1,
)
reply = "".join(
chunk.content or ""
for chunk in client.chat.completions.stream(request)
if chunk.type == "text"
)
return JudgeResult(parse_judgement(reply), reply)
return judge
def chunk_share(claim: ClaimEvidence, chunk_count: int) -> float:
"""Return the share of the claim's attribution that comes from the retrieved chunks."""
total = sum(source.attribution_score for source in claim.sources)
from_chunks = sum(
source.attribution_score
for source in claim.sources
if source.source_type == "message"
and source.source_id is not None
and 1 <= source.source_id <= chunk_count
)
return from_chunks / total if total > 0 else 0.0
def needs_judge(claim: ClaimEvidence, chunk_count: int) -> bool:
"""Escalate claims attribution would not show, and shown claims that barely use the chunks."""
return claim_action(claim) != "show" or chunk_share(claim, chunk_count) < MIN_CHUNK_SHARE
def judged_action(attribution_action: Action, result: JudgeResult) -> Action:
"""Supported claims are shown; confirmed hallucinations are withheld; unclear ones stay."""
if result.judgement == "supported":
return "show"
if result.judgement in ("contradicted", "not_found"):
return "withhold"
return attribution_action
def answer_with_escalation(
client: InveraClient,
question: str,
chunks: Sequence[str],
judge: Judge,
*,
system_prompt: str = SYSTEM_PROMPT,
) -> EscalatedAnswer:
"""Answer with evidence, then let the judge decide only the claims attribution cannot."""
answer = answer_with_evidence(client, question, chunks, system_prompt=system_prompt)
judged: list[JudgedClaim] = []
for claim in answer.claims:
action = claim_action(claim)
if not needs_judge(claim, len(chunks)):
judged.append(JudgedClaim(claim, action, None, action))
continue
# An unsupported claim may cite no chunk at all, so the judge sees every chunk.
result = judge(question, claim.text, chunks)
judged.append(JudgedClaim(claim, action, result, judged_action(action, result)))
final = max(
(item.action for item in judged), key=SEVERITY.__getitem__, default="withhold"
)
return EscalatedAnswer(final, answer.text, tuple(judged))
def main() -> None:
client = InveraClient(
os.environ["INVERA_BASE_URL"],
# Only when the Engine deployment requires an API key.
api_key=os.environ.get("INVERA_API_KEY"),
)
chunks = [ # in your application: the top results of your retriever
"The Vellmar ferry line opened in 1931 and connects Ostrand with the island of Kessel.",
"The Kessel lighthouse was designed by the engineer Mira Talbot and first lit in 1874.",
"Kessel island has about 2,300 residents, most of whom work in fishing.",
]
# A prompt that lets the model add its own knowledge, so that the demo produces claims the
# chunks do not support. The first question gets one; the second only repeats the context.
prompt = "Answer the question in two sentences. Use the context passages and your own knowledge."
judge = invera_judge(client)
for question in (
"Who was the first person to walk on the Moon, and how many people live on Kessel island?",
"Is it true that the engineer Mira Talbot designed the Kessel lighthouse? "
"Answer yes or no and repeat the claim.",
):
answer = answer_with_escalation(client, question, chunks, judge, system_prompt=prompt)
print(f"{question}\n[{answer.action}] {answer.text}")
for item in answer.claims:
judged = f"judge={item.judge.raw.strip()!r}" if item.judge else "not escalated"
print(
f" {item.attribution_action} -> {item.action:<8} "
f"grounding={item.claim.grounding_score:.2f} "
f"chunks={chunk_share(item.claim, len(chunks)):.2f} {judged} {item.claim.text!r}"
)
if __name__ == "__main__":
main()
The demo uses a prompt that lets the model add its own knowledge. Against a Qwen3.8-27B Engine it printed:
Who was the first person to walk on the Moon, and how many people live on Kessel island?
[withhold] Neil Armstrong was the first person to walk on the Moon. Kessel island has about 2,300 residents.
show -> withhold grounding=0.76 chunks=0.04 judge='NOT_FOUND' 'Neil Armstrong was the first person to walk on the Moon.'
show -> show grounding=0.81 chunks=0.67 not escalated 'Kessel island has about 2,300 residents.'
Is it true that the engineer Mira Talbot designed the Kessel lighthouse? Answer yes or no and repeat the claim.
[show] Yes, the claim is true. The engineer Mira Talbot designed the Kessel lighthouse.
show -> show grounding=0.82 chunks=0.06 judge='SUPPORTED' 'Yes, the claim is true.'
show -> show grounding=0.84 chunks=0.33 judge='SUPPORTED' 'The engineer Mira Talbot designed the Kessel lighthouse.'
- The Armstrong claim scores 0.76 grounding, so the first recipe alone would show it. Only 4% of its attribution comes from the chunks, though, so it is escalated. The judge finds it in no chunk and confirms the hallucination. The claim is true, but it is not supported by your documents, which is what a RAG answer must be.
- The second answer mostly repeats the question, so its chunk shares are low too. The judge finds both claims in chunk 2 and rejects the hallucination, so the answer is shown.
How many claims reach the judge
We ran the six questions of the SDK's attribution-ranking fixture
(sdk/test-fixtures/attribution-ranking.json) against a Qwen3.8-27B Engine,
with the invera judge. Three questions are answered by a chunk. Three ask
something no chunk mentions (the Moon landing, the capital of France, the
boiling point of water). Each ran with the first recipe's strict prompt and
with a prompt that allows model knowledge:
| Prompt | Questions | Claims | Sent to the judge | Judge confirmed | Judge rejected |
|---|---|---|---|---|---|
| strict | answered by a chunk | 3 | 0 | – | – |
| strict | not in the chunks | 3 | 3 (all "I do not know.") | 3 | 0 |
| permissive | answered by a chunk | 6 | 1 | 0 | 1 |
| permissive | not in the chunks | 6 | 5 | 4 | 1 |
| total | 18 | 9 | 7 | 2 |
What the numbers show:
- For answerable questions, the judge is hardly needed. Only 1 of 9 claims went to the judge. It was a correct claim with a chunk share of 0.47, and the judge rejected the suspicion. Judging every claim would have cost 9 calls for the same result.
- Attribution finds model knowledge that grounding misses. The four knowledge claims (Armstrong, Apollo 11, Paris, 100 °C) had grounding scores of 0.64–0.78. The first recipe alone would have shown three of them and flagged one. Their chunk shares were 0.02–0.15, so all four went to the judge, which confirmed them.
- The judge prevents false alarms. Two claims with a low chunk share were correct: the refund claim above, and a sentence stating that the passages do not cover the question. Attribution alone would have withheld both; the judge showed them.
- Refusals cost judge calls. "I do not know." draws on the system prompt, not on a chunk, so it is escalated and withheld. The outcome is the same as showing it (no answer), but each refusal costs a call. If refusals are common in your application, detect them before the judge.
Six questions are not a benchmark. Run the same count on your own data before you rely on these numbers.
Things to consider
- Cost and latency. The judge runs after the stream has ended, once per
escalated claim, and the calls can run in parallel. Raise
MIN_CHUNK_SHAREto escalate more claims; lower it to save calls. - Calibrate
MIN_CHUNK_SHAREtogether with the thresholds of the first recipe on your own labelled questions (see Choose thresholds on your own data). In our runs, claims from model knowledge had a share of 0.02–0.15. Most claims built from a chunk had 0.6–0.9, but correct claims that repeat the question went as low as 0.06: these are the cases the judge resolves. - The judge can be wrong too. Log every escalated claim together with the judge's reply, and review a sample. An unclear reply never overrides attribution.
- Wrong chunks stay wrong. Like attribution, the judge checks against
your chunks. If a chunk is wrong, a claim that repeats it is
SUPPORTED. - Shown claims are not judged. A claim with a high chunk share that combines chunk facts in a wrong way is shown without a judge call. If that matters for you, judge every claim through the verifier hook described in Hallucination detection.