Escalate Unsupported Claims to an LLM Judge
This recipe extends Hallucination detection. Attribution detects the suspicious claims while the answer streams: it measures which input each generated word came from. An LLM judge then gives a second opinion on those claims only. It either confirms the hallucination, and the claim is withheld, or rejects it, and the claim is shown. The judge's job is to prevent false alarms, not to find hallucinations.
The examples on this page are covered by the SDK's unit tests against a stubbed stream and judge.
Why not just judge every claim?
A judge without attribution can check claims too. What it cannot do:
| Attribution (invera) | LLM judge alone | |
|---|---|---|
| Which chunk and which words a claim comes from | yes, per word: citations, highlighting, audit trail | no, only a verdict |
| When it is available | while the answer streams | after the answer, one model call per claim |
| What it measures | what the model actually used to produce each word | what a second model believes about the claim |
| Reproducible | yes: greedy decoding, no sampling | depends on the judge model; it can hallucinate too |
| Cost per claim | none beyond the attributed stream | one LLM call |
Attribution alone has one weakness: it is strict. A correct claim that mostly repeats the question draws little attribution from the context and looks like model knowledge. Withholding it is a false alarm. The judge resolves exactly these cases, and it is only called for claims attribution flags.
When a claim goes to the judge
A claim is escalated when attribution alone would not show it: its action
from the first recipe is flag or withhold (grounding below SHOW_AT).
Grounding does not count the words a claim repeats from the question, so a
claim from model knowledge scores low even when it echoes the question.
Every other claim is shown without a judge call (see How many claims reach the judge for measured numbers).
| Judge reply | Meaning | Claim action |
|---|---|---|
SUPPORTED | hallucination rejected | show |
CONTRADICTED | hallucination confirmed: the chunks say otherwise | withhold |
NOT_FOUND | hallucination confirmed: the chunks do not say this | withhold |
| anything else | unclear | keep the action from attribution |
The answer gets the strictest action of its claims, as in the first recipe.
The judge
The judge receives the question, the claim, and all retrieved chunks, not only the cited ones: an unsupported claim often cites no chunk at all. The question matters as well. Claims such as "Yes, that is correct." can only be judged as part of the answer to it.
inveraJudge / invera_judge uses the model behind your invera Engine, so
the example needs no second provider. The judge is a plain function
(question, claim, chunks) → { judgement, raw }. To use another LLM, write a
function that sends the same prompt (JUDGE_PROMPT) to that LLM and passes its
reply to parseJudgement / parse_judgement.
The complete example
The example imports the first recipe, so put both files in the same
directory. Set INVERA_BASE_URL (and INVERA_API_KEY if your deployment
needs a key), then run npx tsx llm-judge-escalation.ts or
python llm_judge_escalation.py.
- TypeScript
- Python
// LLM judge escalation: ask a judge only about the claims attribution does not support.
import { InveraClient, type ClaimEvidence } from "@layerwise-ai/invera-sdk";
import { pathToFileURL } from "node:url";
// The first recipe, in the same directory: hallucination-detection.ts
import { SEVERITY, SYSTEM_PROMPT, answerWithEvidence, claimAction, type Action } from "./hallucination-detection.js";
export type Judgement = "supported" | "contradicted" | "not_found";
/** The judge's answer for one claim; `judgement` is null when it was not understood. */
export interface JudgeResult {
judgement: Judgement | null;
raw: string;
}
/** Judges one claim of the answer to `question` against the retrieved chunks. */
export type Judge = (question: string, claim: string, chunks: readonly string[]) => Promise<JudgeResult>;
export const JUDGE_PROMPT =
"You check whether a claim from an answer is supported by numbered context passages. " +
"Read the claim as part of the answer to the question; it may refer to the question. " +
"Reply with exactly one word: SUPPORTED if the passages state the claim, " +
"CONTRADICTED if the passages state something that conflicts with it, " +
"NOT_FOUND if the passages do not mention it.";
/** One claim with the action from attribution and, if it was escalated, the judge's view. */
export interface JudgedClaim {
claim: ClaimEvidence;
attributionAction: Action;
judge: JudgeResult | null;
action: Action;
/** True if the judge confirmed a hallucination, false if it rejected one, null if not judged. */
hallucination: boolean | null;
}
/** The streamed answer, the final action, and every claim with its decision. */
export interface EscalatedAnswer {
action: Action;
text: string;
claims: JudgedClaim[];
}
/** Read the first word of the judge's reply as a judgement. */
export function parseJudgement(text: string): Judgement | null {
const word = (text.trim().split(/\s+/)[0] ?? "").replace(/^[.:*`'"]+|[.:*`'"]+$/g, "").toUpperCase();
const judgements: Record<string, Judgement> = {
SUPPORTED: "supported",
CONTRADICTED: "contradicted",
NOT_FOUND: "not_found",
};
return judgements[word] ?? null;
}
/** Use the model behind the invera Engine as the judge. Any other LLM works the same way. */
export function inveraJudge(client: Pick<InveraClient, "chat">): Judge {
return async (question, claim, chunks) => {
const passages = chunks.map((chunk, index) => `[${index + 1}] ${chunk}`).join("\n");
let reply = "";
for await (const chunk of client.chat.completions.stream({
messages: [
{ role: "system", content: JUDGE_PROMPT },
{ role: "user", content: `Context passages:\n${passages}\n\nQuestion: ${question}\nClaim: ${claim}` },
],
maxTokens: 8,
topKAttributions: 1,
})) {
if (chunk.type === "text") reply += chunk.content ?? "";
}
return { judgement: parseJudgement(reply), raw: reply };
};
}
/** Escalate every claim that attribution alone would not show. */
export function needsJudge(claim: ClaimEvidence): boolean {
return claimAction(claim) !== "show";
}
/** Supported claims are shown; confirmed hallucinations are withheld; unclear ones stay. */
export function judgedAction(attributionAction: Action, result: JudgeResult): Action {
if (result.judgement === "supported") return "show";
if (result.judgement === "contradicted" || result.judgement === "not_found") return "withhold";
return attributionAction;
}
/** Answer with evidence, then let the judge decide only the claims attribution cannot. */
export async function answerWithEscalation(
client: Pick<InveraClient, "chat">,
question: string,
chunks: readonly string[],
judge: Judge,
systemPrompt: string = SYSTEM_PROMPT,
): Promise<EscalatedAnswer> {
const answer = await answerWithEvidence(client, question, chunks, undefined, systemPrompt);
const claims: JudgedClaim[] = [];
for (const claim of answer.claims) {
const action = claimAction(claim);
if (!needsJudge(claim)) {
claims.push({ claim, attributionAction: action, judge: null, action, hallucination: null });
continue;
}
// An unsupported claim may cite no chunk at all, so the judge sees every chunk.
const result = await judge(question, claim.text, chunks);
claims.push({
claim,
attributionAction: action,
judge: result,
action: judgedAction(action, result),
hallucination: result.judgement === null ? null : result.judgement !== "supported",
});
}
const final = claims.reduce<Action>(
(worst, item) => (SEVERITY[item.action] > SEVERITY[worst] ? item.action : worst),
claims.length > 0 ? "show" : "withhold",
);
return { action: final, text: answer.text, claims };
}
export async function main(): Promise<void> {
const client = new InveraClient({
baseUrl: process.env.INVERA_BASE_URL!,
// Only when the Engine deployment requires an API key.
apiKey: process.env.INVERA_API_KEY,
});
const chunks = [ // in your application: the top results of your retriever
"The Vellmar ferry line opened in 1931 and connects Ostrand with the island of Kessel.",
"The Kessel lighthouse was designed by the engineer Mira Talbot and first lit in 1874.",
"Kessel island has about 2,300 residents, most of whom work in fishing.",
];
// A prompt that lets the model add its own knowledge, so that the demo produces claims the
// chunks do not support. The first question gets one; the second only repeats the context.
const prompt = "Answer the question in two sentences. Use the context passages and your own knowledge.";
const judge = inveraJudge(client);
for (const question of [
"Who was the first person to walk on the Moon, and how many people live on Kessel island?",
"Is it true that the engineer Mira Talbot designed the Kessel lighthouse? Answer yes or no and repeat the claim.",
]) {
const answer = await answerWithEscalation(client, question, chunks, judge, prompt);
console.log(`${question}\n[${answer.action}] ${answer.text}`);
for (const item of answer.claims) {
const judged = item.judge ? `judge=${JSON.stringify(item.judge.raw.trim())}` : "not escalated";
console.log(
` ${item.attributionAction} -> ${item.action.padEnd(8)} ` +
`grounding=${item.claim.groundingScore.toFixed(2)} ${judged} ${JSON.stringify(item.claim.text)}`,
);
}
}
}
// Run with Node.js (for example `npx tsx llm-judge-escalation.ts`).
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) await main();
"""LLM judge escalation: ask a judge only about the claims attribution does not support."""
from __future__ import annotations
import os
from collections.abc import Callable, Sequence
from dataclasses import dataclass
from typing import Literal
from invera_sdk import ChatMessage, ClaimEvidence, CompletionRequest, InveraClient
# The first recipe, in the same directory: hallucination_detection.py
from hallucination_detection import (
SEVERITY,
SYSTEM_PROMPT,
Action,
answer_with_evidence,
claim_action,
)
Judgement = Literal["supported", "contradicted", "not_found"]
Judge = Callable[[str, str, Sequence[str]], "JudgeResult"]
"""Judges one claim: ``judge(question, claim, chunks)``."""
JUDGE_PROMPT = (
"You check whether a claim from an answer is supported by numbered context passages. "
"Read the claim as part of the answer to the question; it may refer to the question. "
"Reply with exactly one word: SUPPORTED if the passages state the claim, "
"CONTRADICTED if the passages state something that conflicts with it, "
"NOT_FOUND if the passages do not mention it."
)
@dataclass(frozen=True)
class JudgeResult:
"""The judge's answer for one claim; ``judgement`` is None when it was not understood."""
judgement: Judgement | None
raw: str
@dataclass(frozen=True)
class JudgedClaim:
"""One claim with the action from attribution and, if it was escalated, the judge's view."""
claim: ClaimEvidence
attribution_action: Action
judge: JudgeResult | None
action: Action
@property
def hallucination(self) -> bool | None:
"""True if the judge confirmed it, False if the judge rejected it, None if not judged."""
if self.judge is None or self.judge.judgement is None:
return None
return self.judge.judgement != "supported"
@dataclass(frozen=True)
class EscalatedAnswer:
"""The streamed answer, the final action, and every claim with its decision."""
action: Action
text: str
claims: tuple[JudgedClaim, ...]
def parse_judgement(text: str) -> Judgement | None:
"""Read the first word of the judge's reply as a judgement."""
words = text.strip().split()
word = words[0].strip(".:*`'\"").upper() if words else ""
return {
"SUPPORTED": "supported",
"CONTRADICTED": "contradicted",
"NOT_FOUND": "not_found",
}.get(word)
def invera_judge(client: InveraClient) -> Judge:
"""Use the model behind the invera Engine as the judge. Any other LLM works the same way."""
def judge(question: str, claim: str, chunks: Sequence[str]) -> JudgeResult:
passages = "\n".join(
f"[{number}] {chunk}" for number, chunk in enumerate(chunks, start=1)
)
request = CompletionRequest(
[
ChatMessage("system", JUDGE_PROMPT),
ChatMessage("user", f"Context passages:\n{passages}\n\nQuestion: {question}\nClaim: {claim}"),
],
max_tokens=8,
top_k_attributions=1,
)
reply = "".join(
chunk.content or ""
for chunk in client.chat.completions.stream(request)
if chunk.type == "text"
)
return JudgeResult(parse_judgement(reply), reply)
return judge
def needs_judge(claim: ClaimEvidence) -> bool:
"""Escalate every claim that attribution alone would not show."""
return claim_action(claim) != "show"
def judged_action(attribution_action: Action, result: JudgeResult) -> Action:
"""Supported claims are shown; confirmed hallucinations are withheld; unclear ones stay."""
if result.judgement == "supported":
return "show"
if result.judgement in ("contradicted", "not_found"):
return "withhold"
return attribution_action
def answer_with_escalation(
client: InveraClient,
question: str,
chunks: Sequence[str],
judge: Judge,
*,
system_prompt: str = SYSTEM_PROMPT,
) -> EscalatedAnswer:
"""Answer with evidence, then let the judge decide only the claims attribution cannot."""
answer = answer_with_evidence(client, question, chunks, system_prompt=system_prompt)
judged: list[JudgedClaim] = []
for claim in answer.claims:
action = claim_action(claim)
if not needs_judge(claim):
judged.append(JudgedClaim(claim, action, None, action))
continue
# An unsupported claim may cite no chunk at all, so the judge sees every chunk.
result = judge(question, claim.text, chunks)
judged.append(JudgedClaim(claim, action, result, judged_action(action, result)))
final = max(
(item.action for item in judged), key=SEVERITY.__getitem__, default="withhold"
)
return EscalatedAnswer(final, answer.text, tuple(judged))
def main() -> None:
client = InveraClient(
os.environ["INVERA_BASE_URL"],
# Only when the Engine deployment requires an API key.
api_key=os.environ.get("INVERA_API_KEY"),
)
chunks = [ # in your application: the top results of your retriever
"The Vellmar ferry line opened in 1931 and connects Ostrand with the island of Kessel.",
"The Kessel lighthouse was designed by the engineer Mira Talbot and first lit in 1874.",
"Kessel island has about 2,300 residents, most of whom work in fishing.",
]
# A prompt that lets the model add its own knowledge, so that the demo produces claims the
# chunks do not support. The first question gets one; the second only repeats the context.
prompt = "Answer the question in two sentences. Use the context passages and your own knowledge."
judge = invera_judge(client)
for question in (
"Who was the first person to walk on the Moon, and how many people live on Kessel island?",
"Is it true that the engineer Mira Talbot designed the Kessel lighthouse? "
"Answer yes or no and repeat the claim.",
):
answer = answer_with_escalation(client, question, chunks, judge, system_prompt=prompt)
print(f"{question}\n[{answer.action}] {answer.text}")
for item in answer.claims:
judged = f"judge={item.judge.raw.strip()!r}" if item.judge else "not escalated"
print(
f" {item.attribution_action} -> {item.action:<8} "
f"grounding={item.claim.grounding_score:.2f} {judged} {item.claim.text!r}"
)
if __name__ == "__main__":
main()
The demo uses a prompt that lets the model add its own knowledge. Against a Qwen3.8-27B Engine it printed:
Who was the first person to walk on the Moon, and how many people live on Kessel island?
[withhold] Neil Armstrong was the first person to walk on the Moon. Kessel island has about 2,300 residents.
flag -> withhold grounding=0.52 judge='NOT_FOUND' 'Neil Armstrong was the first person to walk on the Moon.'
show -> show grounding=0.80 not escalated 'Kessel island has about 2,300 residents.'
Is it true that the engineer Mira Talbot designed the Kessel lighthouse? Answer yes or no and repeat the claim.
[show] Yes, the claim is true. The engineer Mira Talbot designed the Kessel lighthouse.
flag -> show grounding=0.64 judge='SUPPORTED' 'Yes, the claim is true.'
show -> show grounding=0.78 not escalated 'The engineer Mira Talbot designed the Kessel lighthouse.'
- The Armstrong claim echoes the question, but the question does not count as grounding, so it scores 0.52 and is flagged. The judge finds it in no chunk and confirms the hallucination. The claim is true, but it is not supported by your documents, which is what a RAG answer must be.
- "Yes, the claim is true." only refers to the question, so it is flagged as well. The judge reads it as part of the answer, finds it in chunk 2, and rejects the hallucination, so the answer is shown.
How many claims reach the judge
We ran six questions against a Qwen3.8-27B Engine, with the invera judge.
Three are the questions of the SDK's attribution-ranking fixture
(sdk/test-fixtures/attribution-ranking.json): the lighthouse designer, a
battery capacity, and a refund window, each answered by a chunk. Three ask
something no chunk mentions (the Moon landing, the capital of France, the
boiling point of water). Each ran with the first recipe's strict prompt and
with a prompt that allows model knowledge. "Judge: not in chunks" counts
CONTRADICTED and NOT_FOUND replies, "Judge: supported" counts SUPPORTED:
| Prompt | Questions | Claims | Sent to the judge | Judge: not in chunks | Judge: supported |
|---|---|---|---|---|---|
| strict | answered by a chunk | 3 | 0 | – | – |
| strict | not in the chunks | 3 | 0 (all "I do not know.") | – | – |
| permissive | answered by a chunk | 6 | 0 | – | – |
| permissive | not in the chunks | 6 | 5 | 5 | 0 |
| total | 18 | 5 | 5 | 0 |
What the numbers show:
- For answerable questions, the judge is not needed. All 9 claims scored 0.72 or higher and were shown without a judge call. Judging every claim would have cost 9 calls for the same result.
- Attribution finds model knowledge. The four knowledge claims (Armstrong, Apollo 11, Paris, 100 °C) scored 0.43–0.64, so all four were flagged and sent to the judge, which confirmed them.
- The judge is the last word on flagged claims. The fifth escalated claim
said that the boiling point does not depend on the passages. The judge
replied
NOT_FOUND, because no passage says so. The answer was withheld anyway, because its first claim was a confirmed hallucination. - Refusals cost nothing. "I do not know." follows the system prompt, so it stays grounded (0.87) and is shown without a judge call.
Six questions are not a benchmark. Run the same count on your own data before you rely on these numbers.
Things to consider
- Cost and latency. The judge runs after the stream has ended, once per
escalated claim, and the calls can run in parallel. Raise
SHOW_ATto escalate more claims; lower it to save calls. - Calibrate
SHOW_ATon your own labelled questions (see Choose thresholds on your own data). In our runs, claims from model knowledge scored 0.43–0.64 and claims built from a chunk 0.72–0.83. Correct claims that only refer to the question can fall belowSHOW_ATtoo: these are the cases the judge resolves. - The judge can be wrong too. Log every escalated claim together with the judge's reply, and review a sample. An unclear reply never overrides attribution.
- Wrong chunks stay wrong. Like attribution, the judge checks against
your chunks. If a chunk is wrong, a claim that repeats it is
SUPPORTED. - Shown claims are not judged. A grounded claim that combines chunk facts in a wrong way is shown without a judge call. If that matters for you, judge every claim through the verifier hook described in Hallucination detection.