Skip to main content
Version: 0.7.3

Escalate Unsupported Claims to an LLM Judge

This recipe extends Hallucination detection. Attribution detects the suspicious claims while the answer streams: it measures which input each generated word came from. An LLM judge then gives a second opinion on those claims only. It either confirms the hallucination, and the claim is withheld, or rejects it, and the claim is shown. The judge's job is to prevent false alarms, not to find hallucinations.

The examples on this page are covered by the SDK's unit tests against a stubbed stream and judge.

Why not just judge every claim?​

A judge without attribution can check claims too. What it cannot do:

Attribution (invera)LLM judge alone
Which chunk and which words a claim comes fromyes, per word: citations, highlighting, audit trailno, only a verdict
When it is availablewhile the answer streamsafter the answer, one model call per claim
What it measureswhat the model actually used to produce each wordwhat a second model believes about the claim
Reproducibleyes: greedy decoding, no samplingdepends on the judge model; it can hallucinate too
Cost per claimnone beyond the attributed streamone LLM call

Attribution alone has one weakness: it is strict. A correct claim that mostly repeats the question draws little attribution from the context and looks like model knowledge. Withholding it is a false alarm. The judge resolves exactly these cases, and it is only called for claims attribution flags.

When a claim goes to the judge​

A claim is escalated when attribution alone would not show it: its action from the first recipe is flag or withhold (grounding below SHOW_AT). Grounding does not count the words a claim repeats from the question, so a claim from model knowledge scores low even when it echoes the question.

Every other claim is shown without a judge call (see How many claims reach the judge for measured numbers).

Judge replyMeaningClaim action
SUPPORTEDhallucination rejectedshow
CONTRADICTEDhallucination confirmed: the chunks say otherwisewithhold
NOT_FOUNDhallucination confirmed: the chunks do not say thiswithhold
anything elseunclearkeep the action from attribution

The answer gets the strictest action of its claims, as in the first recipe.

The judge​

The judge receives the question, the claim, and all retrieved chunks, not only the cited ones: an unsupported claim often cites no chunk at all. The question matters as well. Claims such as "Yes, that is correct." can only be judged as part of the answer to it.

inveraJudge / invera_judge uses the model behind your invera Engine, so the example needs no second provider. The judge is a plain function (question, claim, chunks) → { judgement, raw }. To use another LLM, write a function that sends the same prompt (JUDGE_PROMPT) to that LLM and passes its reply to parseJudgement / parse_judgement.

The complete example​

The example imports the first recipe, so put both files in the same directory. Set INVERA_BASE_URL (and INVERA_API_KEY if your deployment needs a key), then run npx tsx llm-judge-escalation.ts or python llm_judge_escalation.py.

llm-judge-escalation.ts
// LLM judge escalation: ask a judge only about the claims attribution does not support.
import { InveraClient, type ClaimEvidence } from "@layerwise-ai/invera-sdk";
import { pathToFileURL } from "node:url";

// The first recipe, in the same directory: hallucination-detection.ts
import { SEVERITY, SYSTEM_PROMPT, answerWithEvidence, claimAction, type Action } from "./hallucination-detection.js";

export type Judgement = "supported" | "contradicted" | "not_found";

/** The judge's answer for one claim; `judgement` is null when it was not understood. */
export interface JudgeResult {
judgement: Judgement | null;
raw: string;
}

/** Judges one claim of the answer to `question` against the retrieved chunks. */
export type Judge = (question: string, claim: string, chunks: readonly string[]) => Promise<JudgeResult>;

export const JUDGE_PROMPT =
"You check whether a claim from an answer is supported by numbered context passages. " +
"Read the claim as part of the answer to the question; it may refer to the question. " +
"Reply with exactly one word: SUPPORTED if the passages state the claim, " +
"CONTRADICTED if the passages state something that conflicts with it, " +
"NOT_FOUND if the passages do not mention it.";

/** One claim with the action from attribution and, if it was escalated, the judge's view. */
export interface JudgedClaim {
claim: ClaimEvidence;
attributionAction: Action;
judge: JudgeResult | null;
action: Action;
/** True if the judge confirmed a hallucination, false if it rejected one, null if not judged. */
hallucination: boolean | null;
}

/** The streamed answer, the final action, and every claim with its decision. */
export interface EscalatedAnswer {
action: Action;
text: string;
claims: JudgedClaim[];
}

/** Read the first word of the judge's reply as a judgement. */
export function parseJudgement(text: string): Judgement | null {
const word = (text.trim().split(/\s+/)[0] ?? "").replace(/^[.:*`'"]+|[.:*`'"]+$/g, "").toUpperCase();
const judgements: Record<string, Judgement> = {
SUPPORTED: "supported",
CONTRADICTED: "contradicted",
NOT_FOUND: "not_found",
};
return judgements[word] ?? null;
}

/** Use the model behind the invera Engine as the judge. Any other LLM works the same way. */
export function inveraJudge(client: Pick<InveraClient, "chat">): Judge {
return async (question, claim, chunks) => {
const passages = chunks.map((chunk, index) => `[${index + 1}] ${chunk}`).join("\n");
let reply = "";
for await (const chunk of client.chat.completions.stream({
messages: [
{ role: "system", content: JUDGE_PROMPT },
{ role: "user", content: `Context passages:\n${passages}\n\nQuestion: ${question}\nClaim: ${claim}` },
],
maxTokens: 8,
topKAttributions: 1,
})) {
if (chunk.type === "text") reply += chunk.content ?? "";
}
return { judgement: parseJudgement(reply), raw: reply };
};
}

/** Escalate every claim that attribution alone would not show. */
export function needsJudge(claim: ClaimEvidence): boolean {
return claimAction(claim) !== "show";
}

/** Supported claims are shown; confirmed hallucinations are withheld; unclear ones stay. */
export function judgedAction(attributionAction: Action, result: JudgeResult): Action {
if (result.judgement === "supported") return "show";
if (result.judgement === "contradicted" || result.judgement === "not_found") return "withhold";
return attributionAction;
}

/** Answer with evidence, then let the judge decide only the claims attribution cannot. */
export async function answerWithEscalation(
client: Pick<InveraClient, "chat">,
question: string,
chunks: readonly string[],
judge: Judge,
systemPrompt: string = SYSTEM_PROMPT,
): Promise<EscalatedAnswer> {
const answer = await answerWithEvidence(client, question, chunks, undefined, systemPrompt);
const claims: JudgedClaim[] = [];
for (const claim of answer.claims) {
const action = claimAction(claim);
if (!needsJudge(claim)) {
claims.push({ claim, attributionAction: action, judge: null, action, hallucination: null });
continue;
}
// An unsupported claim may cite no chunk at all, so the judge sees every chunk.
const result = await judge(question, claim.text, chunks);
claims.push({
claim,
attributionAction: action,
judge: result,
action: judgedAction(action, result),
hallucination: result.judgement === null ? null : result.judgement !== "supported",
});
}
const final = claims.reduce<Action>(
(worst, item) => (SEVERITY[item.action] > SEVERITY[worst] ? item.action : worst),
claims.length > 0 ? "show" : "withhold",
);
return { action: final, text: answer.text, claims };
}

export async function main(): Promise<void> {
const client = new InveraClient({
baseUrl: process.env.INVERA_BASE_URL!,
// Only when the Engine deployment requires an API key.
apiKey: process.env.INVERA_API_KEY,
});
const chunks = [ // in your application: the top results of your retriever
"The Vellmar ferry line opened in 1931 and connects Ostrand with the island of Kessel.",
"The Kessel lighthouse was designed by the engineer Mira Talbot and first lit in 1874.",
"Kessel island has about 2,300 residents, most of whom work in fishing.",
];
// A prompt that lets the model add its own knowledge, so that the demo produces claims the
// chunks do not support. The first question gets one; the second only repeats the context.
const prompt = "Answer the question in two sentences. Use the context passages and your own knowledge.";
const judge = inveraJudge(client);
for (const question of [
"Who was the first person to walk on the Moon, and how many people live on Kessel island?",
"Is it true that the engineer Mira Talbot designed the Kessel lighthouse? Answer yes or no and repeat the claim.",
]) {
const answer = await answerWithEscalation(client, question, chunks, judge, prompt);
console.log(`${question}\n[${answer.action}] ${answer.text}`);
for (const item of answer.claims) {
const judged = item.judge ? `judge=${JSON.stringify(item.judge.raw.trim())}` : "not escalated";
console.log(
` ${item.attributionAction} -> ${item.action.padEnd(8)} ` +
`grounding=${item.claim.groundingScore.toFixed(2)} ${judged} ${JSON.stringify(item.claim.text)}`,
);
}
}
}

// Run with Node.js (for example `npx tsx llm-judge-escalation.ts`).
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) await main();

The demo uses a prompt that lets the model add its own knowledge. Against a Qwen3.8-27B Engine it printed:

Who was the first person to walk on the Moon, and how many people live on Kessel island?
[withhold] Neil Armstrong was the first person to walk on the Moon. Kessel island has about 2,300 residents.
flag -> withhold grounding=0.52 judge='NOT_FOUND' 'Neil Armstrong was the first person to walk on the Moon.'
show -> show grounding=0.80 not escalated 'Kessel island has about 2,300 residents.'
Is it true that the engineer Mira Talbot designed the Kessel lighthouse? Answer yes or no and repeat the claim.
[show] Yes, the claim is true. The engineer Mira Talbot designed the Kessel lighthouse.
flag -> show grounding=0.64 judge='SUPPORTED' 'Yes, the claim is true.'
show -> show grounding=0.78 not escalated 'The engineer Mira Talbot designed the Kessel lighthouse.'
  • The Armstrong claim echoes the question, but the question does not count as grounding, so it scores 0.52 and is flagged. The judge finds it in no chunk and confirms the hallucination. The claim is true, but it is not supported by your documents, which is what a RAG answer must be.
  • "Yes, the claim is true." only refers to the question, so it is flagged as well. The judge reads it as part of the answer, finds it in chunk 2, and rejects the hallucination, so the answer is shown.

How many claims reach the judge​

We ran six questions against a Qwen3.8-27B Engine, with the invera judge. Three are the questions of the SDK's attribution-ranking fixture (sdk/test-fixtures/attribution-ranking.json): the lighthouse designer, a battery capacity, and a refund window, each answered by a chunk. Three ask something no chunk mentions (the Moon landing, the capital of France, the boiling point of water). Each ran with the first recipe's strict prompt and with a prompt that allows model knowledge. "Judge: not in chunks" counts CONTRADICTED and NOT_FOUND replies, "Judge: supported" counts SUPPORTED:

PromptQuestionsClaimsSent to the judgeJudge: not in chunksJudge: supported
strictanswered by a chunk30––
strictnot in the chunks30 (all "I do not know.")––
permissiveanswered by a chunk60––
permissivenot in the chunks6550
total18550

What the numbers show:

  • For answerable questions, the judge is not needed. All 9 claims scored 0.72 or higher and were shown without a judge call. Judging every claim would have cost 9 calls for the same result.
  • Attribution finds model knowledge. The four knowledge claims (Armstrong, Apollo 11, Paris, 100 °C) scored 0.43–0.64, so all four were flagged and sent to the judge, which confirmed them.
  • The judge is the last word on flagged claims. The fifth escalated claim said that the boiling point does not depend on the passages. The judge replied NOT_FOUND, because no passage says so. The answer was withheld anyway, because its first claim was a confirmed hallucination.
  • Refusals cost nothing. "I do not know." follows the system prompt, so it stays grounded (0.87) and is shown without a judge call.

Six questions are not a benchmark. Run the same count on your own data before you rely on these numbers.

Things to consider​

  • Cost and latency. The judge runs after the stream has ended, once per escalated claim, and the calls can run in parallel. Raise SHOW_AT to escalate more claims; lower it to save calls.
  • Calibrate SHOW_AT on your own labelled questions (see Choose thresholds on your own data). In our runs, claims from model knowledge scored 0.43–0.64 and claims built from a chunk 0.72–0.83. Correct claims that only refer to the question can fall below SHOW_AT too: these are the cases the judge resolves.
  • The judge can be wrong too. Log every escalated claim together with the judge's reply, and review a sample. An unclear reply never overrides attribution.
  • Wrong chunks stay wrong. Like attribution, the judge checks against your chunks. If a chunk is wrong, a claim that repeats it is SUPPORTED.
  • Shown claims are not judged. A grounded claim that combines chunk facts in a wrong way is shown without a judge call. If that matters for you, judge every claim through the verifier hook described in Hallucination detection.