Skip to main content
Version: 0.7.2

Escalate Unsupported Claims to an LLM Judge

This recipe extends Hallucination detection. Attribution detects the suspicious claims while the answer streams: it measures which input each generated word came from. An LLM judge then gives a second opinion on those claims only. It either confirms the hallucination, and the claim is withheld, or rejects it, and the claim is shown. The judge's job is to prevent false alarms, not to find hallucinations.

The examples on this page are covered by the SDK's unit tests against a stubbed stream and judge.

Why not just judge every claim?​

A judge without attribution can check claims too. What it cannot do:

Attribution (invera)LLM judge alone
Which chunk and which words a claim comes fromyes, per word: citations, highlighting, audit trailno, only a verdict
When it is availablewhile the answer streamsafter the answer, one model call per claim
What it measureswhat the model actually used to produce each wordwhat a second model believes about the claim
Reproducibleyes: greedy decoding, no samplingdepends on the judge model; it can hallucinate too
Cost per claimnone beyond the attributed streamone LLM call

Attribution alone has one weakness: it is strict. A correct claim that mostly repeats the question draws little attribution from the chunks and looks like model knowledge. Withholding it is a false alarm. The judge resolves exactly these cases, and it is only called for claims attribution flags.

When a claim goes to the judge​

A claim is escalated in two cases:

  1. Attribution would not show it. Its action from the first recipe is flag or withhold (grounding below SHOW_AT).
  2. It is grounded, but hardly in the chunks. Grounding counts attribution to any message, including the question and the system prompt. A claim that the model takes from its own knowledge often scores high anyway, because it repeats the words of the question. The chunk share catches this: the part of the claim's attribution score that comes from the chunk messages. A claim below MIN_CHUNK_SHARE (0.5) is escalated.

Every other claim is shown without a judge call (see How many claims reach the judge for measured numbers).

Judge replyMeaningClaim action
SUPPORTEDhallucination rejectedshow
CONTRADICTEDhallucination confirmed: the chunks say otherwisewithhold
NOT_FOUNDhallucination confirmed: the chunks do not say thiswithhold
anything elseunclearkeep the action from attribution

The answer gets the strictest action of its claims, as in the first recipe.

The judge​

The judge receives the question, the claim, and all retrieved chunks, not only the cited ones: an unsupported claim often cites no chunk at all. The question matters as well. Claims such as "Yes, that is correct." can only be judged as part of the answer to it.

inveraJudge / invera_judge uses the model behind your invera Engine, so the example needs no second provider. The judge is a plain function (question, claim, chunks) → { judgement, raw }. To use another LLM, write a function that sends the same prompt (JUDGE_PROMPT) to that LLM and passes its reply to parseJudgement / parse_judgement.

The complete example​

The example imports the first recipe, so put both files in the same directory. Set INVERA_BASE_URL (and INVERA_API_KEY if your deployment needs a key), then run npx tsx llm-judge-escalation.ts or python llm_judge_escalation.py.

llm-judge-escalation.ts
// LLM judge escalation: ask a judge only about the claims attribution does not support.
import { InveraClient, type ClaimEvidence } from "@layerwise-ai/invera-sdk";
import { pathToFileURL } from "node:url";

// The first recipe, in the same directory: hallucination-detection.ts
import { SEVERITY, SYSTEM_PROMPT, answerWithEvidence, claimAction, type Action } from "./hallucination-detection.js";

// A claim that draws less than this share of its attribution from the retrieved chunks is
// escalated even when its grounding is high: it then rests on the question or the instructions,
// which is how model knowledge looks. Calibrate it on your own data, like SHOW_AT.
export const MIN_CHUNK_SHARE = 0.5;

export type Judgement = "supported" | "contradicted" | "not_found";

/** The judge's answer for one claim; `judgement` is null when it was not understood. */
export interface JudgeResult {
judgement: Judgement | null;
raw: string;
}

/** Judges one claim of the answer to `question` against the retrieved chunks. */
export type Judge = (question: string, claim: string, chunks: readonly string[]) => Promise<JudgeResult>;

export const JUDGE_PROMPT =
"You check whether a claim from an answer is supported by numbered context passages. " +
"Read the claim as part of the answer to the question; it may refer to the question. " +
"Reply with exactly one word: SUPPORTED if the passages state the claim, " +
"CONTRADICTED if the passages state something that conflicts with it, " +
"NOT_FOUND if the passages do not mention it.";

/** One claim with the action from attribution and, if it was escalated, the judge's view. */
export interface JudgedClaim {
claim: ClaimEvidence;
attributionAction: Action;
judge: JudgeResult | null;
action: Action;
/** True if the judge confirmed a hallucination, false if it rejected one, null if not judged. */
hallucination: boolean | null;
}

/** The streamed answer, the final action, and every claim with its decision. */
export interface EscalatedAnswer {
action: Action;
text: string;
claims: JudgedClaim[];
}

/** Read the first word of the judge's reply as a judgement. */
export function parseJudgement(text: string): Judgement | null {
const word = (text.trim().split(/\s+/)[0] ?? "").replace(/^[.:*`'"]+|[.:*`'"]+$/g, "").toUpperCase();
const judgements: Record<string, Judgement> = {
SUPPORTED: "supported",
CONTRADICTED: "contradicted",
NOT_FOUND: "not_found",
};
return judgements[word] ?? null;
}

/** Use the model behind the invera Engine as the judge. Any other LLM works the same way. */
export function inveraJudge(client: Pick<InveraClient, "chat">): Judge {
return async (question, claim, chunks) => {
const passages = chunks.map((chunk, index) => `[${index + 1}] ${chunk}`).join("\n");
let reply = "";
for await (const chunk of client.chat.completions.stream({
messages: [
{ role: "system", content: JUDGE_PROMPT },
{ role: "user", content: `Context passages:\n${passages}\n\nQuestion: ${question}\nClaim: ${claim}` },
],
maxTokens: 8,
topKAttributions: 1,
})) {
if (chunk.type === "text") reply += chunk.content ?? "";
}
return { judgement: parseJudgement(reply), raw: reply };
};
}

/** Return the share of the claim's attribution that comes from the retrieved chunks. */
export function chunkShare(claim: ClaimEvidence, chunkCount: number): number {
let total = 0;
let fromChunks = 0;
for (const source of claim.sources) {
total += source.attributionScore;
if (source.sourceType === "message" && source.sourceId !== null && source.sourceId >= 1 && source.sourceId <= chunkCount) {
fromChunks += source.attributionScore;
}
}
return total > 0 ? fromChunks / total : 0;
}

/** Escalate claims attribution would not show, and shown claims that barely use the chunks. */
export function needsJudge(claim: ClaimEvidence, chunkCount: number): boolean {
return claimAction(claim) !== "show" || chunkShare(claim, chunkCount) < MIN_CHUNK_SHARE;
}

/** Supported claims are shown; confirmed hallucinations are withheld; unclear ones stay. */
export function judgedAction(attributionAction: Action, result: JudgeResult): Action {
if (result.judgement === "supported") return "show";
if (result.judgement === "contradicted" || result.judgement === "not_found") return "withhold";
return attributionAction;
}

/** Answer with evidence, then let the judge decide only the claims attribution cannot. */
export async function answerWithEscalation(
client: Pick<InveraClient, "chat">,
question: string,
chunks: readonly string[],
judge: Judge,
systemPrompt: string = SYSTEM_PROMPT,
): Promise<EscalatedAnswer> {
const answer = await answerWithEvidence(client, question, chunks, undefined, systemPrompt);
const claims: JudgedClaim[] = [];
for (const claim of answer.claims) {
const action = claimAction(claim);
if (!needsJudge(claim, chunks.length)) {
claims.push({ claim, attributionAction: action, judge: null, action, hallucination: null });
continue;
}
// An unsupported claim may cite no chunk at all, so the judge sees every chunk.
const result = await judge(question, claim.text, chunks);
claims.push({
claim,
attributionAction: action,
judge: result,
action: judgedAction(action, result),
hallucination: result.judgement === null ? null : result.judgement !== "supported",
});
}
const final = claims.reduce<Action>(
(worst, item) => (SEVERITY[item.action] > SEVERITY[worst] ? item.action : worst),
claims.length > 0 ? "show" : "withhold",
);
return { action: final, text: answer.text, claims };
}

export async function main(): Promise<void> {
const client = new InveraClient({
baseUrl: process.env.INVERA_BASE_URL!,
// Only when the Engine deployment requires an API key.
apiKey: process.env.INVERA_API_KEY,
});
const chunks = [ // in your application: the top results of your retriever
"The Vellmar ferry line opened in 1931 and connects Ostrand with the island of Kessel.",
"The Kessel lighthouse was designed by the engineer Mira Talbot and first lit in 1874.",
"Kessel island has about 2,300 residents, most of whom work in fishing.",
];
// A prompt that lets the model add its own knowledge, so that the demo produces claims the
// chunks do not support. The first question gets one; the second only repeats the context.
const prompt = "Answer the question in two sentences. Use the context passages and your own knowledge.";
const judge = inveraJudge(client);
for (const question of [
"Who was the first person to walk on the Moon, and how many people live on Kessel island?",
"Is it true that the engineer Mira Talbot designed the Kessel lighthouse? Answer yes or no and repeat the claim.",
]) {
const answer = await answerWithEscalation(client, question, chunks, judge, prompt);
console.log(`${question}\n[${answer.action}] ${answer.text}`);
for (const item of answer.claims) {
const judged = item.judge ? `judge=${JSON.stringify(item.judge.raw.trim())}` : "not escalated";
console.log(
` ${item.attributionAction} -> ${item.action.padEnd(8)} ` +
`grounding=${item.claim.groundingScore.toFixed(2)} ` +
`chunks=${chunkShare(item.claim, chunks.length).toFixed(2)} ${judged} ${JSON.stringify(item.claim.text)}`,
);
}
}
}

// Run with Node.js (for example `npx tsx llm-judge-escalation.ts`).
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) await main();

The demo uses a prompt that lets the model add its own knowledge. Against a Qwen3.8-27B Engine it printed:

Who was the first person to walk on the Moon, and how many people live on Kessel island?
[withhold] Neil Armstrong was the first person to walk on the Moon. Kessel island has about 2,300 residents.
show -> withhold grounding=0.76 chunks=0.04 judge='NOT_FOUND' 'Neil Armstrong was the first person to walk on the Moon.'
show -> show grounding=0.81 chunks=0.67 not escalated 'Kessel island has about 2,300 residents.'
Is it true that the engineer Mira Talbot designed the Kessel lighthouse? Answer yes or no and repeat the claim.
[show] Yes, the claim is true. The engineer Mira Talbot designed the Kessel lighthouse.
show -> show grounding=0.82 chunks=0.06 judge='SUPPORTED' 'Yes, the claim is true.'
show -> show grounding=0.84 chunks=0.33 judge='SUPPORTED' 'The engineer Mira Talbot designed the Kessel lighthouse.'
  • The Armstrong claim scores 0.76 grounding, so the first recipe alone would show it. Only 4% of its attribution comes from the chunks, though, so it is escalated. The judge finds it in no chunk and confirms the hallucination. The claim is true, but it is not supported by your documents, which is what a RAG answer must be.
  • The second answer mostly repeats the question, so its chunk shares are low too. The judge finds both claims in chunk 2 and rejects the hallucination, so the answer is shown.

How many claims reach the judge​

We ran the six questions of the SDK's attribution-ranking fixture (sdk/test-fixtures/attribution-ranking.json) against a Qwen3.8-27B Engine, with the invera judge. Three questions are answered by a chunk. Three ask something no chunk mentions (the Moon landing, the capital of France, the boiling point of water). Each ran with the first recipe's strict prompt and with a prompt that allows model knowledge:

PromptQuestionsClaimsSent to the judgeJudge confirmedJudge rejected
strictanswered by a chunk30––
strictnot in the chunks33 (all "I do not know.")30
permissiveanswered by a chunk6101
permissivenot in the chunks6541
total18972

What the numbers show:

  • For answerable questions, the judge is hardly needed. Only 1 of 9 claims went to the judge. It was a correct claim with a chunk share of 0.47, and the judge rejected the suspicion. Judging every claim would have cost 9 calls for the same result.
  • Attribution finds model knowledge that grounding misses. The four knowledge claims (Armstrong, Apollo 11, Paris, 100 °C) had grounding scores of 0.64–0.78. The first recipe alone would have shown three of them and flagged one. Their chunk shares were 0.02–0.15, so all four went to the judge, which confirmed them.
  • The judge prevents false alarms. Two claims with a low chunk share were correct: the refund claim above, and a sentence stating that the passages do not cover the question. Attribution alone would have withheld both; the judge showed them.
  • Refusals cost judge calls. "I do not know." draws on the system prompt, not on a chunk, so it is escalated and withheld. The outcome is the same as showing it (no answer), but each refusal costs a call. If refusals are common in your application, detect them before the judge.

Six questions are not a benchmark. Run the same count on your own data before you rely on these numbers.

Things to consider​

  • Cost and latency. The judge runs after the stream has ended, once per escalated claim, and the calls can run in parallel. Raise MIN_CHUNK_SHARE to escalate more claims; lower it to save calls.
  • Calibrate MIN_CHUNK_SHARE together with the thresholds of the first recipe on your own labelled questions (see Choose thresholds on your own data). In our runs, claims from model knowledge had a share of 0.02–0.15. Most claims built from a chunk had 0.6–0.9, but correct claims that repeat the question went as low as 0.06: these are the cases the judge resolves.
  • The judge can be wrong too. Log every escalated claim together with the judge's reply, and review a sample. An unclear reply never overrides attribution.
  • Wrong chunks stay wrong. Like attribution, the judge checks against your chunks. If a chunk is wrong, a claim that repeats it is SUPPORTED.
  • Shown claims are not judged. A claim with a high chunk share that combines chunk facts in a wrong way is shown without a judge call. If that matters for you, judge every claim through the verifier hook described in Hallucination detection.