LLM Hallucination Mitigation: RAG, Confidence, Self-Critique

Hallucination is an architecture problem, not a model quality problem

When a production system starts hallucinating, the default fix is a model upgrade: swap Claude Sonnet for Opus, or step up from GPT-5.4 to GPT-5.5. The intuition isn't wrong — flagship models do hallucinate less. GPT-5.5 scores roughly 12 percentage points higher than GPT-5.4-mini on TruthfulQA, and Opus 4.8 handles closed-book QA noticeably better than Haiku 4.5.

But this approach has a hard ceiling. Every production LLM in use today is a next-token predictor. The model learned which token sequences look plausible, not whether a claimed fact exists in the real world. Hallucination is a structural property of autoregressive language models. It's not a version bug. It doesn't disappear as parameter count grows.

The engineering framing is different: instead of making the model guess better, reduce how often it needs to guess — and catch the wrong guesses early. Here's a three-layer stack that does exactly that.

---

Layer 1 — RAG grounding: lock knowledge sources into context

Retrieval-Augmented Generation is the most mature LLM hallucination mitigation technique available. The core idea is simple: instead of asking the model to recall facts from training weights, inject the relevant documents directly into the prompt at inference time. The task shifts from "remember this" to "read and summarize this." That's a much easier problem.

Three implementation details that matter more than most teams realize:

Chunking strategy drives retrieval quality. Fixed 512-token chunks are the default, but they perform poorly on structured documents like contracts or API references. A better pattern is semantic boundary chunking with parent-child chunks: use small chunks to locate the right passage, then expand to the parent chunk when building the context window. Precision at retrieval, completeness in context.

Hybrid retrieval beats pure vector search. Dense vector search struggles with proper nouns, version numbers, and exact figures. Combining BM25 with vector search via Reciprocal Rank Fusion (RRF) typically lifts Recall@5 by 8–15 percentage points over vector-only retrieval.

Force citations at the prompt level. Require the model to answer only using content tagged with a [source_id] and to append a reference list. This converts hallucination verification from "no way to check" into "fully auditable."

A minimal system prompt that works:

You are a factual assistant. Answer ONLY based on the retrieved context below.
If the context does not contain enough information, say "I don't have enough information."
Do NOT use your parametric knowledge for factual claims.
Cite sources as [doc_id] inline.

Retrieved context:
{context}

This instruction, paired with solid retrieval, routinely brings hallucination rates in internal knowledge-base QA from 30%+ down to under 5%.

---

Layer 2 — Confidence calibration: make the model tell you when it's guessing

RAG handles situations where external knowledge is available to retrieve. But models still reason from internal knowledge in plenty of scenarios. That's where confidence calibration comes in.

The simplest technique is verbalized confidence: ask the model to append [confidence: high/medium/low] and a brief uncertainty note to its answer. It sounds crude, but research on RLHF-trained models (GPT-5 series, Claude 4 series) shows a meaningful positive correlation between verbalized confidence and actual accuracy. Low-confidence answers really do fail more often.

For higher-stakes use cases, multi-sample consistency checking is more rigorous:

import asyncio

async def sample_n(prompt, model, n=5):
    tasks = [call_llm(prompt, model) for _ in range(n)]
    return await asyncio.gather(*tasks)

responses = await sample_n(user_query, model="gpt-5.4")
# Semantic clustering on key factual fields
consistency_score = semantic_agreement(responses)
if consistency_score < 0.7:
    flag_for_human_review(user_query, responses)

Run five samples and compute semantic agreement on the critical fields. A consistency score below 0.7 is a reliable signal that the model is uncertain — route the query to human review or return a "cannot confirm" fallback. The cost is N times the inference spend, so this is best reserved for high-risk fields: dollar amounts, dates, regulatory citations.

---

Layer 3 — Self-critique loops: have the model audit its own output

Self-critique is a two-phase pattern. The first call produces a draft. The second call plays the role of a skeptical reviewer: identify factual errors, unsupported claims, and logical contradictions in the draft. A third call synthesizes the critique and produces a revised answer.

Why does this work? The same model activates different attention patterns when generating versus reviewing. In generation mode, it defaults toward plausible completion. When explicitly prompted to find errors, it engages more skeptical reasoning. The mode switch is real, and it's exploitable.

The three-step sequence:

1. Generate: standard prompt, get draft
2. Critique: "Review the following answer for factual errors, unsupported claims, and logical inconsistencies. List each issue with a severity rating (critical/minor)." Input: draft
3. Revise: pass draft + critique together, request a revised answer with each change annotated

In high-precision tasks — legal document summarization, medical information extraction — this three-step loop improves factual accuracy by roughly 18–22% over single-pass generation, based on internal evaluations. Results vary by task.

Note that this pattern is already baked into coding-focused tools like Claude Code and Codex: generate code, run tests, read errors, patch and repeat. That's a self-critique loop with an external feedback signal.

---

Combining all three into a production pipeline

Each technique alone has blind spots. Used together, the result is better than the sum of the parts. The key design principle is risk-tiered triggering:

Layer	Technique	When to use	Cost
L1	RAG + forced citation	Knowledge base queries	Low (~100ms retrieval latency)
L2	Verbalized confidence	General QA	Near-zero (no extra calls)
L3	Multi-sample consistency	High-risk fields (amounts, dates, regulations)	High (N× inference)
L4	Self-critique loop	Long-form output, structured documents	Medium (2× calls)

Routine queries go through L1 + L2 only. Fields that touch compliance or money trigger L3. Final document generation runs L4. This keeps cost manageable while pushing overall hallucination rates into a business-acceptable range.

Model selection becomes a secondary decision inside this framework. Run a cost-efficient model like GPT-5.4-mini for L1 and L2. Reserve Opus 4.8 or GPT-5.5 for L4 revisions where accuracy is worth the spend. Model capability and engineering controls complement each other — they don't substitute.

---

For the multi-model routing this architecture requires, I use XycAi. It exposes a single OpenAI-compatible API across 200+ models including the full GPT-5 and Claude 4 lineups, starting at 14% of list price — which makes the cost math on multi-sample consistency checks a lot more reasonable. Claude Code, Codex, and Gemini CLI all connect with a single command, so switching models while debugging a self-critique loop takes seconds rather than an afternoon.