Hallucination

What it means

A hallucination is output that sounds authoritative but is not grounded in truth or in the provided context. It comes in a few forms - inventing facts, citing sources that do not exist, contradicting the documents given to it, or fabricating details to fill a gap. Hallucinations happen because a language model predicts plausible text rather than retrieving verified facts, so when it lacks the right information it produces something that merely reads well. The danger is that fluent, confident phrasing makes a wrong answer look just as trustworthy as a right one.

Why it matters

Hallucination is the failure mode that undermines trust in LLM applications, especially in domains like healthcare, law, and finance where a fabricated fact carries real cost. It is also the hardest failure to spot, because nothing about the surface of the text signals that it is wrong. For retrieval-augmented systems, hallucination is the gap between what the context supported and what the model actually claimed. Reducing and measuring it is a primary goal of LLM evaluation, and rising hallucination rates are a common reason a model swap or prompt change gets reverted.

In practice

You measure hallucination by checking outputs against ground truth or against the retrieved context. Faithfulness scoring breaks a response into individual claims and verifies each one against the source documents, which surfaces unsupported statements. LLM-as-a-judge is also used to flag contradictions. Platforms like Galileo and DeepEval offer hallucination and groundedness scorers you can run over an eval dataset or live traces. See the guides on measuring hallucination and evaluating RAG for the concrete metrics and how to wire them in.