Ground Truth

What it means

Ground truth is the reference output you trust as correct for a given input. In an eval dataset, each case pairs an input with its ground-truth answer - the expected classification label, the correct extracted field, the ideal response, or the set of documents that should have been retrieved. Scoring works by comparing the model’s actual output to this reference, whether by exact match, semantic similarity, or an LLM judge that checks equivalence. Ground truth is usually produced by human experts, drawn from authoritative sources, or agreed by labelling consensus.

Why it matters

Every objective evaluation needs something to measure against, and ground truth is that anchor. Metrics like accuracy, precision, and recall are meaningless without it. Its quality directly caps the quality of your evaluation - if the reference answers are wrong or inconsistent, the scores mislead you no matter how good the scorer is. This is why labelling guidelines and reviewer agreement matter so much. For open-ended generation, exact ground truth may not exist, which is where reference-free approaches like LLM-as-a-judge or faithfulness checks against retrieved context come in.

In practice

Collecting ground truth is often the most expensive part of building an eval set, so teams prioritise the cases that matter most and grow the labelled set over time. Some tasks have cheap ground truth - a math problem has one answer, a SQL query either runs correctly or does not. Others, like summarisation, need human judgment or a proxy. Tools such as DeepEval and Braintrust let you store reference outputs alongside inputs and pick scorers that compare against them. See the metrics guide for which scoring methods suit which kinds of ground truth.