Fluency is not
evidence.

A language model can repeat a familiar falsehood.

A language model generates a response from patterns learned during training and the context it receives now. That can produce useful explanations, but fluency does not show whether a statement was checked against a reliable source. A model may reproduce a common misconception, combine related details incorrectly, or supply a specific fact that its available evidence does not establish.

TruthfulQA made one part of this problem measurable with 817 questions across 38 categories. The researchers wrote questions around popular misconceptions that some people would answer falsely, then tested whether models repeated those errors. The benchmark helps study that failure mode; it is not a complete score for truthfulness across every topic, model, or real-world use.

Read the TruthfulQA paper ↗

Even a scorecard can push a system toward confident guesses.

In a 2026 Nature paper, Kalai and colleagues argue that two pressures help explain persistent hallucinations. Next-token pretraining can create statistical pressure to produce errors about rare or one-off facts, even with idealized training data. Later, an evaluation that awards credit for correct answers but gives no credit for abstaining can reward guessing instead of a careful “I don’t know.”

The researchers analyzed popular benchmarks and tested alternative scoring instructions with four frontier models on SimpleQA. When the rubric made the cost of wrong answers and the reward for abstaining explicit, the models adjusted how often they answered, and a tested consistency-based mitigation scored better across the studied settings. This is evidence about their analysis and experiment—not a claim that one scoring prompt solves hallucination everywhere.

The lesson for evaluation is direct: measure wrong answers and useful abstentions, not accuracy in isolation. The right balance depends on the stakes. A mistaken trivia answer and a mistaken medical instruction do not carry the same cost.

Read “Evaluating large language models for accuracy incentivizes hallucinations” ↗

A citation must support the sentence beside it.

Search and retrieval can bring useful evidence into an answer. They do not guarantee that a result is authoritative, current, or relevant to the exact claim. A citation may point to a page that mentions the topic without supporting the sentence, or to an outdated version of information that has since changed.

NIST’s Generative AI Profile treats confabulation as a risk and recommends reviewing sources and citations in system outputs during risk measurement and monitoring. A citation is a route to inspect the evidence, not proof that the claim follows from it.

For a consequential or time-sensitive question, open the cited source. Check the relevant passage, date, definition, and scope. If the evidence conflicts or leaves something unresolved, preserve that uncertainty instead of filling the gap with another plausible sentence.

Read NIST’s Generative AI Profile ↗

Turn a paragraph into a short evidence trail.

When an answer matters, separate it into claims that could be true or false. Then check the pieces that carry the conclusion:

For an AI-generated summary, compare each important statement with its cited passage. For a model benchmark, inspect the task and scoring rules. For a current policy or product fact, consult the responsible organization’s latest documentation. Keep direct evidence separate from the model’s interpretation of it.

This is useful whether a person is checking an answer or a product is designed to show its work. A polished explanation can support understanding; evidence and a way to challenge the conclusion make it reviewable.

Read AI benchmark results in context ↗
Apply source checks to images ↗
Understand what AI detectors can tell you ↗