What is an LLM judge?
A score is a claim.

An LLM judge applies a rubric to a model output.

In LLM-as-a-judge, a language model receives an answer and evaluation instructions, then returns a judgment: a score, a category, a pass/fail result, or a preference between responses. The evaluator can also be asked to point to evidence for its decision.

The method is useful when a task involves language qualities that are hard to capture with exact-match code alone, such as whether a summary covers the important points or whether an explanation answers the question. It can help review large sets of outputs, but it does not remove the need to define the standard or check the evaluator.

A judge score is a measurement signal, not ground truth. When the question has a clear, testable answer, a programmatic check is usually easier to inspect. When correctness has important consequences, qualified human review still matters.

Choose the comparison that matches the question.

Pairwise judgments can be easier than assigning an absolute score when the task is “which answer handles this criterion better?” A reference can help when the expected evidence is known. Neither setup makes an ambiguous rubric precise, and reference answers may omit valid alternatives.

Keep the evaluation target narrow. “Did the answer identify the two exceptions in the supplied policy?” is easier to inspect than “Is this a good answer?” If several qualities matter, score them separately so a strong style score cannot hide missing evidence.

A fluent decision can still be a biased decision.

Research has documented cases where a judge responds to surface cues instead of the property a benchmark intends to measure. A pairwise judge may change its preference when the order of two answers is swapped; a longer or more polished answer may win without being more correct. A 2023 study also examined self-enhancement, but its authors said the available data was too limited to establish that effect reliably.

These are observed risks, not proof that every judge fails in the same way. A 2025 study of position bias found that its strength varied across judges and tasks. That variation is exactly why a result from one evaluator or benchmark should not be treated as a universal guarantee.

Asking for a written rationale does not verify the verdict. The explanation can sound coherent while citing the wrong evidence. Check the cited passage or requirement yourself, and measure whether the judgment remains stable when irrelevant details change.

Calibrate it before you scale it.

• Write one observable question for each criterion; avoid “overall quality” unless you can define it with examples.
• Create human-reviewed examples that include clear passes, clear failures, and borderline answers.
• Give the judge only the instructions and source material needed to assess the criterion.
• For pairwise comparisons, randomize candidate order and test whether swapping it changes the result.
• Compare the judge with human decisions by criterion and case type; inspect disagreements instead of hiding them in an average.
• Use code for objective checks and route high-impact or uncertain decisions to an appropriate person.
• Recheck the evaluator when its model, prompt, rubric, source material, or task distribution changes.

Start with a small set of examples. If the judge rewards verbosity, style, or a superficial cue, revise the rubric and test it against cases designed to expose that shortcut. Only scale up when the judge’s errors are understood and acceptable for the decision it supports.

For a complete workflow, see how to evaluate AI agents ↗ and how to evaluate a RAG system ↗.

Agreement is evidence about a setup, not a verdict on every setup.

After addressing several limitations in its evaluation setup, the MT-Bench and Chatbot Arena study reported that GPT-4 judgments agreed with human preferences more than 80% of the time on the tasks it tested, at a level similar to human agreement in that study. Its findings do not predict agreement for every task or evaluator. The paper also examines position, verbosity, and possible self-enhancement bias; for the latter, the authors said their data was not enough to establish the effect conclusively.

Compare systems under the same prompt, source material, criteria, and evaluation conditions. Report the judge model and rubric alongside the score. Keep objective results, human judgments, and model-judge estimates distinguishable rather than blending them into one number.

A benchmark is still shaped by its tasks, data, scoring rule, and evaluator. Read how to interpret benchmark results ↗ before treating a leaderboard score as a product decision.

Four useful boundaries.