Before you trust an agent,
test the work.

Describe the job in a way another person could run.

Give each case a clear input, starting state, goal, allowed tools, constraints, and success criteria. Draw from the work people actually want the agent to do; include ordinary cases, edge cases, and failures that have already appeared.

For a coding agent, “fix the bug” is not enough. Specify the repository state, the behavior that is wrong, what a valid fix must do, and what must remain untouched. An evaluation task should test the product’s promise, not a vague impression that the agent seemed helpful.

Check the state of the world, not the claim of success.

An agent can report that it finished while leaving the work incomplete. Inspect the artifact or environment it was meant to change: tests pass, the requested record exists, the right file was updated, or the intended action was safely declined.

Choose metrics that match the product. A code test may be binary; a research summary may need several criteria for coverage, evidence, and source quality. A single blended score can hide which part failed.

Automate the objective checks. Review the judgment calls.

Use code for things code can decide reliably: exact fields, file state, unit tests, permitted tool calls, or a resource limit. When quality depends on nuance, a written rubric and a model grader can help. Compare a sample of those judgments with people who understand the task, and revise the grader when its scores do not match their reasons. See how LLM judges work and how to validate them ↗.

For consequential decisions, keep an appropriate human review step. An automated grader can scale a check; it cannot make an unclear requirement meaningful or guarantee that its own judgment is correct.

One successful run is a demonstration, not a reliability estimate.

Agents can take different paths across runs. Repeat a task under the same documented conditions, then record how often it succeeds and what kinds of failure recur. Keep the model, prompt, tools, permissions, and environment version with the result so a later comparison has a fair baseline.

When a run fails, inspect the trace and the final state. The cause may be the agent, an unreliable tool, an ambiguous task, or a grader that rejected a valid solution. Fixing the wrong part can make the system worse while making one number look better.

Include the cases that reveal unsafe shortcuts.

• The request is incomplete and the agent should ask a question.
• A document or web page contains instructions the user did not give.
• A tool times out, returns malformed data, or produces an unexpected result.
• The requested action exceeds the permissions granted for the task.
• The agent reports success even though the final state is still wrong.

Put authorization checks where tools act. Keep a passing test for each important boundary, and include an outcome check as well as a review of consequential tool calls.

Use one set to discover limits and another to catch regressions.

Capability cases should probe tasks the agent does not yet handle reliably. Regression cases preserve behavior that should keep working after a change. When a real failure is understood, reduce it to a reproducible task and add it to the suite.

Review the suite itself over time. Remove stale assumptions, check whether the grader still measures the goal, and look for tasks that have become too easy to show meaningful improvement. Read failures rather than treating the aggregate score as a verdict. For coding agents, also test later requirement changes against the code they already produced; the maintainability guide ↗ explains what to inspect.

Make the evidence part of the workflow.

Read how software agents make work reviewable ↗. For adversarial testing of the wider product, see the AI red-teaming guide ↗; for broader questions, explore evaluation and safety research ↗.