What an evaluation harness has to prove

An evaluation is not a scoreboard. It is the argument that a system is fit for a specific job, and that you would notice if it stopped being fit.

Teams collect scores and still cannot decide whether to ship. The score is often measuring a convenient proxy: similarity to a reference answer, a judge-model's preference, a public benchmark that does not resemble the workflow. Those signals are useful. They are not, by themselves, a release decision.

A harness has to prove a smaller and harder claim. On the tasks this system is allowed to do, quality is good enough, the important failure modes are detected, and a change in model, prompt, or retrieval can be compared with the last known-good version. That means datasets drawn from real work, labels you trust, and metrics tied to the failure you actually fear: invented facts, missed obligations, unsafe tool use, irrelevant retrieval.

Human review belongs inside that design, not as an apology after it. People should score the cases a model judge cannot be given the last word on, and the sample should be large enough to change a decision. Continuous evaluation then keeps the harness alive in the path to production, so a Tuesday prompt edit does not silently retire the evidence you had on Monday.

Done this way, evaluation is a product surface. It produces a bar, a report, and a pipeline. It also produces something cultural: a shared definition of good. Without that definition, every model upgrade is a debate, and every incident is a surprise.