Ask an enterprise how they will know whether their AI system is working and you will usually get a description of a dashboard. That is monitoring, not evaluation. The difference matters more than almost anything else in the programme.

Monitoring tells you the system is running. Evaluation tells you whether it is right. Most organisations invest heavily in the first and improvise the second, which is why so many AI programmes cannot answer basic questions: is this version better than the last one? Did that prompt change help? Is the model degrading, or has the input distribution shifted?

Build the harness first

The sequence most teams follow is: build a prototype, get excited, deploy something, then scramble to work out how to measure it. Inverting that order costs a week and changes everything downstream.

A harness has four parts:

  • A golden dataset. Two to five hundred cases that represent the real distribution, including the awkward tail. Not the easy ones.
  • An agreed labelling protocol. Written down, with worked examples of the ambiguous cases and a rule for resolving disagreement.
  • Metrics tied to the decision. Not accuracy in the abstract — the cost of a false positive versus a false negative in your actual process.
  • A regression suite that runs automatically. Every change, every version, same set, comparable numbers.

Labels are a business decision, not a data task

The most valuable hours in any AI engagement are the ones spent arguing about labels. When two experienced people disagree about whether a case is a defect, you have found something important: either the definition is ambiguous, or the process tolerates variance nobody has acknowledged.

If your domain experts cannot agree on the label, your model cannot learn it and your users will not trust it. Resolve the ambiguity in the process before you try to automate it.

Choose metrics that map to money

Accuracy is rarely the right headline. In quality inspection a false negative ships a defect to a customer; a false positive sends a good part for rework. Those costs are wildly asymmetric and the ratio should set your threshold, not a default of 0.5.

Write the cost matrix down explicitly. It converts a modelling argument into a business one, which is a much easier conversation to have with a steering committee.

A test worth applying

If you cannot state, in one sentence, what it costs your business when the model is wrong in each direction — you are not ready to set a threshold, and any threshold you pick is arbitrary.

Generative systems need the same discipline

The rise of LLM applications has made evaluation harder and more necessary at once. Outputs are open-ended, so exact-match scoring is meaningless. But the alternative is not "vibes" — it is a graded rubric applied consistently, a set of adversarial cases, and a check on refusal behaviour and citation accuracy.

We typically run three layers: deterministic checks for format and grounding, rubric-based scoring for quality, and a small human review sample every week. The third catches what the first two miss, and its findings feed back into the rubric.

The strategic payoff

Once a harness exists, several hard questions become easy. Should we use a smaller model? Run the harness. Is the vendor's system better than ours? Run the harness. Did last month's prompt refactor help? Run the harness.

One of our clients now requires every AI vendor to submit to their harness before procurement will progress. It has changed what gets sold to them — which is, arguably, the highest return any evaluation investment can produce.