It is easy to measure the things that do not matter. Messages sent, sessions started, tokens spent. A dashboard full of those numbers can climb all quarter while the product quietly fails the people using it. The hard part, and the only part that counts, is whether the product actually helped the person in front of it. The teams who have built these systems longest have converged on a short answer. Measure the process, validate your graders, and watch the moments where trust is won or lost.
None of those three is the obvious metric. The obvious metric is the outcome, did the user get better, and that one is slow, noisy, and nearly impossible to attribute to any single turn. The three below update daily and point at a specific thing to fix.
Process before outcome
Outcomes are slow, noisy, and hard to attribute. A coaching product might change someone's month, and by the time the month is over you have no idea which turn did the work. Process metrics, meaning did the system follow the method this turn, are fast and actionable. You know within seconds, and you know which step broke.
Google's Vertex team argues the point in concrete terms. You have to evaluate the trajectory, the sequence of steps the system took, not only the final response, because the trajectory is where the reasoning and the errors show up. Score only the final answer and you are grading a suspect on the alibi, not the evening. Two runs land on the same closing line, one because the system asked the clarifying question the method calls for and one because it guessed and got lucky. The final response cannot tell them apart. The trajectory can. "Did the system follow the method" maps almost one to one onto that.
The useful move is that they turn the trajectory into real numbers instead of a vibe. Their metrics grade the predicted sequence of steps against a reference sequence you wrote down. trajectory_exact_match returns 1 only when the steps are identical to the reference in the exact same order. trajectory_in_order_match is looser, it returns 1 when every reference step appears in the right order and tolerates extra steps in between. Above those sit trajectory_precision, the share of the steps the system took that were warranted, and trajectory_recall, the share it should have taken that it did. Precision falling means the system is doing work it should not. Recall falling means it is skipping a step the method requires.
That skip is the failure to watch for. A guidance system that is supposed to check a risk flag before it offers advice fails invisibly when it drops that step and still produces a plausible answer. An in-order trajectory metric catches it. An outcome metric averaged over a week never will. When your method is really a sequence, the tool calls and their order are a first-class thing to evaluate.
Make adherence a number you can move
Turn the method into graded criteria and score every interaction against them. Not one global quality score, but a criterion per step the method demands, so a failure points at the step that broke rather than at the whole run. Adherence is then a number you can watch, regress on, and improve release by release. It is the closest thing to a real quality signal that updates daily, and it feeds the loop where each production failure becomes a new eval case.
The caution from people who do this at scale is to define those criteria narrowly and from your own data, not from a public benchmark built for someone else's product. The authors of What We Learned From a Year of Building with LLMs, six practitioners writing together, warn that the moment a benchmark becomes the target it stops being a good measure, the old Goodhart problem in new clothes. Optimize hard enough against a proxy and the model learns to satisfy the proxy while the thing you cared about drifts away underneath it. Narrow criteria drawn from your own transcripts are harder to game because they are closer to the behavior you actually want.
This is also why adherence is a moat and not a commodity. Anyone can call the same model. The graded criteria that encode your method are the asset that compounds while the model stays swappable.
Validate the grader before you trust it
Most teams grade open ended answers with another model, an LLM as judge, because a human cannot read every transcript. That works only if you check the judge. OpenAI's own evals guide states the catch directly. Model grading has an error rate, so you have to validate it against human judgment before you run it at scale, and grade with a stronger model than the one under test.
The Berkeley EvalGen study goes one step further. Because the criteria themselves drift as you grade, the grader cannot be calibrated once and left alone. You start grading and discover the rubric you wrote does not survive contact with real outputs, so a judge tuned to last week's rubric is now measuring the wrong thing. It has to track human judgment continuously.
A judge model also carries its own biases, position, verbosity, self-preference, that skew scores in ways that have nothing to do with quality. Those need their own handling, and validating the judge covers how to detect and correct for them. The narrower point to carry here is that a grader you have not checked against humans is not a measurement, it is a second unvalidated model sitting on top of the first.
Watch the trust moments
A handful of turns decide whether someone comes back, and they are not evenly spread through the conversation. The boundary request, where the user asks for something the system should decline. The moment of doubt, where the user is deciding whether to keep trusting it. The hard disclosure, where the user says something that raises the stakes. Most turns are ordinary and forgiving. These few are neither.
There is good reason an average will mislead you on exactly these turns. Surveying what makes human labels reliable, Lilian Weng points to a safety labeling study where rater agreement runs from 0.96 on violence and gore down to 0.25 on personal topics. Raters line up almost perfectly on the clear cut cases and split badly on the personal ones, and the personal ones are your trust moments. The turns that matter most are the turns your labels are shakiest on, so the aggregate is smoothing over precisely the place you cannot afford to be wrong. The average looks fine right up until the moment that wasn't.
Work one through. Say a user asks your coaching product for a medical dosage, a boundary the method says to decline and redirect. Fold that turn into a global quality average and it is one interaction among thousands, invisible. Slice it out as its own metric, the boundary-request slice, and you can see the system handled the boundary correctly 82 percent of the time while the overall score sat at a comfortable 94. That gap is the whole risk of the product, and the aggregate hid it. Do the same for the moment of doubt and the hard disclosure, each on its own dashboard, and a healthy average can no longer launder a failing boundary.
When process metrics are the wrong tool
Process adherence is not free and it is not always right. It costs you a written-down method to grade against, and if you do not have one yet, forcing the trajectory into criteria just encodes your guesses as if they were the standard. Early on, reading transcripts by hand teaches you more than any exact-match score, because you are still discovering the method, not enforcing it.
There is also a failure mode where perfect adherence hides a broken method. The system can follow every step you wrote and still fail the user because the steps were wrong. Process metrics tell you whether the system did what you told it to, not whether what you told it to do was any good. That is what the slow outcome signal is for, and why you keep a thin outcome measure running even though it is noisy. Adherence catches the regression day to day. The outcome check, over a longer window, tells you the method itself needs to change.
The rule to carry
Measure the process, not the vibe. Write the method down as a reference trajectory, grade every interaction against it step by step, and validate the grader against humans before you trust its scores. Then pull the three or four turns that decide trust out of the aggregate and watch them on their own, because that is the place an average is designed to lie to you. Keep one slow outcome signal in the background as a check on the method itself.
The open question is the one Weng's numbers point at. On the personal turns that matter most, humans agree only a quarter of the time. If your best human labels split there, what does it even mean for your grader to be right, and how much of trust is a thing you can score at all versus a thing you can only watch closely and stay honest about.
Sources and further reading
Work with Hunter Green