A coaching framework, a teacher's sequence, a clinician's intake flow. Each is full of judgment that lives in a person's head. The expert knows when to push and when to back off, which question comes next, and the line they will not cross, but almost none of it is written down. Turning one of these into a product means making that judgment explicit enough to build and measurable enough to trust. The leading labs have spent the last few years running a version of this same translation on their own models, and their methods transfer.
To keep it concrete, one method runs through the whole piece. Take a parenting coach, a product a parent opens at 11pm when a toddler will not sleep and they are out of ideas. Each move below gets shown on that one example.
Write the behavior down
The first move is to stop treating good behavior as something the model will infer and start writing it down. OpenAI publishes a Model Spec, a living document that states in plain language how its models should behave. It is organized around a chain of command, where each instruction carries a level of authority and a higher-authority instruction overrides a lower one. Root-level safety rules sit above platform instructions, which sit above the developer building the app, which sits above the end user, and untrusted tool output carries no authority at all. That ordering lets the model resolve a conflict on purpose instead of obeying whichever instruction it saw last. The spec also separates an outright refusal from a safe completion, a partial answer that stays inside the boundary.
Anthropic took a related path with Constitutional AI. Instead of paying humans to label harmful outputs one by one, the model is supervised against a short written set of principles. It samples an answer, critiques that answer against a principle, and revises it, and the revised answers finetune the model. A second stage trains a preference model from the model's own comparisons, so the written principles, not a wall of human harm labels, are the only human oversight in the loop. Anthropic did the same for tone in its Claude's Character work, writing down the traits it wanted and training the model to rank its own responses against them. Conduct that used to live in implicit taste gets written down first, then trained toward.
Your method deserves the same treatment. The states a user arrives in, the response that is right for each, and the lines the product must not cross, all written down before you build. For the parenting coach that document says, in plain sentences, how the product talks to a scared parent, what a good answer to a sleep question looks like, and the point where it stops coaching and tells someone to call a doctor. Everything below is a way of making that document concrete and gradable.
Start with archetypes
Name the recurring states a user actually arrives in, not the features you want to build. An archetype is a repeatable pattern. The user's state, the need underneath it, the path to a good outcome, and the ways it goes wrong.
For the parenting coach, one archetype is the exhausted parent of a toddler who will not sleep. Their visible state is frustration, sometimes near tears. The need underneath the sleep question is usually reassurance that they are not failing, plus one concrete thing to try tonight. The path to a good outcome is to name the feeling, ask a question or two about the specific situation, then offer a small, safe next step. The ways it goes wrong are a lecture that makes them feel judged, a stack of three questions when they are too tired to answer any, or generic advice that ignores what they just told you.
Archetypes are the unit of design and the unit of evaluation. If you cannot name the moment, you cannot build for it or tell whether a change helped it. A handful of well-drawn archetypes covers most of what a product sees, and each one becomes a row you can score.
Golden examples set the bar
Sit with your experts and capture the exchanges they would endorse, word for word. For the tired-parent archetype, that is a real transcript where a veteran parenting coach reads the message, reflects the exhaustion back in one sentence, asks a single grounding question about the bedtime routine, and hands over one thing to try, all without a hint of judgment. That exchange is the bar. These golden examples are worth more than a style guide. They are the reference your evals grade against and the fastest way to align a model with how the work is really done, because they show the behavior instead of describing it.
Be ready for the bar to move as you look at real outputs. The first time the model answers a sleep question with a competent-but-cold paragraph, you realize warmth was doing more work than your spec captured, so you revise the spec and the golden set. That is criteria drift, which the Berkeley EvalGen study found is not a bug in the process but how grading real outputs works, the reason a golden set is something you revise rather than finish. Hamel Husain recommends putting one domain expert in charge of that judgment rather than splitting it across a committee, so the bar stays coherent as it moves. The same expert who writes the golden examples is the one who stays in the loop as real transcripts come in.
The order of moves in that transcript, reflect first, then one question, then one step, is also a small piece of the path you are designing through the hard turns. Capture the sequence, not just the final answer.
Boundaries are a feature, designed up front
Just as important as what the product does is what it must never do. The parenting coach must not diagnose, must not tell a parent to change a prescribed medication, and must not try to hold a turn where a parent mentions their newborn has a fever and has not woken in hours. That last one is not a coaching moment. It is a call-your-pediatrician-now moment, and the product's job is to say so and get out of the way.
Write the boundary the way you wrote the behavior, as small, checkable rules rather than one paragraph that says be safe. The strongest precedent is DeepMind's Sparrow, a dialogue agent its researchers shaped with 23 explicit rules, informed by experts on language harms, and graded one rule at a time. A separate rule model estimates the probability that any single rule was broken at any point in the conversation, so a failure has an address. Small units are what let you say the fever turn broke the do-not-give-medical-advice rule specifically, rather than rereading the whole transcript to guess.
Refusal and escalation are product features, designed up front and expressed in the interaction itself, not a policy bolted on at the end. For the fever turn the right move is a safe completion that names the concern and points to a person, then a real handoff, not a wall of silence and a disclaimer. Route the hardest cases to a person by design, where the full boundary treatment lives, down to the over-refusal tax and the tested escalation path. None of this is captured by asking whether a demo felt good. It is captured by writing the method down, turning it into examples and rules, and grading against them.
From a written method to a number you can move
The point of writing the method down, drawing the archetypes, capturing the golden examples, and specifying the boundary is that the pipeline ends in something you can grade. Each archetype becomes a set of test cases, each golden example becomes the reference a grader scores against, and each boundary rule becomes a check a turn passed or broke. Run that suite on every model change and adherence to the method turns into a number you watch, regress on, and improve release by release, which is the whole reason evals are the moat and not the demo.
The tradeoff is worth naming. This is slow, expert-heavy work before you ship a single feature, and it does not pay off on a throwaway prototype. Sparrow still broke a rule 8 percent of the time even after all that rule-writing, so the method makes failures legible and addressable, it does not make them vanish. If your domain is low-stakes and forgiving, a good prompt and a spot-check may be all it earns. The moment a wrong answer can hurt someone, the parent who needed a doctor and got a coaching question instead, the written method stops being overhead and becomes the only way to know whether the product is doing its job.
Start with one archetype. Write the state, the need, and the path. Capture three golden exchanges from your expert. Name the one boundary that archetype can hit and the escalation it triggers. Turn all of it into graded cases and decide what that grade should measure before you trust the number. Once the loop runs for one moment you repeat it for the next, until the judgment that lived in one person's head is a system you can build on and a number you can move.
Sources and further reading
Work with Hunter Green