[{"data":1,"prerenderedAt":444},["ShallowReactive",2],{"mdc-13kusx-key":3},{"data":4,"body":5},{},{"type":6,"children":7},"root",[8,17,24,38,43,50,55,60,65,101,106,112,117,239,244,249,259,265,270,280,290,300,305,311,316,321,326,331,354,359,365,370,375,380,386,391,420,425,429],{"type":9,"tag":10,"props":11,"children":13},"element","h3",{"id":12},"making-llm-output-trustworthy",[14],{"type":15,"value":16},"text","Making LLM output trustworthy",{"type":9,"tag":18,"props":19,"children":21},"h1",{"id":20},"how-to-evaluate-an-llm-feature-before-you-ship-it",[22],{"type":15,"value":23},"How to evaluate an LLM feature before you ship it",{"type":9,"tag":25,"props":26,"children":27},"p",{},[28,30,36],{"type":15,"value":29},"You evaluate an LLM feature by writing down what a correct answer looks like ",{"type":9,"tag":31,"props":32,"children":33},"em",{},[34],{"type":15,"value":35},"before",{"type":15,"value":37}," you look at what the system produces, running a fixed set of cases on every change, and building that set out of the inputs most likely to break it rather than the ones most likely to pass. Everything else is commentary.",{"type":9,"tag":25,"props":39,"children":40},{},[41],{"type":15,"value":42},"This is the step teams skip, and they skip it for an understandable reason: it produces nothing visible. A week spent on an eval harness ships no feature. But without one you cannot answer the only question that matters when you change a prompt — did that make it better or worse? — and so every subsequent change is a guess dressed as an improvement.",{"type":9,"tag":44,"props":45,"children":47},"h2",{"id":46},"why-doesnt-normal-testing-work-here",[48],{"type":15,"value":49},"Why doesn't normal testing work here?",{"type":9,"tag":25,"props":51,"children":52},{},[53],{"type":15,"value":54},"Because normal testing asserts that a function returns a value, and this function returns a different value every time.",{"type":9,"tag":25,"props":56,"children":57},{},[58],{"type":15,"value":59},"Deterministic software has a comfortable property: the same input produces the same output, so one passing test is proof. An LLM call is a sample from a distribution. Running it once and seeing a good answer tells you the distribution contains good answers. It tells you nothing about how often.",{"type":9,"tag":25,"props":61,"children":62},{},[63],{"type":15,"value":64},"That has three consequences that break the usual habits:",{"type":9,"tag":66,"props":67,"children":68},"ul",{},[69,81,91],{"type":9,"tag":70,"props":71,"children":72},"li",{},[73,79],{"type":9,"tag":74,"props":75,"children":76},"strong",{},[77],{"type":15,"value":78},"A single run is not evidence.",{"type":15,"value":80}," You need many samples of the same input to see the shape of what you are shipping.",{"type":9,"tag":70,"props":82,"children":83},{},[84,89],{"type":9,"tag":74,"props":85,"children":86},{},[87],{"type":15,"value":88},"Pass and fail are not binary.",{"type":15,"value":90}," Most outputs are partially right, so you need a rubric, not an assertion.",{"type":9,"tag":70,"props":92,"children":93},{},[94,99],{"type":9,"tag":74,"props":95,"children":96},{},[97],{"type":15,"value":98},"The tail is the product.",{"type":15,"value":100}," Average quality is comfortable and irrelevant; the 5% of outputs bad enough to break trust are what users remember.",{"type":9,"tag":25,"props":102,"children":103},{},[104],{"type":15,"value":105},"None of this means testing is impossible. It means the unit of testing moves from \"does this call work\" to \"what does this distribution look like, and did my change move it.\"",{"type":9,"tag":44,"props":107,"children":109},{"id":108},"what-goes-in-the-case-set",[110],{"type":15,"value":111},"What goes in the case set?",{"type":9,"tag":25,"props":113,"children":114},{},[115],{"type":15,"value":116},"Thirty is enough to start. What matters far more than the count is the mix, and the mix should be deliberately unfair.",{"type":9,"tag":118,"props":119,"children":120},"table",{},[121,145],{"type":9,"tag":122,"props":123,"children":124},"thead",{},[125],{"type":9,"tag":126,"props":127,"children":128},"tr",{},[129,135,140],{"type":9,"tag":130,"props":131,"children":132},"th",{},[133],{"type":15,"value":134},"Category",{"type":9,"tag":130,"props":136,"children":137},{},[138],{"type":15,"value":139},"Share",{"type":9,"tag":130,"props":141,"children":142},{},[143],{"type":15,"value":144},"Why it earns its place",{"type":9,"tag":146,"props":147,"children":148},"tbody",{},[149,168,186,204,221],{"type":9,"tag":126,"props":150,"children":151},{},[152,158,163],{"type":9,"tag":153,"props":154,"children":155},"td",{},[156],{"type":15,"value":157},"Ordinary, well-formed inputs",{"type":9,"tag":153,"props":159,"children":160},{},[161],{"type":15,"value":162},"~40%",{"type":9,"tag":153,"props":164,"children":165},{},[166],{"type":15,"value":167},"Regression floor. Every version passes; that is the point",{"type":9,"tag":126,"props":169,"children":170},{},[171,176,181],{"type":9,"tag":153,"props":172,"children":173},{},[174],{"type":15,"value":175},"Ambiguous inputs",{"type":9,"tag":153,"props":177,"children":178},{},[179],{"type":15,"value":180},"~20%",{"type":9,"tag":153,"props":182,"children":183},{},[184],{"type":15,"value":185},"Two readings are defensible. Does it ask, or does it guess?",{"type":9,"tag":126,"props":187,"children":188},{},[189,194,199],{"type":9,"tag":153,"props":190,"children":191},{},[192],{"type":15,"value":193},"Incomplete inputs",{"type":9,"tag":153,"props":195,"children":196},{},[197],{"type":15,"value":198},"~15%",{"type":9,"tag":153,"props":200,"children":201},{},[202],{"type":15,"value":203},"A required field is missing. Does it invent one?",{"type":9,"tag":126,"props":205,"children":206},{},[207,212,216],{"type":9,"tag":153,"props":208,"children":209},{},[210],{"type":15,"value":211},"Unanswerable inputs",{"type":9,"tag":153,"props":213,"children":214},{},[215],{"type":15,"value":198},{"type":9,"tag":153,"props":217,"children":218},{},[219],{"type":15,"value":220},"Your data genuinely cannot answer. Does it say so?",{"type":9,"tag":126,"props":222,"children":223},{},[224,229,234],{"type":9,"tag":153,"props":225,"children":226},{},[227],{"type":15,"value":228},"Adversarial or out-of-scope",{"type":9,"tag":153,"props":230,"children":231},{},[232],{"type":15,"value":233},"~10%",{"type":9,"tag":153,"props":235,"children":236},{},[237],{"type":15,"value":238},"Off-topic, contradictory, or trying to get out of scope",{"type":9,"tag":25,"props":240,"children":241},{},[242],{"type":15,"value":243},"The instinct is to build the set out of the first row, because those are the cases you thought about when designing the feature. They are also the cases where every version of the system performs identically, which makes them almost worthless for choosing between versions.",{"type":9,"tag":25,"props":245,"children":246},{},[247],{"type":15,"value":248},"The rows that separate a good system from a bad one are the last three. And the last one — unanswerable — is the one most people never test at all, which is unfortunate, because a confident wrong answer is the single most damaging output an LLM feature can produce. It doesn't look wrong. Nobody catches it. It propagates.",{"type":9,"tag":25,"props":250,"children":251},{},[252,257],{"type":9,"tag":74,"props":253,"children":254},{},[255],{"type":15,"value":256},"Treat \"I don't know\" as a correct answer when it is true, and score it as one.",{"type":15,"value":258}," A system that abstains honestly is deployable behind a fallback path. A system that fabricates fluently is not, no matter how well it does on the easy rows.",{"type":9,"tag":44,"props":260,"children":262},{"id":261},"how-do-you-score-without-fooling-yourself",[263],{"type":15,"value":264},"How do you score without fooling yourself?",{"type":9,"tag":25,"props":266,"children":267},{},[268],{"type":15,"value":269},"Three rules, all of which exist to stop you from grading toward what the system already does.",{"type":9,"tag":25,"props":271,"children":272},{},[273,278],{"type":9,"tag":74,"props":274,"children":275},{},[276],{"type":15,"value":277},"Write the answer key first, without the system.",{"type":15,"value":279}," Research each expected answer by hand before running anything. The moment you have seen the output, your sense of what a good answer looks like has been contaminated by it — you will find yourself accepting an answer because it is plausible rather than because it is right.",{"type":9,"tag":25,"props":281,"children":282},{},[283,288],{"type":9,"tag":74,"props":284,"children":285},{},[286],{"type":15,"value":287},"Score blind.",{"type":15,"value":289}," Strip which version or which model produced each output, shuffle them, and grade. This is not paranoia; it is the difference between measuring the system and measuring your hopes for it. It matters most when you are comparing two prompts you wrote, because you know which one you prefer.",{"type":9,"tag":25,"props":291,"children":292},{},[293,298],{"type":9,"tag":74,"props":294,"children":295},{},[296],{"type":15,"value":297},"Score on separate axes, not one number.",{"type":15,"value":299}," At minimum: did it produce anything, is the substance correct, did it fabricate, and is it grounded in something citable. Collapsing those into a single score hides the interesting failure — a system can have excellent coverage and terrible truthfulness, and one aggregate number will look fine.",{"type":9,"tag":25,"props":301,"children":302},{},[303],{"type":15,"value":304},"That last split is the one that changes decisions. Coverage — did something come back — tends to be high for everything. Truthfulness is where options actually separate. If you only measure coverage you will conclude the tools are equivalent, and they are not.",{"type":9,"tag":44,"props":306,"children":308},{"id":307},"what-about-using-a-model-to-grade",[309],{"type":15,"value":310},"What about using a model to grade?",{"type":9,"tag":25,"props":312,"children":313},{},[314],{"type":15,"value":315},"It works, with a specific caution: the grader must have something to compare against.",{"type":9,"tag":25,"props":317,"children":318},{},[319],{"type":15,"value":320},"Asking a model \"is this answer good?\" produces a judgement of plausibility, and plausibility is exactly the axis on which fabricated answers score well. That is the failure mode to design around, and it is not hypothetical — a graded case can pass on structure alone. Ask whether a response \"describes the product's cold-weather rating,\" and a response that invents a rating describes one beautifully. The check passes. The fact is fiction.",{"type":9,"tag":25,"props":322,"children":323},{},[324],{"type":15,"value":325},"So: give the grader the reference answer and ask whether the output is consistent with it. That turns a taste judgement into a comparison, which models are much better at.",{"type":9,"tag":25,"props":327,"children":328},{},[329],{"type":15,"value":330},"The practical split:",{"type":9,"tag":66,"props":332,"children":333},{},[334,344],{"type":9,"tag":70,"props":335,"children":336},{},[337,342],{"type":9,"tag":74,"props":338,"children":339},{},[340],{"type":15,"value":341},"Exact expected values",{"type":15,"value":343}," where a wrong fact is the failure — specifications, figures, dates, anything a user would act on.",{"type":9,"tag":70,"props":345,"children":346},{},[347,352],{"type":9,"tag":74,"props":348,"children":349},{},[350],{"type":15,"value":351},"Semantic pattern matching",{"type":15,"value":353}," where many phrasings are valid — a confirmation message, a greeting, a summary. \"Should include a date, a time, and a confirmation\" is the right level of assertion there.",{"type":9,"tag":25,"props":355,"children":356},{},[357],{"type":15,"value":358},"Use the strict form wherever fabrication would be costly, and the loose form only where variation is genuinely acceptable. Getting this backwards is how test suites go green while the product is wrong.",{"type":9,"tag":44,"props":360,"children":362},{"id":361},"what-do-you-do-with-the-results",[363],{"type":15,"value":364},"What do you do with the results?",{"type":9,"tag":25,"props":366,"children":367},{},[368],{"type":15,"value":369},"Set a bar, run it on every change, and let it be allowed to block you.",{"type":9,"tag":25,"props":371,"children":372},{},[373],{"type":15,"value":374},"The mechanism is unglamorous. Store the case set in the repo. Run it in CI, or at minimum before any prompt change reaches production. Record the score per axis over time. When a change moves the number down, you now know something you could not have known by looking at three outputs.",{"type":9,"tag":25,"props":376,"children":377},{},[378],{"type":15,"value":379},"Two things make this stick. First, keep the harness cheap to run — if it takes an hour, nobody runs it, and an eval nobody runs is a file. Second, feed production failures back into the set: every time a user reports a bad answer, that input becomes a case. Over a few months the set stops being your guesses about what is hard and becomes a record of what actually is.",{"type":9,"tag":44,"props":381,"children":383},{"id":382},"what-this-is-worth",[384],{"type":15,"value":385},"What this is worth",{"type":9,"tag":25,"props":387,"children":388},{},[389],{"type":15,"value":390},"A directional test — thirty cases, a hand-written key, scored blind — is not a statistically powered study, and it should not be presented as one. It is enough to catch a behavioural difference at the edge of a system's competence, and that is usually where the differences are large enough that more samples would be decoration.",{"type":9,"tag":25,"props":392,"children":393},{},[394,396,403,405,411,412,418],{"type":15,"value":395},"It is also, in my experience, the cheapest week of engineering available on an LLM feature. Everything downstream — ",{"type":9,"tag":397,"props":398,"children":400},"a",{"href":399},"/blog/constrain-the-model-dont-review-it",[401],{"type":15,"value":402},"constraining the output",{"type":15,"value":404},", ",{"type":9,"tag":397,"props":406,"children":408},{"href":407},"/blog/test-one-prompt-a-hundred-times",[409],{"type":15,"value":410},"testing at volume",{"type":15,"value":404},{"type":9,"tag":397,"props":413,"children":415},{"href":414},"/blog/what-an-llm-feature-costs-to-run",[416],{"type":15,"value":417},"choosing a cheaper model",{"type":15,"value":419}," — depends on being able to tell whether a change helped. Without the eval set, none of those decisions can be made on evidence. With it, all of them can.",{"type":9,"tag":25,"props":421,"children":422},{},[423],{"type":15,"value":424},"Before you wire a model into anything that answers on your behalf, it is worth a week of yours.",{"type":9,"tag":426,"props":427,"children":428},"hr",{},[],{"type":9,"tag":25,"props":430,"children":431},{},[432],{"type":9,"tag":31,"props":433,"children":434},{},[435,437,442],{"type":15,"value":436},"This is the pillar post for ",{"type":9,"tag":397,"props":438,"children":440},{"href":439},"/blog?series=trustworthy-llm-output",[441],{"type":15,"value":16},{"type":15,"value":443},".",1787908866326]