How Do You Know Your Prompt Change Made Things Better?

Change a prompt, try it once, and it looks better. That's not evidence — it's one sample from a distribution, chosen by the person hoping it improved. The minimum viable answer is a fixed set of inputs you run before and after, and assertions about properties rather than exact text. You don't need an evaluation framework for this; a JSON file of cases and a script that runs them is enough to catch the regressions that actually happen. And because every case is a billable model call, run it in CI before you deploy rather than on the container you're already paying to keep alive.

Assert properties, not text#

The instinct is to record the model's output and compare against it. That fails immediately, because models vary between runs and every trivial rewording shows as a failure until you stop trusting the suite.

Assert things that are stably true instead:

• Contains / doesn't contain. The answer mentions the refund policy; it never mentions a competitor.
• Shape. It's valid JSON, it parses into your schema, the required fields are present.
• Behaviour. It called the right tool, with an argument in the right range.
• Refusal. For inputs that should be declined, it declines.
• Budget. Output is under a token or character limit — which doubles as a cost check.

The good ones are mechanical. Anything a plain assertion can decide is cheap, deterministic, and won't argue with you at 2am.

Build the set from real failures#

A suite invented up front tests what you imagined. A suite grown from production tests what actually breaks.

So make it a habit: when something goes wrong — a bad answer, a mis-called tool, an injection attempt that worked — add the input as a case with an assertion that would have caught it. That's how the set becomes worth running, and it's the same discipline as adding a regression test after a bug.

Twenty cases that each represent a real failure beat two hundred generated ones.

Run it in CI, before the deploy#

Where you run this matters more than usual, because every case costs money.

• CI, on prompt or model changes. That's the moment a regression is introduced, and CI has the resources and the secrets already.
• Not on the container. Your app's allocation is sized for serving, not for a batch of model calls on a fractional CPU — and you'd be paying for the container while it waits on an API.
• Not in production traffic. Evaluating against real users is a different activity with different consent implications.

The natural place is the job before deploying: smoke test the image for structure, run evals for quality, then push to deploy. If the evals fail, nothing ships.

Keep the full set for prompt and model changes, and a small subset per commit. Running two hundred cases on every push to a branch is how a suite gets deleted.

Non-determinism, handled honestly#

Models are not deterministic, and pretending otherwise makes a flaky suite that erodes trust.

• Lower the temperature where your provider supports it. It reduces variance; it doesn't remove it.
• Run borderline cases a few times and require a pass rate rather than a single pass. Three out of three is a different claim from one out of one.
• Track the rate, not just pass/fail. "18/20 passed, was 19/20" is information. A red X is an argument.

Accepting a threshold feels unrigorous next to normal tests, and it's the honest model of what you're testing.

Model-as-judge, sparingly#

For qualities you can't assert mechanically — is this answer helpful, is the tone right — you can ask a model to grade the output. It works, and it comes with costs worth naming.

Every judged case is a second model call, so your suite doubles in price. The judge has its own variance, so a failure might be the judge rather than the change. And a vague rubric produces vague grades — if you use one, make the criteria specific enough that you'd agree with the verdict.

Use mechanical assertions wherever they'll do, and reserve judging for the small number of cases where they won't.

Keep it in the repository#

Two practical reasons to store cases and prompts as files next to your code rather than in a spreadsheet or a service:

• They version with the change. A prompt edit and the case it broke land in the same commit, so you can see what you traded.
• Your coding agent can run them. If the suite is a script and a JSON file, an agent iterating on a prompt can check its own work — which is the difference between "this looks better" and evidence, and it's the same artefacts-over-summaries principle that applies to deploys.

Include a short README describing what a case looks like. It's the file future-you will read first.

When it's worth it#

Not every AI feature needs this. The threshold is roughly:

• You've changed a prompt twice and weren't sure it helped. That uncertainty is the cost you're paying now.
• Other people use it. A regression is now their problem, not just yours.
• You want to change models. This is the big one — switching providers is guesswork without a fixed set, and an afternoon of work with one.

Below that threshold, "try it a few times and read the output" is a reasonable engineering decision, honestly made.

FAQ#

How do I test an LLM feature?#

Fix a set of inputs and assert properties of the output — contents, shape, tool calls, refusals, length — rather than comparing exact text.

Do I need an evals framework?#

No. A JSON file of cases and a script that runs them catches the regressions that actually occur.

Where should evals run?#

In CI, before deploying. They're model calls, so they cost money and don't belong on your app's container or in production traffic.

How do I deal with non-deterministic output?#

Lower temperature, run borderline cases several times, and require a pass rate rather than a single pass. Track the rate over time.

Should I use a model to grade outputs?#

Sparingly. It doubles the cost and adds its own variance. Prefer mechanical assertions where they work.

When is this worth setting up?#

When you've changed a prompt and couldn't tell if it helped, when other people depend on it, or when you're considering switching models.

Wrapping up#

Prompt engineering without measurement is editing until you like the look of it. A dozen cases in a file, assertions about properties rather than wording, and a CI job that runs them before anything deploys converts that into a signal you can act on — including the signal that a change you were proud of made things slightly worse. Grow the set from real failures, keep the mechanical assertions in the majority, and be honest that a pass rate is what you're measuring.