Optimize a prompt by measurement, not by feel: define a small labeled test set, run two candidate prompts against it, score each by accuracy, and pick the winner.

Prompt optimization is an evaluation problem. Without a test set, "this prompt seems better" is a guess that often does not survive contact with real inputs. With even a handful of labeled examples, you can put a number on each prompt and make the choice defensible, which is exactly how prompt changes get validated in production.

Requirements

Expected Output

The more specific prompt scores at least as well, and usually better, because its output is easier to match against the labels.

Solution

The test set turns prompt comparison into a measurement. Normalizing the prediction (lowercasing and stripping a trailing period) before checking it against the label is what makes a free-text reply gradable, and it is also why the constrained Prompt B tends to win: its output is closer to the label to begin with. The same loop scales to dozens of examples and as many candidate prompts as you want to compare.

Comparing two prompts is the warm-up; real optimization searches a set of candidates. Score several prompts on the same labeled test set, rank them by accuracy, and report the winner. This is a one-step prompt search, the basis for automated prompt tuning.

Requirements

Expected Output

The most specific, label-constrained prompt usually ranks first; vaguer prompts score lower because their output is harder to match.

Solution

Scoring every candidate on the same fixed test set turns prompt choice into a ranking problem with a number behind each option, so the winner is defensible rather than a hunch. Sorting by accuracy surfaces it directly. This is exactly the loop an automated prompt optimizer runs at scale: propose candidate prompts, evaluate each against held-out labels, and keep the best, which is why a reliable accuracy function and a representative test set matter more than any single clever prompt.