Evaluator-optimizer

Evaluator-optimizer pairs two calls in a loop. One generates a response, the other evaluates it against clear criteria and returns feedback, and the generator tries again with that feedback in hand. The loop ends when the evaluator is satisfied or when a fixed number of rounds runs out.

I reach for it when I can write down what good looks like well enough that a model can check for it, and when a first draft is reliably improved by specific criticism. Literary translation, where a reviewer can name the nuance the first pass missed, is the classic case. Search tasks that need several rounds of refining and re-checking are another.

The cost is rounds. Each pass through the loop is two calls, and a loose evaluator can keep a generator circling for a long time while the output barely changes. I always cap the iterations and I always log every round, because the bill for this pattern arrives quietly.

It goes wrong when the evaluator's criteria are vague. Asked whether something is good, a model will usually say it could be better, and the generator will dutifully change things that did not need changing. The other failure is the two roles collapsing into one voice, so the evaluator approves whatever the generator would have produced anyway.

In a good one I look for evaluation criteria I could hand to a person, a feedback format that names specific problems rather than issuing grades, and evidence that the loop actually converges. If the third draft is not measurably better than the first on the stated criteria, the pattern is costing me money for nothing.