Parallelization

Parallelization runs several model calls at the same time and combines what comes back. There are two flavors. Sectioning splits a task into independent pieces and stitches the results together. Voting runs the same task several times and takes the consensus, or the strictest answer, depending on what the task needs.

I reach for sectioning when the pieces genuinely do not depend on each other, such as screening a request for policy problems in one call while answering it in another. I reach for voting when I care more about confidence than speed, such as reviewing code for vulnerabilities where three independent passes catch more than one careful one.

The cost is tokens. Five parallel calls are five calls, and the combine step is one more. It is fast in wall-clock time, which is why it is tempting, but it is not cheap, and the combining logic has to handle disagreement, partial failure, and outputs that do not line up the way I expected.

It goes wrong when the pieces were not as independent as I thought. If the summary of section two needs to know what section one said, running them at once produces a result that reads like it was written by two people who never met. Voting goes wrong when the voters share a bias, because agreement then means nothing.

In a good one I look for a split that I can justify in one sentence, a combine step that is deterministic where possible, and a clear rule for what happens when one branch fails. I also want to see the voting threshold written down, and a reason for it.