I build agentic systems for a living, and the design reviews have started to sound the same. Someone presents a diagram with a generator, a critic, a retry loop and a confidence threshold, and every box on it has a name from the pattern literature. The design is legible. It is also, about half the time, going to spend five times the tokens to produce a worse answer than a single call would have, and nothing on the diagram says which half it is in.
So I stopped reading these diagrams as architectures and started reading them as claims about knowledge. Each box either knows something the model does not, or it does not. A retriever knows what is in the corpus. A compiler knows whether the code parses. A schema knows what shape the output has to be. A second call to the same model, with the same weights, asked whether the first call was right, knows nothing at all, and the published record on that is much harsher than the pattern names suggest.
That single question, what does this step know that the model does not, sorts a pattern catalogue better than any of the categories usually printed next to it.
Where the truth comes from, and the three places it can come from
There are exactly three sources of truth available to a system built around a language model, and none of them is the model.
The first is construction. If the output is generated against a grammar, a schema or a type, then whole classes of wrongness cannot occur, because the decoder could not emit them. This is a guarantee, not a check. Nothing has to run afterwards and nothing can regress.
The second is an external oracle: some other system that returns a verdict the model did not author. A test suite, a type checker, a retriever, a permissions lookup, an HTTP status code. Its verdict may be narrow, and it may be wrong about things outside its remit, but it is independent, and independence is the whole value.
The third is a person, which is the only oracle that answers "should we" rather than "is it correct", and the only one that can be held responsible.
Everything else is the model's own opinion wearing a different hat. That includes a critic prompt, a reviewer agent, a debate between instances of the same model, and a self-reported confidence score. The design question is whether any new fact enters the system on each pass, and in these cases none does.
Self-correction worked in the papers because the papers knew the answer
The strongest results for self-critique come from a family of methods where a large language model (LLM) produces an answer, criticises it, and revises. Reported gains were large enough that critic loops became a default. Huang et al. (2024) went back and checked what was producing them.
The finding is not subtle. On GSM8K (Grade School Math, a benchmark of grade-school word problems, 1,319 in the test split used here), GPT-4 scores 95.5 with standard prompting. After one round of self-correction it scores 91.5. After two rounds it scores 89.0, having spent five model calls instead of one. On HotpotQA, a multi-hop question answering set, GPT-4 goes from 49.0 to 49.0 to 43.0. Llama-2 does not so much decline as fall over: 62.0 on GSM8K becomes 43.5 after one round and 36.5 after two, and its score on CommonSenseQA, a multiple-choice commonsense benchmark, drops from 64.0 to 36.5.
Now run the same loop with one change. Use the ground-truth label to decide when to stop. GPT-4 on GSM8K goes from 95.5 to 97.5, on CommonSenseQA from 82.0 to 85.5, on HotpotQA from 49.0 to 59.0. Those are the improvements the earlier papers reported, and that is how they were obtained: the correct answer was used to decide whether another round was needed.
If you hold the answer key, you do not need the loop, because you already have the answer. The oracle label was not a detail of the evaluation harness. It was the entire mechanism. Strip it out and the same procedure, on the same models, on the same benchmarks, makes things worse.
There is a related result from a different direction. Stechly, Valmeekam and Kambhampati (2023) put GPT-4 on graph colouring, which has the useful property that a candidate solution can be checked by a short program. They found the model is weak at producing correct colourings, no better at verifying them, and therefore no good as its own critic. When they swapped the self-critic for a sound external verifier, iteration worked. They also found that the content of the criticism barely mattered. What mattered was that something outside the model was deciding when to stop.
A critic loop with no external verdict does not improve an answer. It resamples it, at several times the cost, with a bias toward change.
Debate is voting with extra steps
The obvious repair is to use more than one model instance. Let several of them propose answers and argue, and let the argument converge. This reads as though it introduces independence, and the reported numbers were good.
Huang et al. ran it against the correct baseline, which is not a single call but the same token budget spent on plain sampling. On GSM8K, standard prompting scores 76.7. Self-consistency with three sampled answers and a majority vote scores 82.5. Multi-agent debate, after one round and six responses, scores 83.2, against self-consistency at six responses scoring 85.3. After a second round, nine responses, debate scores 83.0 and self-consistency scores 88.2.
Debate loses to counting, and the gap widens as you spend more. Whatever debate is doing, it is a worse way of doing what majority voting already does.
This also settles something that looked like a contradiction in my own position. Self-consistency has no external oracle either. It samples the same model repeatedly and takes the most common answer, and it works: Wang et al. (2023) report gains of 17.9 points on GSM8K and 11.0 and 12.2 points on SVAMP and AQuA, two further arithmetic sets, over chain-of-thought prompting alone.
The difference is what the extra calls are asked to do. Sampling several independent attempts and counting them reduces variance, which is an honest statistical operation that needs no new information. Asking the model to judge its own attempt is a different operation: the judgement is drawn from the same distribution that produced the error, so it is correlated with the error in exactly the way that makes it useless. Variance you can average away. Bias you cannot.
So the useful version of the rule is narrower than "the model cannot help itself". The model can be aggregated. It cannot be its own referee.
What the loops that work have in common
Two of the best-known iterative methods do work, and they work for the same reason.
Reflexion reports 91% pass@1 on HumanEval, a Python coding benchmark, against GPT-4's 80%. The loop looks like self-critique: the agent reflects in words on what went wrong and keeps those reflections in memory for the next attempt. But on a coding task the signal it reflects on is a test suite that either passes or fails. The reflection is the model's, the verdict is not.
ReAct interleaves reasoning steps with actions, and its result on HotpotQA and on Fever, a fact verification set, comes from the model calling a Wikipedia search API between thoughts. The paper is explicit that this is what reduces the hallucination and error propagation that chain-of-thought prompting alone suffers from. The reasoning did not get better; it got interrupted by facts.
The same shape shows up in the retrieval patterns. Corrective Retrieval Augmented Generation adds a lightweight retrieval evaluator that scores the documents a query pulled back and, when confidence is low, falls back to a web search. The word "corrective" invites you to read it as self-correction. It is not. The evaluator is a separate model with a separate job, and the fallback brings in a corpus the system did not have a moment ago.
Retrieval-augmented generation (RAG) belongs in this group for the same reason, and it is worth saying plainly because the pattern is usually justified on other grounds. Retrieval is not primarily a context-window trick or a cost optimisation. It is the cheapest external oracle most systems have: a body of text that exists independently of what the model believes, and that a citation can be checked against. That independence is only as good as the ingest that produced it, which is its own problem: chunking and embedding destroy the properties you would want to check against. It is also only as good as what retrieval puts in front of the model, and the plausible-but-wrong passages around the right one can drag the answer down.
Every working iterative pattern I can find has this property. Something outside the model closes the loop. The patterns that fail are the ones where the arrow goes back to where it started.
A judge is a measurement instrument, and instruments drift
Using a strong model to grade output is the one place where the model does get to have an opinion, and the evidence there is genuinely positive, with limits that matter for where you put it.
Zheng et al. (2023) found that GPT-4 as a judge reaches over 80% agreement with human preferences on MT-Bench, a multi-turn question set, which is the same level of agreement humans reach with each other. That is a real result and it is why model-graded evaluation is now standard. The same paper documents the failure modes: position bias, verbosity bias, and self-enhancement bias, which is the judge preferring output from its own model family.
The follow-up work is more cautious. Bavaresco et al. (2024) evaluated thirteen judge models against nine exam-taker models on tasks where humans agree strongly with each other. Only the largest judges align reasonably with people, they still sit well below inter-human agreement, and individual scores can differ from human scores by as much as five points. Judges are sensitive to prompt length and complexity, and they are lenient.
Those two results are compatible, and the reconciliation is about what you use the number for. A judge is reliable in aggregate and over rankings: which of these two systems is better, did this week's change make things worse, which fifty examples should a person look at. It is unreliable per item, which is precisely what an inline critic gate needs it to be. Putting a judge in the request path as a pass/fail gate takes an instrument calibrated for averages and asks it to make individual decisions, which is how a leniency bias becomes a production incident.
Half of this was settled in 1987, and it does not port cleanly
A large part of any agent pattern list is distributed systems work with new labels on it. Compensating transactions were Garcia-Molina and Salem's sagas in 1987. Circuit breakers, timeouts, bulkheads and graceful degradation are Michael Nygard's Release It! from 2007. Idempotency keys, dead-letter queues, canary releases and shadow traffic predate every model any of us are deploying.
These belong on the list. They work, and they work because their oracle is a return code, which is about as external and as cheap as truth gets.
But they were designed against an assumption that the component being protected is deterministic, and that assumption is now false in two places.
Retry assumes a repeat. When a request fails after the side effect landed, retrying a deterministic service either replays the same operation or is rejected by an idempotency key. Retrying a model call produces a different answer. If the first attempt already called a tool that shipped something, the retry is not a repeat, it is a second, different action on top of the first. The idempotency key has to sit on the tool call rather than the agent turn, and the tool has to be the thing that owns it, because the agent cannot be trusted to send the same key twice. Whether that boundary is even yours to own depends on whether you rent the agent runtime or build it.
Circuit breakers assume failures announce themselves. A breaker trips on errors and timeouts. The characteristic model failure is a confident, well-formed, fast, wrong answer, which registers as a success on every signal a breaker watches. The breaker needs a quality signal fed to it from an evaluator, and that is a different piece of engineering from wrapping a client.
Borrow them, but do not assume the guarantee survived the substitution.
The catalogue, sorted by what closes the loop
Here is the list I actually use in reviews. The pattern names are the usual ones. The column that does the work is the second.
| Pattern | What closes the loop | What it buys |
|---|---|---|
| True by construction | ||
| Structured output, function calling | the grammar or schema | the shape cannot be wrong, only the content |
| Citation enforcement | the source span | a claim citing text not in the corpus fails to render |
| Idempotency key | the key store | a repeated side effect collapses to one |
| Metadata filtering | the permission record | a document the caller may not see is never a candidate |
| Prompt and model registry | the version identifier | you can say what produced an output six months ago |
| An external oracle | ||
| ReAct and tool use | the tool's response | facts the model did not have |
| Reflexion on code | compiler and test suite | a verdict the model cannot argue with |
| Retrieval-augmented generation | the corpus | text that exists whether or not the model believes it |
| Corrective and agentic RAG | a separate retrieval evaluator | a second opinion on whether retrieval worked |
| Reranking, hybrid search | the scoring model, exact match | precision the embedding alone does not give you |
| Guardrails, jailbreak and toxicity defence | a classifier outside the generator | policy enforced at a boundary the model cannot talk past |
| Retry, timeout, circuit breaker, saga | the return code | the 1987 and 2007 guarantees, with the caveats above |
| Golden dataset, regression evaluation | stored human labels | a number that moves when quality moves |
| Red teaming | an adversary | the failures your own test set was never going to contain |
| Tracing, token and cost analytics | the record of what happened | the ability to answer why, after the fact |
| A person | ||
| Human approval and escalation | a person | the only oracle for "should we", and the only one accountable |
| Aggregation, which is not verification | ||
| Self-consistency | nothing external, but samples are independent | variance reduction: 17.9 points on GSM8K |
| Multi-model verification | partly decorrelated errors | weaker than it looks, stronger than self-critique |
| The model's own opinion | ||
| Self-reflection, reviewer or critic agent | nothing | 95.5 to 89.0 on GSM8K, at five times the calls |
| Reflection and debate between same-model agents | nothing | 83.0 against 88.2 for majority voting at equal budget |
| Self-reported confidence scoring | nothing | a number that tracks fluency rather than correctness |
The rows in the last group are the ones worth arguing about, because they are the ones that look like engineering. A critic agent has a box, a prompt, a latency cost and a line in the architecture diagram. It just has no way of being right more often than the thing it is checking.
And the table has a practical use in the other direction. Walk it during a review and ask, for each step in the design, which group it belongs to. Steps that land in the last group either need an oracle attached or need deleting. In my experience the answer is usually deleting, and the answer quality goes up when you do, because you have stopped paying for rounds that move the output away from a correct first attempt.
What I am still unsure about
Three things, and the first is the one most likely to make this article wrong.
All of the negative evidence above was collected on models from 2023 and early 2024. Since then, training with reinforcement learning against verifiable rewards has become standard for reasoning models, which means the check that Huang et al. had to bolt on from outside has been pushed inside the weights. I would expect a current reasoning model to self-correct better than GPT-4 did, and I have not seen a replication at the same standard that tells me how much better. If someone runs it and the degradation is gone, the section on self-critique needs rewriting and the rest of the argument does not, because the rule was never about the year. It was about where the information enters.
Second, I have treated "external" as binary and it is a spectrum. A judge from a different model family, trained on overlapping data, is more independent than a second call to the same model and less independent than a compiler. How different two models actually are is not something their names tell you, because the usual vocabulary for model types describes design decisions that combine freely rather than distinct families, and two models sharing an architecture and a data pipeline will share errors. Kambhampati et al. (2024) argue for what they call LLM-Modulo frameworks, pairing models with sound external verifiers in a tight loop, and their position is that models cannot verify themselves but are extremely useful everywhere else in the loop. I agree with that, and I do not know how to price the middle of the spectrum, which is where most real systems sit.
Third, I have said nothing about cost, and cost is where this argument has teeth. A critic loop is not neutral when it fails, it is three to five times the calls, plus latency, to arrive somewhere worse. Each of those extra passes also resends the whole transcript, which is where an agent's bill actually comes from. That is the version of the finding that gets a design changed, and it is the number I would put in front of whoever is paying.
The rule I would defend today is the narrow one. Aggregate the model, do not let it referee itself, and for every step in the design be able to name the thing outside the model that decides whether that step worked. Where you cannot name it, you have a preference, not a guarantee, and it belongs in the part of the system where preferences are allowed.