Your AI Judge Is Being Bribed by Its Own Memory

You built a self-improving loop where an LLM scores outputs, and those scores gate the next iteration. Congratulations — you may have built a bias amplifier, not a quality filter. New research shows the judge isn’t blind; it’s peeking at the last verdict and then largely rubber-stamping it.

What happened

Researchers tested whether LLM-as-judge systems produce independent evaluations when prior scores appear in the prompt context — the exact setup used in iterative refinement pipelines. They ran 185,271 successful evaluations across eight models and 20 fixed texts, using three conditions: no metadata, revision framing, and anchored metadata (revision number, attempt count, and a prior score). The verdict is damning: 7 of 8 models showed statistically significant downward or upward drift toward the anchor, with a Cohen’s d of 0.71 — a large effect by any social-science standard. On real categorical industry data with human-labeled ground truth, anchored metadata blocked 48% of error corrections and flipped 10.18% of correct judgments to a wrong label. The researchers then tried the obvious fixes — Chain-of-Thought reasoning and an explicit metadata-disregard instruction in the system prompt — and neither eliminated the total effect, though the warning did improve paired accuracy in the industry experiment. The core mechanism appears threshold-like: introducing any anchored score causes a sharp redistribution of output probabilities, but changing the anchor value within the tested range adds surprisingly little further distortion.

Cold read

This study uses 20 fixed texts — a narrow corpus that may not generalize to your domain, your data distribution, or your prompt structure. The “industry categorical data” experiment is the most operationally relevant finding, but the paper gives limited detail on what industry, what label taxonomy, or how the human ground truth was constructed; you can’t replicate this on your own data without that information. Cohen’s d of 0.71 is large in aggregate, but aggregate statistics hide enormous model-to-model variance — the one model that didn’t show the effect isn’t identified in the abstract, and that gap matters if you’re choosing infrastructure. The threshold-like token redistribution finding is interesting but comes from “selected model-task probes,” not a systematic sweep, so calling it a general mechanism is premature. And the paper’s conclusion — “careful context engineering” — is almost tautologically true of any prompt engineering problem; it doesn’t hand you a specific, validated fix.

What it means for you

• Signal maturity: 3/5 — effect is real and large, but mitigations are unvalidated at scale
• Who gets hurt: Any team running iterative AI refinement loops (content graders, RLHF-adjacent pipelines, automated QA systems) where the judge sees the previous score
• What breaks if this is true: Your “improving” pipeline may be converging on the prior score, not on actual quality — your golden dataset validation metrics look good while your production outputs drift
• Why it might not land: If your pipeline strips prior scores from the judge context entirely, you’re already immune; the attack surface is specifically context contamination, not the judge paradigm itself
• Watch for: Model providers adding explicit anchoring-resistance benchmarks to model cards; that would signal the industry has accepted this as a real infrastructure problem

Forecast as of 2026-08-27

By Q2 2027, at least two major LLM evaluation frameworks (e.g., LangChain Evals, Ragas, or a comparable library) will ship explicit “context isolation” modes that strip prior-score metadata by default — directly citing anchoring bias research — as this class of bug becomes common enough to appear in production postmortems.

Source: Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence — Ante Kapetanovic, Kemal Altwlkany, Andro Mercep, Tomislav Duricic, Emanuel Lacic. https://arxiv.org/abs/2608.25869v1