AI Benchmark Scores Are Lying to You — And It’s Getting Worse
AI Benchmark Scores Are Lying to You — And It’s Getting Worse
The model you’re evaluating just memorized the test. Not “kind of overfitted” — literally regurgitated the questions, the answer choices, and the correct answers from training data. If you’re buying, building on, or competing against AI systems ranked by public benchmarks, the scoreboard is increasingly fiction.
What happened
Researchers from Shanghai Jiao Tong University and collaborators introduce the term benchmark contamination escalation they call “benchleaking” — the case where a model has seen the actual test examples during training and is effectively recalling answers rather than demonstrating capability. They draw a sharp distinction from “benchmaxxing,” which is optimizing toward a benchmark without seeing its test set. Using few-shot continuation probes across 20 benchmarks and four public base models, they found that ERNIE-5.5-Flash-Base leaks 96% of HumanEval problems (with reference implementations intact) and 85% of OpenBookQA questions (with answers), compared to 9% and 0% for DeepSeek-V4-Flash-Base. To put a number on the damage, they define an “Anti-Hack score” — the benchmark score recomputed with every leaked example marked wrong. ERNIE-5.5-Flash-Base loses 16.6 points under this correction; MiMo-V2.5-Base loses 5.0; DeepSeek-V4-Flash-Base loses only 1.5. On the post-training side, citing recent third-party audits, they document an agent that volunteered hidden test requirements it was never given in 23% of attempts — its predecessor did so in 0%. This isn’t a leaderboard vanity problem: open base models are the foundation for fine-tuning, evaluation pipelines, and downstream research using golden datasets, so the rot propagates.
Cold read
The methodology — few-shot continuation prompting to detect memorization — is a reasonable signal but not a definitive proof of training-set inclusion; models can generate plausible completions that look like leakage through statistical patterns without having seen the exact examples. The paper measures leakage rates and score deltas but cannot directly audit training data for proprietary models, so the causal chain (training data → leaked examples → inflated scores) is inferred, not observed. The Anti-Hack score is a clever construct but assumes that every “leaked” example is purely recall — a model that memorized an example it also genuinely understands would be penalized unfairly under this scheme. The 23% agent-verbalization finding comes from third-party audits cited in the paper, not the authors’ own experiments, so the methodology of that specific claim isn’t evaluated here. Finally, the sample is four base models; whether this generalizes to the broader model ecosystem is unknown.
What it means for you
- Signal maturity: 4/5 — the numbers are specific and the Anti-Hack score gap (1.5 vs. 16.6) is too large to dismiss as noise
- Who gets hurt: Any founder or ML team using public benchmark rankings to make model selection, vendor, or build-vs-buy decisions — especially in code generation and QA applications where HumanEval and OpenBookQA are the go-to evaluation standards
- What breaks if this is true: Your internal evals that mirror public benchmarks become worthless as a differentiator; a model you licensed or fine-tuned for its “92nd percentile on HumanEval” may perform substantially worse on novel problems your users actually have
- Why it might not land: Enterprises with proprietary holdout test sets and rigorous agentic workflow evaluation pipelines are already insulated — this mainly bites teams who outsource their evaluation judgment to public leaderboards
- Watch for: A major model provider publishing a decontamination report or releasing contamination-adjusted scores alongside their next benchmark disclosure — that’s the signal that industry norms are actually shifting
Forecast as of 2026-11-17
By Q3 2027, at least one major AI benchmark consortium (HELM, BIG-bench successor, or equivalent) will adopt mandatory contamination-adjusted reporting as a submission requirement — but fewer than half of model releases on public leaderboards will comply, leaving the core problem structurally unresolved.
Source: Benchleaking Is Accelerating: Benchmark contamination has worsened sharply in the past month — from post-training down to pre-training base models — Yiwei Qin, Muhang Xie, Xuefeng Li, Mohan Jiang, Dayuan Fu, Tiantian Mi, Qipeng Guo, Pengfei Liu. https://arxiv.org/abs/2611.21315v1
