Two AIs Fight Each Other So Your Model Learns to Actually Think

What if the biggest bottleneck in reasoning AI wasn’t compute or data — it was that models have never once been graded on how they think, only what they answer? One researcher just ran that experiment. The results are uncomfortable for everyone selling “better reasoning” through scale alone.

What happened

Standard reinforcement learning from verifiable rewards — the technique powering today’s frontier reasoning models — scores only the final answer. The chain of thought in between? Graded by nobody. Vladislav Beliaev’s Agon system fixes this by making two models fight: one drafts a solution, the rival reads the draft and then solves the same problem, and both are rewarded purely for out-scoring each other. No process labels, no LLM-as-judge reward model, no human annotation — the competitive dynamic itself forces implicit grading of chain-of-thought reasoning. The benchmark numbers on DeepMath (hard split) with Qwen3 are striking: Agon doubles GRPO’s pass@1, and that gain is roughly eight times larger than what you’d get from an untrained Mixture-of-Agents pass over the same base model. The effect replicates on competitive-programming code and across model families (Qwen3.5, Gemma 4). At inference the pair runs as a two-stage cascade — draft, then read-and-answer — so you’re paying for two model calls, not one.

Cold read

This is a single-author preprint with no peer review yet — treat the numbers as “promising lab result,” not “production-ready technique.” The gains are measured on math and competitive programming, two domains with unambiguous verifiable rewards; whether the same dynamic holds for messier real-world tasks (legal reasoning, product strategy, customer support) is entirely unproven. The two-model cascade at inference doubles your token and latency costs at minimum, which is a non-trivial operational constraint the abstract glosses over. The “behaviorally different” requirement for the two rival models is left vague — it’s unclear how sensitive results are to model pairing, and whether benchmark contamination on DeepMath inflates the reported gains. Finally, “the next step is to let them reason together in latent space” is a research horizon, not a roadmap — the current system communicates only in text, which caps how much the two models can actually share.

What it means for you

• Signal maturity: 2/5 — single preprint, no ablations published, no third-party replication yet
• Who gets hurt: Fine-tuning shops selling “better reasoning” through GRPO-style RL alone; their moat just got a credible challenger with a structurally different training signal
• What breaks if this is true: The assumption that scaling a single model’s RL training is the efficient path to hard-problem reasoning — two-model competitive loops may be the better lever, reshuffling how training compute is allocated
• Why it might not land: Inference cost doubles; most production reasoning tasks lack the clean verifiable reward signal that makes this work; pairing models correctly may require more art than the paper admits
• Watch for: A major lab (Qwen team, Google DeepMind) publishing a multi-model competitive RL result with ablations and production benchmarks — that’s the signal this technique is being taken seriously beyond one researcher’s repo

Forecast as of 2026-07-09

By Q1 2027, at least one top-5 open-source reasoning model leaderboard entry will explicitly credit a competitive cross-model or adversarial RL training regime in its model card — but the two-stage inference cascade will be collapsed or distilled away before deployment, because operators won’t pay 2x inference cost for a training trick.

Source: Agon: Competitive Cross-Model RL with Implicit Rival Grading of Reasoning — Vladislav Beliaev. https://arxiv.org/abs/2607.07690v1