AI Still Makes Up Half Your Citations — Even the Good Models Lie About Authors

You handed your research assistant a task it cannot reliably do. Across 26 models tested right now, in September 2026, more than half of all biomedical references produced were pure fabrication. The best model on the market was fully correct less than 55% of the time. This is not a 2023 problem that got solved — it is today’s problem.

What happened

Topaz et al. prompted 26 large language models from eight developers (spanning 2023–2026 releases) to complete a missing reference for each of 69 biomedical passages across ten domains — a narrow, concrete task with objectively verifiable answers. They classified outputs as verifiable, partial matches, fabricated, or declined, and scored “fully correct” only when journal, year, and authors all matched the real paper. The headline number: 55.4% of all responses were fabricated, and only 14.9% were correct in every bibliographic field. The range was brutal — Ministral 3B fabricated 98.4% of references, while Claude Opus 4.8 hit a low of 10.2% fabrication by refusing to answer 52.1% of the time (a strategy, not a solution). The most telling result: Claude Opus 4.6 and Claude Sonnet 4.5 produced nearly identical rates of verifiable references (77.6% vs. 76.6%), yet Opus named authors correctly in 78.7% of cases versus Sonnet’s 28.7% — proving that finding the real paper and accurately reporting its metadata are two entirely different failure modes. This is hallucination operating at a fine-grained level that coarse benchmarks miss entirely, and it is precisely the kind of error that factual consistency evaluations need to catch. The 2026-released models improved somewhat — 35.3% fabrication rate vs. 55.4% overall — but their full-correctness rate of 31.8% still means you’d be wrong on two out of three citations without checking.

Cold read

This is a single-task study on one narrow use case — reference completion in biomedical text — and the results should not be extrapolated to claim LLMs are uniformly unreliable across all factual tasks. The 69-passage dataset is small enough that performance variance across domains could be significant noise, not signal. “Correct in every field” is a strict composite metric that penalizes models heavily for any single metadata error; a model that gets the paper right but lists “2021” instead of “2022” fails the same as one that invents the paper entirely — which obscures the practical severity gap between those two outcomes. Claude Opus 4.8’s best-in-class fabrication rate of 10.2% is largely purchased by refusing to answer more than half the time; if your use case requires an answer, that refusal rate is its own operational cost, not a free win. Finally, the study uses zero-shot prompting with no retrieval-augmented generation — the most obvious mitigation that any production biomedical writing tool would deploy — so the numbers here represent a floor of bad behavior, not the ceiling of what’s possible with proper architecture.

What it means for you

• Signal maturity: 4/5 — Large sample of current models, objective ground truth, reproducible methodology
• Who gets hurt: Healthtech and life-sciences startups shipping AI writing assistants, literature review tools, or clinical documentation products that pass LLM-generated citations to end users without a verification layer
• What breaks if this is true: Any compliance or regulatory argument that “the model finds real papers” is sufficient due diligence — metadata errors mean even verifiable references require human review, which destroys the unit economics of most AI-assisted research products
• Why it might not land: RAG-powered pipelines that retrieve from PubMed or similar indexed databases before generating references would be expected to dramatically outperform these bare-prompt results; the study does not test that architecture, so vendors will correctly say “our product doesn’t work this way”
• Watch for: FDA or IRB guidance specifically addressing LLM-generated citations in clinical research submissions — that’s the moment this academic finding becomes a liability event

Forecast as of 2026-09-15

By Q2 2027, at least two major biomedical AI writing platforms will publicly add mandatory citation-verification layers (either human-in-the-loop or automated PubMed cross-checks) and cite regulatory pressure or institutional customer requirements as the driver — not voluntary quality improvement.

Source: Biomedical Reference Generation Remains Unreliable across 26 Large Language Models — Maxim Topaz, Zhihong Zhang, Nir Roguin, Pallavi Gupta, Zichao Li, Laura-Maria Peltonen. https://arxiv.org/abs/2609.14988v1