Someone Just Built a Weapon That Makes AI Lie in Your Name

Your RAG system’s citations are supposed to be proof. This paper shows one poisoned document can make the model confidently cite you for a claim you never made — and the correct answer is sitting right there in context, ignored. Citation trust isn’t a feature anymore. It’s an attack surface.

What happened

Researcher Guo Fuzheng introduces CiteShade, framed as the first “citation laundering” attack against Retrieval-augmented generation systems. The setup: an attacker controls a single source document in a multi-source retrieval pool. The goal isn’t just to corrupt the answer — it’s to make the model produce the wrong answer and attribute it to a different, trusted source that never supported it. The correct answer remains visible in context; the model ignores it anyway. On multi-source multi-hop QA benchmarks, the attack raises the wrong-answer rate from 0.01 to 0.68 — a 68x increase — and source deletion confirms the malicious document is the causal driver in every measured case. The attack exploits a model’s citation-proneness rather than its scale: the most citation-prone model tested hit a Citation Laundering Rate (CLR) of 0.84 under explicit instruction and 0.64 with no instruction at all. Critically, two obvious defenses — perplexity filtering and faithfulness vs. groundedness checking — are each shown to be insufficient. The authors propose a counterfactual defense that tests which source actually drove the answer, distinct from which source is merely cited. This maps directly onto the gap between citation rate vs. mention rate that RAG auditors already struggle to close.

Cold read

The benchmark is multi-hop QA — a controlled, structured environment that is cleaner than any production RAG deployment you’re running. Real retrieval pools are noisier, chunking strategies vary wildly, and re-rankers sit between retrieval and generation in most serious systems; none of that complexity is stress-tested here. The CLR of 0.84 is the ceiling on the most citation-prone model — the paper doesn’t tell us which deployed models that is, which matters enormously for threat modeling. “Source deletion confirms causality in every measured case” is a strong claim, but every measured case is a finite QA dataset, not an adversarial red-team across diverse domains. The counterfactual defense is proposed but not benchmarked for its own false-positive rate or latency cost — two things that will kill any defense in production before security concerns even get raised.

What it means for you

• Signal maturity: 2/5 — Lab result, single attacker-controlled source, no production validation
• Who gets hurt: Any company offering a RAG-powered answer product with cited sources — legal research tools, AI search, enterprise knowledge bases, anything marketing itself on source attribution as a trust signal
• What breaks if this is true: Your citations become a liability, not an audit trail; a competitor or bad actor seeds one document into your retrieval index and your system confidently cites your own trusted partner as the source of a false claim
• Why it might not land: Most production RAG pipelines use re-ranking, chunking, and access-controlled indexes that raise the bar for injecting a single controlling source; the attack surface shrinks dramatically if you don’t ingest open-web documents
• Watch for: A CVE-style disclosure or coordinated attack against a public AI search product where citation laundering is confirmed as the mechanism — that’s the moment this moves from academic to operational threat

Forecast as of 2026-09-15

By Q3 2027, at least one major RAG platform (Perplexity, a Microsoft Copilot product, or an enterprise search vendor) will ship an explicit “citation-support verification” layer in response to demonstrated citation laundering in the wild — not necessarily this paper, but this attack class. If that doesn’t happen, it will be because retrieval pipelines hardened against open-web injection first, rendering the threat moot for the most exposed products.

Source: CiteShade: Citation Laundering in Multi-Source Retrieval-Augmented Generation and Its Counterfactual Defense — Guo Fuzheng. https://arxiv.org/abs/2609.15660v1