Your AI Agent’s Memory Is a Single Lie Away From Collapse

Someone tells your AI agent one false thing. No tricks, no special code, no elaborate attack. Just a plain sentence. Accuracy drops from 85% to 30%—and stays there, session after session, because the lie is now in memory. This paper measured that. The number is real.

What happened

Researcher Arulnidhi Karunanidhi tested what happens when you poison the persistent memory of an agentic workflow with nothing fancier than a plainly worded false statement—no special triggers, no retriever tricks, no prompt injection scaffolding. Poisoning just 1.2% of a LongMemEval corpus drove agent accuracy from 0.850 down to 0.300. The paper then tested two popular defenses and found both broken. A four-stage write-time content screening pipeline caught indirect prompt injection at 0.832 recall—but rejected zero of 360 poisoned memories, because a false factual claim looks syntactically identical to a true one without external grounding. Provenance-weighted retrieval-augmented generation fared no better: the default shipped weight was statistically indistinguishable from no defense at all (p=0.80). A stronger provenance weight did recover some accuracy in a mixed-corpus setting (0.317 → 0.700), but when the legitimate answer itself came from an untrusted source, evidence recall collapsed to zero and accuracy hit 0.0417—worse than random. The paper argues this is a fundamental tradeoff with no workable middle setting, and recommends “bounded occupancy constraints” at retrieval time instead.

Cold read

This is a single-author paper with a single benchmark (LongMemEval), and the poisoning scenario assumes an attacker who can already write into the agent’s memory store—a non-trivial precondition that the abstract doesn’t fully bound. The 1.2% poisoning rate sounds tiny, but the paper doesn’t report how that specific rate was chosen or whether the effect saturates at lower levels, which matters enormously for real threat modeling. The content screening pipeline being tested may not represent what production systems actually ship; a 0.832 recall on injection detection is itself a debatable baseline. The proposed alternative—bounded occupancy constraints—is argued, not demonstrated; no accuracy numbers are given for it. This is a well-structured problem statement, not a solved one.

What it means for you

• Signal maturity: 2/5 — attack is demonstrated, defense is not
• Who gets hurt: Any startup shipping agent memory features where external users, third-party tools, or scraped content can contribute to the memory store—think CRM copilots, research agents, customer-facing AI assistants with session persistence
• What breaks if this is true: A single bad document ingested by your sales or support agent poisons all future sessions that touch similar topics; your factual consistency guarantees to enterprise customers are unenforceable
• Why it might not land: Most production memory systems already use some form of source allowlisting or human-in-the-loop ingestion, which reduces the realistic attack surface well below what this benchmark models
• Watch for: Enterprise AI vendors quietly adding memory-write audit logs and source-gating as a contractual requirement—that’s the signal this finding is hitting procurement conversations

Forecast as of 2026-08-24

By Q2 2027, at least two major agent-platform vendors (e.g. in the Salesforce Agentforce / Microsoft Copilot Studio tier) will publish explicit memory-write policies or attestation requirements in their security documentation, directly citing the class of attack this paper describes—or a closely related CVE will be published against a commercial agent memory implementation.

Source: Utility Under Attack: Agent Memory Poisoning and the Limits of Content Screening and Provenance Ranking — Arulnidhi Karunanidhi. https://arxiv.org/abs/2608.21230v1