The Cache That Remembers Who Poisoned It — and Stays Silent

Shared inference infrastructure just developed a stealth backdoor, and it’s hiding in plain sight. An attacker never needs to touch your users’ prompts — they already won, the moment their context got cached. Welcome to KV Cache Hijacking.

What happened

Researchers from arXiv (Yichi Zhang et al.) have documented and weaponized a structural vulnerability in a new class of LLM serving optimizations called position-independent KV cache reuse. Unlike traditional prefix-based caching — which requires exact token and position matches — these newer systems let inference engines reuse cached key-value states whenever identical text chunks appear anywhere in a sequence, regardless of position. The efficiency gain is real. The attack surface is also real. The authors show that because a KV cache encodes the context in which it was originally computed — not just the token string — an attacker can craft a malicious prefix, compute KV states for a subsequent innocent-looking chunk of text, and then let that poisoned cache entry propagate. Any future user whose query happens to pull that cached chunk gets the attacker’s context silently injected into the model’s reasoning, with zero attacker-controlled text visible in their input. This is a form of prompt injection vs. jailbreak that operates one layer below the prompt itself. Their framework, HIJACKKV, achieves a 94% average success rate in a single attempt, holds up when cache hit rates are as low as 10%, survives 50% recomputation rates, persists across multi-turn conversations, and transfers across models in black-box settings. If you’re building on shared agentic workflow infrastructure or multi-tenant LLM serving, that number should make you put down your coffee.

Cold read

The 94% success rate is striking, but the paper does not disclose which specific models were tested, what tasks were targeted, or how the “success” metric was defined — the abstract is silent on all three, which matters enormously for extrapolation. Position-independent KV reuse is not yet a default in most production serving stacks; it’s an optimization present in select research systems and early commercial deployments, so the population of currently-vulnerable systems may be small. The attack also requires the adversary to seed a cache entry that later gets retrieved by a victim — in practice, achieving reliable retrieval of a specific malicious chunk across a multi-tenant system with unpredictable routing is non-trivial, and the paper’s 10% hit rate condition, while described as “realistic,” may still flatter controlled lab conditions. Black-box transfer is claimed but not quantified with numbers in the abstract; “effective” is doing a lot of work there. Finally, the authors note they provide “design insights for secure KV reuse systems,” which suggests mitigations exist — meaning this is a solvable engineering problem, not a fundamental architectural doom.

What it means for you

• Signal maturity: 2/5 — Real vulnerability class, premature threat for most production stacks today
• Who gets hurt: Multi-tenant LLM API providers and companies building shared inference layers for multi-agent orchestration pipelines — anyone running position-independent caching at scale with mixed user populations
• What breaks if this is true: Trust in shared inference infrastructure collapses; enterprises will demand tenant-isolated cache pools, driving up inference costs and eroding the unit economics of shared serving
• Why it might not land: The attack requires adversarial cache seeding in a shared pool — a precondition that most operators can neutralize by isolating caches per tenant, which many already do for compliance reasons
• Watch for: Major inference providers (Together, Fireworks, Anyscale-style platforms) publishing explicit cache isolation guarantees, or a CVE-style disclosure process emerging for LLM serving systems — either would confirm the threat is being taken seriously operationally

Forecast as of 2026-07-23

By Q1 2027, at least one major LLM API provider will publish explicit documentation of their KV cache isolation architecture in direct response to this vulnerability class — either proactively or following a demonstrated incident. If no provider mentions KV cache isolation in security documentation within 12 months, this paper will have been filed under “interesting but inert.”

Source: HijackKV: New Threat in Position-Independent KV Cache Reuse — Yichi Zhang, Zhiqi Wang, Huan Zhang, Yuchen Yang. https://arxiv.org/abs/2607.19957v1