Your AI Agent Is Paying Frontier Prices to Relearn What You Already Did

Every time your computer-use agent books a flight, files an expense, or pulls a report, it’s burning inference tokens to rediscover a workflow you’ve performed dozens of times before. One researcher just built a deterministic compiler that turns your screen history into agent memory—and ran the numbers on exactly how much that redundancy costs you.

What happened

Nossa Iyamu built a zero-model pipeline that passively captures screen activity and compiles it into typed “activity frames”—structured, bounded episodes tagged with application, site, timing, input volume, and pointers back to raw data. The key claim: no model touches the data during compilation, so output is byte-identical and mechanically auditable. On a single professional’s 128,756-frame corpus across 51 active days, the compiler compressed a full day of raw capture into a prompt-ready context block 86× smaller in just 68 milliseconds. An agent reading that compressed block answered questions about the day at 98.4% accuracy (Wilson 95% CI: 91.7–99.7%), compared to 66–80% for an LLM summary of the same raw capture—and a mid-tier model reading the compiled block matched frontier model performance. The paper also introduces two previously unmeasured parameters for agent memory cost models: the Routine Overhead Ratio R (measured at 60–343×) and delegable routine recurrence, measured at 9.0% in-sample and 7.7% out-of-sample, implying a realistic all-fleet token ceiling near 8%. Compiled routines can then replay deterministically with the model entirely out of the loop, demonstrated live at zero model tokens on a guard-matched hit. The schema, compiler, and evaluation harness are open-source.

Cold read

This is a single-subject study—one professional, 51 days, one corpus—and the authors don’t attempt to obscure that. The 98.4% accuracy figure is evaluated against an “independent oracle,” but the paper doesn’t fully specify how that oracle was constructed or how contested the ground truth was; easy questions inflate accuracy numbers dramatically. The 7.7–9.0% recurrence rate is the paper’s most commercially consequential claim, but derived from one knowledge worker’s behavior—generalization to a five-person ops team, let alone an enterprise fleet, is pure speculation at this point. The 86× compression is real and impressive, but compression ratio is meaningless if the downstream agentic workflow can’t reliably trigger on the right compiled routine—the paper demonstrates a single live replay, not a reliability distribution. The cost savings from the Routine Overhead Ratio (60–343×) are modeled upper bounds, the authors say so themselves; actual savings depend entirely on how often your agents actually hit cacheable routines, which varies by use case in ways this paper cannot tell you.

What it means for you

• Signal maturity: 2/5 — Single-subject proof of concept with open tooling; not yet a validated product primitive
• Who gets hurt: Frontier model API vendors (OpenAI, Anthropic, Google) if routine replay genuinely displaces inference; also RAG infrastructure vendors whose pipelines assume model-in-the-loop summarization
• What breaks if this is true: The assumption that you need to pay frontier inference costs for repetitive agent tasks—the entire pricing model of “agent as a service” startups built on per-token billing starts to look like a leaky bucket
• Why it might not land: Routine recurrence at ~8% means 92% of agent actions still require live inference; the gains are real but narrow, and enterprise deployment requires solving privacy and multi-user screen capture problems the paper doesn’t touch
• Watch for: A second study with N > 5 knowledge workers showing recurrence rates above 10%, or a major agent platform (Cursor, Relay, Lindy) shipping a compiled-routine cache layer built on this schema

Forecast as of 2026-08-07

By Q3 2027, at least one funded agent-infrastructure startup will ship a production routine-caching layer citing this work or independently converging on the same architecture—but measured token savings in enterprise deployments will land below 5%, not the modeled 8% ceiling, because real-world task diversity is higher than a single professional’s corpus.

Source: Activity Frames: Deterministic Screen-Activity Compilation for Agent Memory and Replay — Nossa Iyamu. https://arxiv.org/abs/2608.05784v1