Your AI Agent Is Too Slow to Click the Button Before It Vanishes

Computer-use agents are getting the right answer — just 500 milliseconds too late, after the popup has already closed. A new paper puts a number on this humiliation: baseline success rate of exactly 50% in time-critical GUI windows. A pre-computation trick pushes that to 79%. The question is whether the trick is generally useful or a narrow parlor act.

What happened

Figure 2: AAPT dataflow. Off the critical path, the slow planner compiles and validates a policy tree during the prep phase. On the critical path, event-time execution is a change gate, one low-token

Researchers at identified a specific, underappreciated failure mode in agentic workflow systems: GUI agents that correctly reason about what to do but lose the race against transient UI events because autoregressive decoding is simply too slow. Their fix, called Adaptive Anticipatory Policy Trees (AAPT), sidesteps the problem by doing the heavy large language model decoding work during idle screen time — building a bounded conditional decision tree in advance, keyed to observable screen-state guards. When a time-sensitive event fires, a lightweight observer pattern-matches to a pre-approved branch and executes immediately, without generating new text. In paired McNemar-tested trials, AAPT lifted success rate from 0.50 to 0.79 (p = 1.8 × 10⁻³) inside a contested decision window, while producing zero incorrect actions. Both open-loop and predict-and-replan baselines achieved zero successes in those same windows — they were still decoding while the moment passed. The effect replicated on an independent general-purpose multimodal model across 126 paired trials (p = 4.9 × 10⁻¹³), and ablations fingered branch routing — not planning or observer speed — as the actual causal bottleneck.

Cold read

The 0.50 baseline is a cherry-picked regime: the authors specifically targeted a “contested decision window” sized to expose latency failures, not a representative sample of real-world computer-use tasks. On an external benchmark, AAPT merely matches the reactive baseline overall — meaning outside the narrow latency-critical slice, you’re buying nothing new. The whole system only works when “candidate actions can be enumerated in advance” — the authors say so directly — which excludes the large category of open-ended or context-dependent tasks where you don’t know what branches to pre-build. The pre-registered oracle probe also rejected the authors’ original hypothesis about what was causing failure, which is honest reporting but a reminder that even the researchers didn’t fully understand their own system going in. Finally, this is a single lab’s controlled paired-trial design; there’s no external validation against production-grade computer-use deployments at real companies.

What it means for you

Signal maturity: 2/5 — Reproducible lab result, but external benchmark shows no net gain outside narrow latency windows
Who gets hurt: Teams shipping GUI automation agents for time-critical enterprise workflows — think form-filling under session timeouts, auction bidding UIs, or transient approval modals
What breaks if this is true: The standard reactive agentic AI vs AI agent architecture — “observe, decode, act” — has a structural speed floor that no prompt engineering or faster hardware alone fixes
Why it might not land: The enumerable-actions assumption is brutal. Most interesting GUI automation problems involve dynamic, unpredictable screens where pre-compiling a bounded policy tree is impossible or prohibitively expensive
Watch for: Whether any major computer-use platform (Anthropic Computer Use, OpenAI Operator, or open-source equivalents) ships a “prefetch policy” or “anticipatory branch” feature; that would be the signal this moved from paper to product

Forecast as of 2026-08-01

By Q2 2027, no major commercial computer-use agent platform will have shipped a production feature explicitly based on pre-compiled policy trees; the enumerable-actions constraint will keep this confined to narrow, scripted enterprise automation niches rather than general GUI agents.

Source: Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees — Zihan Dong, Rui Qian, Qishi Zhan, Dongshen Peng, Kaixin Li, Yu Li. https://arxiv.org/abs/2607.28399v1