Open-Source Bots Now Click Through Your Software Better Than Most Contractors

A Chinese academic lab just hit 68.7% on OSWorld — the benchmark that makes enterprise automation VCs sweat at night. The headline reads like a Salesforce acquisition thesis. Read the footnotes first.

What happened

The Tsinghua team behind ScaleCUA attacked two known chokepoints in training agentic workflow systems: you can’t get enough verifiable training tasks, and online reinforcement learning burns compute wastefully. Their answer is a two-part framework. On the data side, VeriGen uses iterative Docker interactions and a multi-agent orchestration feedback loop to auto-generate tasks — scaling to 100+ concurrent agent workers and producing 24,000+ verifiable tasks, of which ~3,000 cleared a high-quality bar for RL training. On the training side, Frontier Sampling allocates rollout budget only to tasks sitting at the current learning edge (not easy wins, not impossible reaches), while Visual Context Segmentation — a sliding window over recent visual state — delivered a 2.83× training speedup over the naive step-wise approach. The result: 68.7% on OSWorld and 54.0% on ScienceBoard, both claimed as open-source state-of-the-art. Code, models, and datasets are public on GitHub.

Cold read

OSWorld and ScienceBoard are controlled benchmark environments — Docker-sandboxed, well-defined success criteria, reproducible resets. Real enterprise desktops are none of those things: legacy apps, dynamic layouts, modal popups, VPN-gated SSO, and workflows that span 14 browser tabs and a Citrix session. A 68.7% score on a curated benchmark does not translate to 68.7% task completion on your Salesforce-to-NetSuite reconciliation workflow. The 3,000 “high-quality RL tasks” out of 24,000+ generated also deserves scrutiny — a ~12.5% yield rate suggests VeriGen is producing a lot of garbage that had to be filtered, and the paper doesn’t detail what benchmark contamination controls were applied to ensure OSWorld test cases didn’t bleed into synthesis. The 2.83× speedup is a training-efficiency number, not an inference-efficiency number — it makes the research cheaper to replicate, but it doesn’t make the deployed agent faster or cheaper for your users. Finally, “open-source state-of-the-art” is a bracket that excludes Anthropic’s Claude Computer Use and whatever OpenAI is shipping — the relevant commercial competition is explicitly not in this comparison.

What it means for you

• Signal maturity: 3/5 — benchmark performance is real but production gap is enormous
• Who gets hurt: Series A RPA startups (UiPath competitors, Robocorp-style players) pitching “no-code automation” — the floor just got pulled from under their moat-by-complexity argument
• What breaks if this is true: Enterprise software vendors who monetize complexity as a switching cost; if agents can navigate arbitrary GUIs without custom integrations, the API-tax business model weakens
• Why it might not land: Docker-sandboxed training tasks don’t generalize to stateful, permission-gated, multi-session enterprise environments — the failure modes in production are qualitatively different from anything in a 3K-task training set
• Watch for: Any company reporting >60% autonomous task-completion rates on unscripted real-world workflows — that’s the signal that benchmark gains are transferring; right now nobody credible is publishing that number

Forecast as of 2026-07-14

By Q2 2027, at least one well-funded CUA startup will publicly claim a ScaleCUA-derived or architecturally similar model exceeds 75% on OSWorld — but no tier-1 enterprise vendor will have shipped a product citing real-world autonomous completion rates above 50% on unstructured workflows, and that gap will be the defining tension in the space.

Source: SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL — Bowen Lv, Xiao Liu, Yanyu Ren, Hanyu Lai, Bohao Jing, Hanchen Zhang, Yanxiao Zhao, Shuntian Yao, Jie Tang, Yuxiao Dong. https://arxiv.org/abs/2607.11185v1