Your Phone Now Has an AI Co-Pilot. Try Not to Panic Yet.

Agents that operate your phone like a human — tapping, swiping, navigating real apps — just got a credible open-model training recipe. The headline number is 83.2% task success on a standard benchmark. Before you assume your mobile workflow automation startup just became obsolete, read the fine print.

What happened

A team from multiple Chinese institutions released PhoneBuddy, an open model and training recipe designed to run agentic workflows on real Android phones. The core problem they tackled is a genuine hard one: real devices are slow, stateful, side-effectful, and nearly impossible to reset at training scale. Their solution is a hybrid: collect supervised fine-tuning (SFT) trajectories in both real apps and a reconstructed mock environment called PhoneWorld, then apply reinforcement learning (RL) in either real apps, or a mix of real and mock. On a 150-task human evaluation on actual phones, task success climbed from 36.67% (post-SFT) to 40.67% (real-app RL) to 45.33% (mixed RL). On the AndroidWorld benchmark, the same progression hits 60.3% → 77.2% → 83.2%. The key finding: mock-app training is additive, not a substitute — real-app RL alone outperforms SFT, but mixing in mock data outperforms both. Gains are concentrated in single-app and mini-app tasks; long, multi-step cross-app workflows remain the unresolved frontier.

Cold read

The 83.2% AndroidWorld number is the one that will travel, and it’s also the one most disconnected from your actual deployment reality. The 150-task human evaluation on real phones — the harder, messier, more honest test — lands at 45.33%, meaning the model fails more than half the time on tasks a human evaluator constructed and watched. That’s not a product; that’s a research prototype. The paper is admirably honest about cross-app workflows being an “important open challenge,” but that’s precisely what enterprise and consumer automation use cases look like — multi-step, multi-app, stateful sequences. Additionally, the gap between AndroidWorld performance and real-device performance (83.2% vs. 45.33%) raises a quiet benchmark contamination concern: structured benchmarks may not capture the chaotic surface area of apps your users actually run. Finally, “open model” here means open weights and recipe, not a plug-and-play SDK — operational cost and integration complexity for a startup remain non-trivial.

What it means for you

• Signal maturity: 2/5 — promising training methodology, but real-device success rates aren’t product-grade yet
• Who gets hurt: Startups selling RPA-style mobile automation at premium prices with proprietary “AI magic” as the moat — the recipe is now open
• What breaks if this is true: The assumption that reliable phone agents require closed, expensive frontier models; open fine-tuned models can close a significant fraction of the gap
• Why it might not land: 45% real-device task success disqualifies most production use cases where failure has user-facing or financial consequences; the cross-app workflow wall is where real enterprise value lives
• Watch for: A third-party replication that hits >60% on the real-device 150-task eval — that’s the threshold where “research” starts threatening “product”

Forecast as of 2026-06-23

By Q2 2027, at least one well-funded mobile-agent startup will publicly benchmark against PhoneBuddy’s recipe and report real-device task success above 60% on a comparable human evaluation — but cross-app workflows will still be the cited limitation, and no production deployment at scale will have cleared the reliability bar for unsupervised consumer use.

Source: Training Open Models for Agentic Phone Use — Zhengyang Tang, Xin Lai, Pengyuan Lyu, Xinyuan Wang, Tianyi Bai, Chenxin Li, Yiduo Guo, Huawen Shen, Yuxuan Liu, Junyi Li, Zhengyao Fang, Yang Ding, Yi Zhang, Weinong Wang, Xingran Zhou, Liang Wu, Fei Tang, Sunqi Fan, Shangpin Peng, Zheng Ruan, Anran Zhang, Benyou Wang, Ji-Rong Wen, Rui Yan, Chengquan Zhang, Han Hu. https://arxiv.org/abs/2606.23049v1