AI Agents Are Writing Your Dependencies — And Nobody’s Counting Them Right

ByJohn

AI Agents Are Writing Your Dependencies — And Nobody’s Counting Them Right

The open-source supply chain is being quietly rewritten by coding agents, and every adoption metric you’ve seen is off by at least 30x. A new census of 180 million repositories just blew up the way the industry measures AI code penetration — right as founders are making bets on that data.

What happened

Figure 2: Monthly AI-attributed commit volume by agent across V2412–V2604. Total agent activity rises from 75K commits in Dec 2024 to 320K by mid-2025.
Figure 2: Monthly AI-attributed commit volume by agent across V2412–V2604. Total agent activity rises from 75K commits in Dec 2024 to 320K by mid-2025.

Researchers Khosravani and Mockus built a multi-method detection framework across World of Code — a corpus of 180M+ Git repositories — to count how many commits are actually being generated by agentic workflow tools like Claude Code, Codex, Cursor, OpenHands, and Aider. They layered four signals: configuration-file scanning, commit-message analysis, author-identity matching, and bot-signature lookup. The punchline is brutal: the single method most adoption studies rely on — bot-account lookup — recovers only 28,154 Claude Code commits out of 850,157 detected by the full framework. That’s a 3.3% recall rate, a 30x gap, meaning single-signal prevalence estimates are biased low by at least that factor. Claude Code alone accounts for 886,122 commits across 17,295 projects in the December 2024–April 2026 snapshot window, generating over 320,000 AI-attributed commits per month across all agents. Crucially, the paper also finds that a pull-request census (the other common measurement channel) misses 79% of commit-detected Claude Code adopters and essentially all Codex adopters — the two measurement channels capture nearly disjoint populations.

Cold read

The 30x recall gap is real and damning for prior studies, but it’s a floor, not a ceiling — the paper’s own multi-method framework may still miss agent activity that leaves no detectable trace at all, so “at least 30x” is doing heavy lifting. The 495 hand-validated labels sound rigorous but cover a framework classifying activity across 180 million repositories; per-cell precision with Wilson confidence intervals is the right move, yet sparse cells in that label set could still skew counts for less-common agents. The finding that “work profile follows deployment and detection mode rather than the tool itself” is interesting but potentially circular — you find maintenance work with commit-level tools because commit-level tools are used for maintenance; this doesn’t tell you whether the agents are driving that pattern or the humans choosing those tools. The paper also covers December 2024 to April 2026, a period of explosive and uneven adoption; these numbers are likely already stale in absolute terms. Finally, “open-source repositories” is a highly self-selected population — it tells you almost nothing about the AI-generated content volume inside private enterprise codebases, which is where most commercial software actually lives.

What it means for you

  • Signal maturity: 3/5 — methodology is solid and validated; the corpus is genuinely massive, but coverage gaps are acknowledged and the window is closing fast
  • Who gets hurt: Security vendors, compliance tools, and OSS dependency auditors whose threat models assume they can detect AI-generated code through bot accounts or PR metadata — they’re flying blind on the other 97%
  • What breaks if this is true: Any founder building AI content detection or provenance tooling on top of single-signal heuristics has a product that’s structurally wrong, not just slightly off — the detection gap is an order of magnitude
  • Why it might not land: Enterprises may not care about AI code provenance in OSS dependencies until a high-profile supply chain incident forces the issue; absent that catalyst, the market for better detection stays small
  • Watch for: A CVE or supply-chain compromise traced specifically to an agentic AI commit pattern — that’s the event that turns this academic finding into a regulatory requirement overnight

Forecast as of 2026-06-24

By Q2 2027, at least one major package registry (npm, PyPI, or Maven Central) will announce a policy or tooling initiative specifically targeting AI agent commit attribution — driven by the visibility gap this and similar studies expose. If no such initiative exists by then, the supply-chain risk framing will have failed to move institutional buyers despite the data.

Source: Detecting AI Coding Agents in Open Source: A Validated Multi-Method Census of 180 Million Repositories — Arsham Khosravani, Audris Mockus. https://arxiv.org/abs/2606.24429v1

Similar Posts