Your SEO Budget Is Burning. Agents Can’t Read Your Website.

You spent three years stuffing forums and listicles so AI would mention your brand. Congratulations: the agent found you, opened your site, and bounced because it couldn’t parse a single page. The optimization game just moved, and almost nobody’s stack is ready.

What happened

Finder, Elovic, and Shalev ran 37,927 simulated buyer journeys across 1,056 real businesses, testing what happens when an agentic workflow goes beyond surfacing a result and actually fetches and reads the business’s own site. They split businesses by “AX level” — essentially, whether the site is crawlable and legible to an agent — while controlling for fame, prior brand entity in LLMs, and two AEO proxies. The results are not subtle: agent-ready businesses had answers built from their own pages 78% of the time vs. 56% for non-ready peers, were clearly recommended 1.9x more often, and their site-grounded answers were 41% more accurate. The dominant failure mode isn’t hallucination — it’s omission: web-built answers about non-agent-ready businesses were 3.7x more likely to contain none of the facts the buyer actually asked for. Training knowledge barely matters anymore either way: only 7–10% of a finished answer draws from it regardless of site readability. The effect held across all four independent harnesses, though clear-recommendation rates varied sevenfold between stacks — a spread that should make anyone nervous about treating any single measurement as ground truth.

Cold read

The four harnesses show a sevenfold variance in clear-recommendation rates, which means the absolute numbers are highly stack-dependent — what “agent-ready” wins by on one platform may be marginal noise on another. The study matched businesses on fame and AEO proxies, but we don’t know the distribution of industries, site complexity, or geographic markets; a finding that “holds across all four harnesses” in one study cohort is not the same as a universal law. The paper defines “AX level” as a treatment variable, but the abstract doesn’t fully specify what makes a site agent-ready — meaning founders can’t directly operationalize the finding without more detail on the scoring rubric. There’s also a survivorship question: businesses with high AI visibility and readable sites may already be better-run organizations in ways that correlate with recommendation quality independent of crawlability. And 37,927 journeys across 1,056 businesses is a real dataset, but it’s a snapshot; agent architectures are changing fast enough that a six-month-old crawl protocol could already be outdated.

What it means for you

• Signal maturity: 3/5 — Large, controlled study with real businesses, but operationalization gaps and stack variance limit direct action
• Who gets hurt: Any B2B or considered-purchase business that invested heavily in off-site AEO content (forum seeding, listicles, third-party citations) while leaving their own site technically hostile to AI crawlers
• What breaks if this is true: The entire off-site content moat strategy — paying agencies to plant mentions across the web — becomes a rounding error if the agent can’t read your homepage when it arrives
• Why it might not land: Agent stacks are not standardized; a site optimized for one agent’s fetch behavior may still fail on another, and the sevenfold variance in the data proves this isn’t theoretical
• Watch for: Major agentic platforms (Perplexity, ChatGPT shopping, Google’s agent layer) publishing explicit crawlability specs or an equivalent of llms.txt adoption becoming a trackable industry metric — that’s when this moves from research finding to procurement checklist

Forecast as of 2026-09-30

By Q3 2027, at least two of the top five B2B SaaS analyst firms will publish “agent readiness” audits as a distinct service category, separate from traditional SEO or AEO offerings — and businesses that haven’t addressed basic agent-crawlability will show measurably lower share of voice in agentic purchase research flows compared to agent-ready competitors in the same category.

Source: AX is the New AEO — Ido Finder, Assaf Elovic, Gad Shalev. https://arxiv.org/abs/2609.34951v1