Your Compressed Model Passed Every Test and Is Now Making Up Steps

You shipped a smaller, cheaper model. It aced perplexity, aced MMLU, aced your fidelity checks. It is also quietly inventing procedure steps that were never in the instructions — and none of your quality gates caught it. This isn’t a theoretical edge case. It’s a systematic blind spot in the compression pipeline that every cost-conscious AI team is running right now.

What happened

Kennedy & Kennedy tested “gently compressed” models — the kind that clear your standard acceptance stack — across three model families and found a specific failure mode: when running a standard operating procedure (SOP) as an agentic workflow, compressed models invent procedure steps that were never in the instructions. The critical detail is operator-specificity: coherent low-rank (SVD) truncation reliably induces the hallucinated steps; magnitude pruning matched to the same perplexity level does not. The governing variable isn’t how much damage compression does — it’s the coherence of the compression error times its rate. Ordinary quality guards (perplexity, MMLU, data-free faithfulness vs. groundedness probes) are blind to this axis by construction, because they’re fidelity oracles — they measure what you asked them to measure, and this failure lives orthogonal to that. The authors also deliver a practical fix: a two-axis statistic (coherent-fraction × error-rate) that flags the failing builds with fixed thresholds across all three architectures, data-free, and without needing labeled task data.

Cold read

This is a two-author paper with a pre-registered canary design, which is credibility-positive — but “three model families” and “three architectures” are doing a lot of work without naming which ones, and the abstract doesn’t quantify the rate of invented steps, only that the canary CI-fails. The dissociation between perplexity-pass and canary-fail is the paper’s strongest claim, but we don’t know the effect size — a model that invents one spurious step per hundred SOP executions is a very different product risk than one that invents steps on every run. The “operator-specific” framing also needs scrutiny: SVD truncation being the culprit is plausible, but this is a narrow compression method; if your team uses quantization (the most common cost-reduction tool in 2026), this paper says nothing about you. Finally, the proposed two-axis screen is data-free but needs independent replication before you treat it as a certified gate — right now it’s a promising heuristic from the same authors who defined the problem.

What it means for you

• Signal maturity: 3/5 — Mechanism is credible and pre-registered, but effect sizes and model identities are unpublished
• Who gets hurt: Ops teams running LLM-driven process automation (RPA replacement, compliance workflows, customer onboarding SOPs) on SVD-compressed models to cut inference cost
• What breaks if this is true: Your model audit log shows the model “completed” a procedure that included steps your legal or compliance team never approved — and your quality gate signed off on the build that caused it
• Why it might not land: Most production cost-cutting in 2026 is quantization and distillation, not SVD truncation; if your compression stack doesn’t use low-rank decomposition, this specific failure mode may not apply
• Watch for: Any inference provider or model vendor advertising “low-rank efficient” or “SVD-compressed” variants of foundation models as drop-in cost reducers — that’s the exact build profile this paper flags

Forecast as of 2026-07-31

By Q1 2027, at least one major model-serving framework (vLLM, llama.cpp, or a cloud provider’s managed inference tier) will add a coherence-fraction screen or equivalent to its compression validation pipeline, citing this or a replicating paper — or this finding will be quietly absorbed into “don’t ship SVD-only compression for agentic tasks” tribal knowledge with no tooling change.

Source: Fidelity Is Not Safety: Gently-Compressed LLMs Pass Every Data-Free Quality Guard Yet Invent Procedure Steps in Agentic Execution — I. Kennedy, T. Kennedy. https://arxiv.org/abs/2607.28196v1