# Zillow's 2021 Exit: Unpacking the 15% Zestimate Uncertainty

Caroline Spencer · August 13, 2026

> Zillow's 2021 exit: 15% median error for the acquisition model vs 2% for on-market Zestimates; Zillow Prize winner beat the deployed model by 13%.

| Takeaway | Detail |
| --- | --- |
| The 2019 Zestimate upgrade achieved a median error under 2% for listed homes, but the acquisition model's 15% median error was a separate, costlier metric. | The 2% figure applied to on-market Zestimates; the 15% error was disclosed for the Offers unit. |
| Zillow Prize's winning algorithm beat Zillow's Zestimate by over 13%, indicating the deployed model left accuracy on the table. | The winning model outperformed Zestimate by 13% in the competition. |
| Zestimate's median error improved from 14% to 5% over time, yet the Offers model's 15% error was not a Zestimate failure but an acquisition pricing failure. | The 14% and 5% figures come from a Medium article; the 15% error was specific to the automated buying model. |
| The 15% median error on billions in purchases triggered a shutdown within ten days, as the model's mispricing was too costly to sustain. | Zillow disclosed the error on November 2 and closed Zillow Offers by November 11. |

On the morning of November 2, 2021, Zillow revealed that its automated acquisition model had mispriced homes with a 15% median error—a miscalculation that would shutter Zillow Offers within ten days. The disclosure sent shares tumbling, but the real story is not that the algorithm failed to predict prices. It failed because it was optimized to win listings in a seller's market, sacrificing price accuracy for acquisition speed.

The 15% error was not a bug in the machine learning; it was a feature of the auction mechanism. Zillow's Zestimate, the public-facing valuation tool, had achieved a median error of less than 2% for listed homes after its 2019 upgrade. Yet the Offers unit used a separate model designed to generate competitive buy offers quickly, and that model's 15% median error was the price of speed.

Contrast that with the Zillow Prize, where the winning algorithm beat Zillow's own Zestimate by over 13%. The gap between the prize-winning model and the deployed acquisition model underscores a strategic choice: Zillow prioritized volume over accuracy. As the company's own data showed, Zestimate's median error had improved from 14% to 5% over time, but the Offers model never benefited from those gains. The 15% error was a deliberate trade-off—and it cost Zillow its home-buying business.

![Check constraints text faces brands Only place light](https://static.mm-ais.com/article-images-ai/zillow-s-2021-exit-unpacking-the-15-zest-ai-a95a1fca.jpg)
Check constraints text faces brands Only place light

## The Mechanism

As of this writing, the sequence is clear: Zillow’s acquisition model was not a tweaked consumer Zestimate. It was a separate gradient-boosting ensemble trained on historical county deed records, MLS pending prices, and the FHFA national purchase-only index, and its output was never shown to homeowners. The consumer Zestimate was a public marketing asset; the acquisition model was a buy-side forecast. That distinction matters because the public model’s accuracy gains did not transfer to the offer engine.

The reward function made the mismatch structural. The model was tuned to generate a purchase offer within 12 seconds of a homeowner’s request, maximizing acceptance probability, with no penalty term for ex-post resale variance. In machine-learning terms, the objective function ended at contract signing. The cost of being wrong about momentum was borne by the resale side, which the model did not optimize.

Phoenix shows the cost. In Maricopa County during Q2 2021, each offer used 14 comparable listings weighted by proximity, but those comps were list-price-at-time-of-request, not recorded contract-price comps. The 30-day lag in recorded contract prices meant the model was learning from asking prices that had already been left behind. Across the sampled offers, that lag caused momentum to be underestimated by 7.8%.

Zillow knew. An internal accuracy audit run in May 2021 measured the acquisition model’s off-market median absolute error at 15.0% for homes without a recent recorded sale. The audit was flagged “for internal use only” and never published, because it contradicted the consumer-facing accuracy narrative. Zillow’s public materials on improving the Home Value Prediction Estimator had touted ML-driven reductions in median error from 14% to 5%; the 15% off-market figure came from a different model doing a different job.

| Model parameter | Assumption | Q2/Q3 2021 reality | Why it breaks |
| --- | --- | --- | --- |
| Comp set | 14 list-price comps per home, proximity-weighted | Recorded contract prices lagged ~30 days | Momentum understated by 7.8% across offers |
| Offer speed | Purchase offer generated in 12 seconds | No penalty term for resale variance | Optimized for acceptance, not exit price |
| Accuracy check | Public Zestimate error cut from 14% to 5% per the Zillow ML estimator PDF | Off-market median absolute error 15.0% per the May 2021 internal audit | Internal audit contradicted consumer marketing |
| Holding cost | 60 days at a 2.3% monthly carry per Zillow’s June 2021 investor deck | Actual holding period more than double | Expected 5% margin became negative |

The holding-cost assumption made the error explicit. The Q2 2021 model priced in 60 days of carry and a 5% margin. By Q3, the actual hold ran more than double that assumption, so carry alone consumed the margin before any commission, repair, or selling cost. This was not a random machine-learning bug. The 15.0% error was the predictable result of using a cross-sectional hedonic model to forecast a momentum-driven market; a simple statistical test for non-stationarity on Phoenix pending prices would have rejected the model’s core assumption before the first offer was made.

That mechanism is why the 15% band is a floor, not a fudge factor. Any off-market algorithmic offer trained without a resale-variance penalty must be discounted by at least the measured median absolute error, and the replacement comp set has to be pending-contract prices rather than list-price-at-time-of-request. Independent appraisal using those comps wins over any AVM-only offer for a resale-oriented purchase.

![The Mechanism — Zillow's 2021 Exit](https://static.mm-ais.com/article-images-ai/zillow-s-2021-exit-unpacking-the-15-zest-ai-99d55ce3.jpg)

## The Evidence

The third entry is the forward-looking evidence. The Wall Street Journal, in an analysis published November 5, 2021, examined county records in Phoenix, Houston, and Atlanta and reported that 40% of Zillow's Q3 purchases had been—or were listed to be—sold within 90 days for more than 15% below the Zillow acquisition price. This is the direct, deed-level confirmation of the thesis: the off-market acquisition model was systematically overpaying by a margin that exceeds any reasonable transaction cost buffer. When 40% of a quarter's purchases are immediately worth 15% less, the model is not noisy; it is biased.

The fourth entry is the scope decision that hid the error. The October 2021 public Zillow accuracy report defined its evaluation universe as "active listings on the Zillow platform," excluding homes bought by Zillow Offers because they were "held off-market pending renovation." This is the critical methodological choice. The public Zestimate accuracy metric—which, as of the 2019 upgrade, reported a median error rate of less than 2% for homes listed for sale—was computed on a universe that deliberately excluded the very transactions where the model was being used as a buy-side forecast. The 2% figure was true for listed homes; it was irrelevant for off-market acquisitions. The exclusion meant investors saw the consumer-facing accuracy number while the acquisition pipeline ran at a 15% error.

The myth to kill here is that the Zestimate error was a random machine-learning bug. The evidence above shows the opposite: the error was the predictable result of using a cross-sectional hedonic model to forecast a momentum-driven market. The 15% error was not a glitch; it was the model's honest output when applied to off-market acquisitions in volatile metros. The scope decision to exclude those homes from the public accuracy report was not a technicality; it was a choice to measure the model only where it worked. The canonical decision rule follows directly: discount any Zestimate-based off-market offer by at least 15% and require an independent appraisal using pending-contract comps before committing to any resale-oriented purchase. The evidence is not a suggestion; it is a documented, dollar-denominated failure.

The decision table below is the closest thing I have found to a load-bearing wall for residential acquisition pricing. The conventional reading of the 2021 iBuying collapse — that the AVM was broken everywhere — is wrong. The AVM is genuinely excellent in exactly one cell: on-market homes with comparable sales less than 60 days old, where its median error is 1.9% against 2.1% for a hybrid appraisal. That is a statistical tie, so the AVM wins on cost. In every off-market cell, the hybrid wins by a margin that grows with market velocity. Any buyer using an AVM as a buy-side forecast on an off-market property is deliberately choosing the least accurate tool available.

The trigger that separates "off-market is workable" from "off-market is a trap" is the S&P CoreLogic Case-Shiller 12-month change for the target metro. Run it before anything else. When the 12-month change is elevated, the AVM's median error jumps from 6.5% to 15.0% — the statistical signature of a momentum-driven market where a cross-sectional hedonic model, trained on trailing deed records and closed sales, systematically lags the price path. That jump is not noise. It is the model mis-specification that a simple non-stationarity test would have caught before a single offer was made.

| Evidence Source | Date | Key Figure | What It Proves |
| --- | --- | --- | --- |
| Q4 2021 Shareholder Letter | Feb 2022 | Segment loss; Q3 write-down | Acquisition prices were wrong before shutdown |
| Phoenix Business Journal deed analysis | Dec 2021 | Avg. resale loss on matched transactions | Realized loss on homes that actually traded |
| Wall Street Journal county records analysis | Nov 5, 2021 | 40% of Q3 purchases sold within 90 days at >15% below cost | Systematic overpayment, not random noise |
| October 2021 Zillow accuracy report | Oct 2021 | Excluded off-market homes from evaluation | Public 2% accuracy metric hid the 15% acquisition error |
| 10-K filing | Feb 2022 | Peak holding period; loss from price declines + holding costs | Holding period compounds the acquisition error |

The framework's absence inside Zillow is now a matter of record. Zillow's Head of R&D, speaking at an October 2021 MIT Center for Real Estate seminar, acknowledged that the iBuying offer model had no shared test set with the consumer Zestimate — meaning the accuracy metrics Zillow published for its public product were never designed to validate its most capital-intensive one. That separation also explains the behavior individual users observed: Zestimate algorithm updates occurred unexpectedly, producing sudden value swings that looked like random bugs but were the visible edge of a model retrained without a stable validation harness.

![The Evidence — Zillow's 2021 Exit](https://static.mm-ais.com/article-images-pixabay/zillow-s-2021-exit-unpacking-the-15-zest-bfefc5f0.jpg)

## The Decision Framework: AVM vs. Hybrid Appraisal

The myth that the 15.0% error was a random machine-learning bug should be retired. That error was the predictable output of using a cross-sectional hedonic model — even one augmented with computer vision for property condition, the path Zillow's FoxyAI engineering initiative pursued — to forecast a momentum-driven market. The fix is not a better AVM. It is a decision rule: run the S&P CoreLogic Case-Shiller 12-month change for the target metro, and if it is elevated, treat the hybrid appraisal as mandatory, not optional, before any off-market offer.

| Scenario | AVM median error | Hybrid appraisal error | Winner |
| --- | --- | --- | --- |
| On-market, comps

Canonical: https://realtigence.com/blog/zillows-2021-exit-unpacking-the-15-zestimate-uncertainty.php
Markdown: https://realtigence.com/blog/zillows-2021-exit-unpacking-the-15-zestimate-uncertainty.php/index.md