How to scrape Zillow at scale in 2026 with proxies that work
Zillow sits on the largest public-facing dataset of US residential real estate: listing prices, Zestimates, price history, days on market, rental comps, agent contact info. If you’re building a market research tool, a PropTech product, or just running comps for your own investing, that data is genuinely useful and there’s no clean API to pull it from anymore. Zillow retired most of its public API endpoints (GetSearchResults, GetZestimate, GetUpdatedPropertyDetails) back in 2021, so the rendered website is what’s left.
This tutorial is for operators who already know their way around Python and a headless browser and just need the Zillow-specific parts sorted out: what actually gets you blocked, which proxy type is worth paying for, and where the data actually lives in the page. It’s not for anyone trying to resell Zillow’s dataset wholesale or build a Zillow clone. That’s a different, much riskier project.
By the end you’ll have a working pipeline that pulls listing data for a defined metro area or zip code list, survives more than a handful of requests without tripping Zillow’s bot detection, and writes clean structured records instead of a folder of raw HTML you have to re-parse later.
what you need
- Python 3.11+ with
playwrightinstalled (pip install playwright && playwright install chromium), or the Node equivalent if that’s your stack - a residential or mobile proxy plan with rotating IPs. Datacenter IPs get burned on Zillow fast. I use Decodo residential for this kind of job, pay-as-you-go was running roughly $7-8/GB as of mid-2026, check current pricing before committing
- a proxy session manager or at least a rotation script, sticky sessions per worker so you don’t rotate IP mid-scrape and break cookies
parselor plainjsonfor parsing, you’ll mostly be pulling a JSON blob out of a<script>tag rather than parsing HTML with CSS selectors- storage: SQLite is fine under a few hundred thousand rows, Postgres once you’re running this daily
- a copy of Zillow’s robots.txt and its terms of use, read them before you decide how aggressive to run this. Not legal advice, just know what you’re opting into
- budget: figure $50-300/month in proxy spend depending on volume, plus whatever compute you’re already running the script on. A $10/month VPS is enough for a single-metro job
step by step
1. define your scope before writing any code
Decide upfront: which metros or zip codes, for-sale or rental or both, and how often you need refreshes (daily, weekly, one-time pull). Zillow’s search results are paginated and rate-limited per session, so scope drives everything downstream, including how many proxy sessions you actually need.
Expected output: a flat list of zip codes or metro/state combos, saved as a config file, not hardcoded into the scraper.
If it breaks: if you’re not sure how granular to go, start with one metro and 20-30 zip codes. It’s cheap to widen scope later, expensive to have built a scraper around the wrong URL structure.
2. set up a stealth-patched headless browser
Plain Playwright or Selenium gets fingerprinted quickly. Use playwright-stealth or run Playwright with the standard evasions (masking navigator.webdriver, matching a real Chrome user-agent string, setting a plausible viewport and timezone). Here’s the baseline setup:
from playwright.sync_api import sync_playwright
def launch_browser(proxy):
p = sync_playwright().start()
browser = p.chromium.launch(
headless=True,
proxy={
"server": proxy["server"],
"username": proxy["username"],
"password": proxy["password"],
},
)
context = browser.new_context(
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36",
viewport={"width": 1440, "height": 900},
locale="en-US",
)
return browser, context
Expected output: a browser context that loads zillow.com’s homepage and returns a 200 without an immediate CAPTCHA wall.
If it breaks: if you’re getting served a “press and hold” challenge on the very first load, your fingerprint is off before you’ve even hit a proxy issue. Check navigator.webdriver isn’t true in the page context, and confirm your viewport and user-agent version match (an old Chrome UA string with a new-Chrome viewport is a common tell). This is also where a proper antidetect browser profile setup, covered in more depth on antidetectreview.org, pays off if you’re running many concurrent identities instead of one.
3. wire up rotating residential proxies
Zillow tracks request velocity per IP. A single proxy hammering search results pages gets flagged within a few dozen requests. Assign one sticky proxy session per “worker” (a worker = one browser context working through one zip code at a time), and rotate to a new session only between zip codes, not mid-scrape.
proxies = [
{"server": "http://gate.decodo.com:10001", "username": "user-session-1", "password": "pw"},
{"server": "http://gate.decodo.com:10001", "username": "user-session-2", "password": "pw"},
]
Expected output: each worker completes a full zip code’s listing pages on one IP before switching.
If it breaks: if you’re seeing 403s appear mid-session, your session is rotating too aggressively on the proxy provider’s end, most providers let you set a sticky session duration (10-30 minutes is typical). Check that setting first before assuming it’s a fingerprint problem.
4. hit the search results pages and capture the embedded JSON
Zillow’s listing pages are a server-rendered React/Apollo app. Instead of parsing rendered HTML with CSS selectors, which breaks every time Zillow ships a frontend change, pull the JSON payload directly out of the page source. It’s usually sitting inside a <script> tag as a large serialized state object.
import json, re
def extract_json_blob(html):
match = re.search(r'<script id="__NEXT_DATA__"[^>]*>(.*?)</script>', html, re.S)
if not match:
return None
return json.loads(match.group(1))
Expected output: a Python dict with listing price, address, beds/baths, Zestimate, and days-on-market nested somewhere inside it. The exact key path shifts periodically, print the top-level keys and walk down rather than hardcoding a path from a tutorial screenshot.
If it breaks: if the regex returns nothing, view-source the page manually and search for a script tag with a large JSON payload, Zillow occasionally renames the script id. If the page is served with no data at all, you likely got served a bot-check page instead of the real listing, check the HTTP status and page title before assuming your parser is broken.
5. handle pagination and search filters
Zillow’s search results paginate through a combination of URL query parameters and an internal search API called by the frontend. For most jobs, iterating the rendered search pages (searchQueryState in the URL) is more stable than trying to reverse-engineer the internal API, since that API’s shape changes without notice and isn’t documented anywhere Zillow publishes.
Expected output: a full sweep of listings for a zip code across all result pages, typically capped around 500-800 results per search depending on Zillow’s pagination limit.
If it breaks: if you hit a wall of duplicate results on later pages, your search filter state probably isn’t being carried over correctly between page loads. Log the full URL for each page request and diff it against the previous one.
6. rate limit and add retry logic
Set a delay between requests inside a session (2-6 seconds, randomized, not fixed) and cap requests per proxy session before rotating. Wrap every fetch in a retry with exponential backoff for 403s and timeouts, but stop after 3 retries and move on, don’t let one blocked zip code stall the whole run.
import time, random
def polite_delay():
time.sleep(random.uniform(2.5, 6.0))
Expected output: a run that completes with a block rate under roughly 5-10%, retried failures logged separately for a second pass.
If it breaks: if your block rate is climbing above 20-30%, that’s a proxy quality problem, not a code problem. Cheap or shared residential proxies get burned on high-value targets like Zillow faster than dedicated pools.
7. store, dedupe, and version your data
Write to a database keyed on Zillow’s internal listing ID (zpid), not address strings, addresses get normalized inconsistently across runs. Keep a last_seen timestamp so you can track price changes and delistings over time instead of just overwriting rows.
Expected output: a table where re-running the scraper updates existing rows and flags price or status changes, instead of creating duplicates.
If it breaks: if you’re seeing duplicate rows for the same property, check you’re keying on zpid and not on address, unit numbers and abbreviation formatting will silently create dupes.
common pitfalls
- Running every zip code through one proxy session. This is the single fastest way to get an IP range flagged. One sticky session per zip code, rotate between them.
- Assuming the old Zillow API still works. GetSearchResults and the Zestimate API were deprecated in 2021. Tutorials or Stack Overflow answers referencing those endpoints are years stale.
- Parsing rendered HTML instead of the embedded JSON. HTML class names and DOM structure change often. The embedded JSON payload is more stable and gives you cleaner, typed data anyway.
- Ignoring robots.txt and ToS entirely and running at high volume from a single server. Not a legal opinion here, just an operational one: the more aggressively you run this against Zillow’s stated terms, the more resources you should assume they’ll dedicate to blocking you specifically.
- Not budgeting for proxy churn. Residential IPs get flagged over time even with good rotation hygiene. Budget for replacing a portion of your pool monthly, not just the initial signup cost.
scaling this
At 10x (a handful of metros, daily refresh), a single VPS, one residential proxy plan, and the setup above is enough. You can run it on a cron job and babysit it manually.
At 100x (dozens of metros, hourly or near-real-time refresh), you need a proper job queue (Redis + a worker pool, or Celery), a dashboard tracking block rate per proxy pool, and geographically distributed exit IPs that roughly match the metros you’re scraping, a residential IP physically in Texas hitting Texas listings draws less suspicion than one that clearly isn’t.
At 1000x (national coverage, continuous refresh), you’re running distributed workers across multiple machines or a cloud provider, a proxy pool sized in the tens of GB/day, and dedicated monitoring for ban rate as a first-class metric, not an afterthought. At this scale it’s also worth seriously pricing out licensed MLS data through something like Bridge Interactive, Zillow’s own data licensing arm, against the ongoing cost and legal exposure of scraping at that volume. For a lot of use cases the licensed route ends up cheaper once you count proxy spend, engineering time, and risk.
where to go next
If Zillow is one of several real estate sources you need, the Realtor.com scraping tutorial covers the same proxy and anti-bot playbook applied to a site with a different anti-bot setup. If you’re still choosing a proxy provider, the Decodo vs Smartproxy comparison breaks down pricing and residential pool quality between the two I get asked about most. For everything else on scraping and proxy infrastructure, the full article index is the place to browse.
Written by Xavier Fok
disclosure: this article may contain affiliate links. if you buy through them we may earn a commission at no extra cost to you. verdicts are independent of payouts. last reviewed by Xavier Fok on 2026-07-18.