How to scrape eBay at scale in 2026 with proxies that work

eBay is one of the harder marketplaces to scrape consistently. It runs aggressive bot detection on both its HTML search results and its GraphQL-backed endpoints, it geo-fences pricing and shipping data by country, and it rate-limits by IP fast enough that a single residential connection gets flagged within a few hundred requests. If you’ve tried pulling listing data with a basic requests script and watched your success rate collapse after twenty minutes, this is normal, not a mistake you made.

This tutorial is for people building price trackers, dropshipping research tools, competitor monitoring dashboards, or sourcing feeds who need eBay data on a recurring basis, not a one-off scrape of fifty listings. I’ll walk through the actual infrastructure I use: proxy selection, request shaping, pagination handling, and the failure modes that eat most people’s time. By the end you’ll have a working pipeline that can pull thousands of listings a day without getting your IPs burned in a week.

One thing up front: this is not legal or tax advice. eBay’s User Agreement restricts automated data collection outside of their official API, and the terms are the terms whether or not enforcement is common. Use eBay’s official Browse API where it covers your use case, and treat HTML scraping as a fallback for data the API doesn’t expose, not a default.

what you need

• Proxies: residential or ISP proxies with per-request rotation. Datacenter IPs get flagged on eBay within minutes at any real volume. I use Decodo and SOAX depending on the job, see my Decodo review for pricing and setup notes.
• A scripting environment: Python 3.11+ with requests or httpx, plus lxml or parsel for parsing. Node with Playwright works too if you need JS rendering.
• An eBay developer account (free) if you’re using the official Browse API instead of, or alongside, HTML scraping. Sign up at developer.ebay.com.
• A queue or scheduler: cron is fine at low volume, something like Celery or a simple SQLite-backed job table once you’re running thousands of requests a day.
• Storage: Postgres or even a flat SQLite file for anything under a few million rows.
• Budget: expect $50-150/month in residential proxy bandwidth for a moderate scrape (10-20k listings/day). Datacenter proxies are cheaper but you’ll burn more on retries and CAPTCHA-solving services, which often nets out worse.

step by step

1. decide between the Browse API and HTML scraping

eBay’s Browse API gives you structured JSON for search, item details, and pricing, authenticated with an OAuth token, no proxies required for reasonable volumes. It covers most sourcing and price-tracking use cases. HTML scraping is what you fall back to for data the API restricts, like certain seller-side details or category browse pages the API doesn’t map cleanly.

Expected output: a decision on which path you’re building. Most operators end up running both, API for bulk structured data, scraping for gaps.

If it breaks: if the API rejects your OAuth request, check your token scope, the Browse API needs https://api.ebay.com/oauth/api_scope at minimum, and application tokens expire every 2 hours.

2. set up your proxy pool

If you’re going the scraping route, get a rotating residential proxy plan. Configure it to rotate on every request, not on a timer, sticky sessions increase your chance of hitting eBay’s per-IP request thresholds.

import httpx

proxy_url = "http://user-session-{rand}:[email protected]:10000"

client = httpx.Client(
    proxies=proxy_url.format(rand="rotate1"),
    timeout=15.0,
)

Expected output: successful GET requests to https://www.ebay.com returning 200s with varying exit IPs. Test with curl -x http://user:[email protected]:10000 https://ifconfig.me a few times in a row and confirm the IP changes.

If it breaks: if every request returns the same IP, your proxy plan is on sticky-session mode by default, check the provider dashboard for a rotation setting or session-ID parameter.

3. build realistic request headers

eBay’s bot detection weighs header consistency heavily. A proxy rotating through US residential IPs sending headers that claim Chrome on Windows but missing sec-ch-ua and accept-language gets flagged fast.

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
    "Accept-Language": "en-US,en;q=0.9",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Sec-Ch-Ua": '"Chromium";v="126", "Not.A/Brand";v="24"',
    "Referer": "https://www.ebay.com/",
}

Expected output: HTML responses instead of eBay’s interstitial “verify you’re a human” page.

If it breaks: if you’re seeing that interstitial regularly, your header set is probably static across requests while your IP rotates, which is itself a fingerprint. Vary the User-Agent string across a pool of 15-20 realistic recent Chrome/Firefox versions and pair it consistently with matching Sec-Ch-Ua values.

4. request search results and parse the listing grid

eBay’s search results (https://www.ebay.com/sch/i.html?_nkw=<query>) return server-rendered HTML with listing cards you can parse directly, no JS execution needed for the basic grid.

from parsel import Selector

resp = client.get(
    "https://www.ebay.com/sch/i.html",
    params={"_nkw": "vintage camera", "_pgn": 1},
    headers=headers,
)
sel = Selector(resp.text)

for item in sel.css("li.s-item"):
    title = item.css(".s-item__title::text").get()
    price = item.css(".s-item__price::text").get()
    link = item.css("a.s-item__link::attr(href)").get()
    print(title, price, link)

Expected output: a list of titles, prices, and item URLs printed per page, typically 60 items per search results page.

If it breaks: if s-item selectors return nothing, eBay has likely changed its markup, they do this every few months. Pull a fresh page in a real browser, inspect the DOM, and update selectors. Also check you’re not getting served the mobile layout, which uses different class names, forced by certain User-Agent strings.

5. handle pagination and rate limiting

eBay paginates via _pgn, and search caps out around 10,000 results (240ish pages) regardless of how many the count badge claims. Space requests out, 1-3 second randomized delays per request per proxy thread, and cap concurrency per proxy exit IP rather than globally.

import time, random

for page in range(1, 50):
    resp = client.get(url, params={"_nkw": query, "_pgn": page}, headers=headers)
    time.sleep(random.uniform(1.2, 3.0))

Expected output: steady successful pulls across pages with no sudden run of 403s.

If it breaks: a run of 403s or CAPTCHA pages mid-pagination usually means one proxy IP got flagged. Rotate immediately rather than retrying the same exit node, retrying on a flagged IP just confirms the block to eBay’s system.

6. deal with CAPTCHAs and interstitials

At volume you will hit eBay’s PerimeterX-style challenge pages. Don’t try to solve these programmatically with OCR, it’s a losing arms race and wastes proxy bandwidth. Instead, treat a challenge response as a signal to retire that proxy IP for a cooldown period (I use 30-60 minutes) and route the retry through a fresh one.

Expected output: a low, stable challenge-page rate (under 2-3% of requests) rather than escalating blocks.

If it breaks: if your challenge rate climbs above 10%, you’re probably scraping too fast for your proxy pool size. Either slow down or add more IPs, don’t try to push through with faster retries.

7. store and deduplicate

Listings churn constantly on eBay, sold items disappear, prices change hourly on auctions. Store item ID (extract it from the listing URL) as your primary key and upsert on each pass rather than inserting duplicates.

INSERT INTO listings (item_id, title, price, seen_at)
VALUES (%s, %s, %s, now())
ON CONFLICT (item_id) DO UPDATE
SET price = EXCLUDED.price, seen_at = EXCLUDED.seen_at;

Expected output: a growing table where re-running the scrape updates existing rows instead of duplicating them.

If it breaks: if you’re seeing duplicate rows, check you’re parsing the numeric item ID out of the URL path (/itm/<id>) rather than using the full URL, which can include tracking query params that vary between crawls.

8. add monitoring

Track your success rate, challenge rate, and proxy cost per 1,000 successful requests over time. This is the metric that tells you when to add proxy volume before it becomes a crisis.

Expected output: a simple daily log or dashboard showing these three numbers trending flat or improving.

If it breaks: if cost-per-1,000 is climbing steadily with no volume increase, your proxy pool is likely getting progressively flagged, time to rotate in a fresh IP allocation from your provider.

common pitfalls

• Using one proxy or a small static pool for everything. eBay’s per-IP thresholds are tighter than Amazon’s or Walmart’s. A pool that works fine for other targets can still get burned fast here.
• Ignoring the Browse API entirely. A lot of what people scrape via HTML is available structured and authenticated through the API. Scraping HTML for data the API already gives you is wasted proxy spend.
• Scraping at a constant, mechanical interval. Fixed 2-second delays are themselves a fingerprint. Randomize.
• Not handling currency and region variance. eBay serves different prices and even different listings by detected IP geography, if your proxies are mixed-country, your dataset will have silent inconsistencies unless you pin proxy geo per scrape run.
• Treating a legal gray area as a solved problem. Read eBay’s User Agreement restrictions on automated access, and be aware of the general legal landscape around scraping public data, the EFF has a good summary of the hiQ v. LinkedIn ruling that’s the closest thing to case law precedent here, though it doesn’t make every scraping approach lawful by extension. This isn’t legal advice, talk to a lawyer if you’re building something commercial on top of scraped data.

scaling this

At 10x (say, 1,000 to 10,000 listings/day), a single proxy plan with rotation and the setup above holds fine. One script, one machine, cron-scheduled.

At 100x (100,000+ listings/day), you need a real job queue (Celery, RQ, or similar), proxy pool size scales roughly linearly with request volume, and you’ll want to split scraping across multiple proxy providers so a rate limit or ban on one doesn’t stall the whole pipeline. This is also where using Scrapy’s built-in AutoThrottle and concurrency settings instead of a hand-rolled loop starts paying off, it manages backoff and concurrency per domain more reliably than most custom code.

At 1000x (millions of listings/day, effectively continuous crawling), you’re running distributed workers across multiple machines or a cloud fleet, proxy costs become your dominant line item, and you need dedicated monitoring for ban-rate anomalies per proxy subnet, not just per IP. At this scale it’s also worth pairing scraping infrastructure with proper browser fingerprint management if any part of your pipeline touches headless browser rendering for JS-heavy pages, worth a read through antidetectreview.org/blog/ if you’re getting into that territory.

where to go next

• How to scrape Airbnb at scale in 2026 with proxies that work, a lot of the pagination and proxy-rotation logic here transfers directly.
• Decodo review 2026: honest pros, cons, and pricing for a breakdown of the residential proxy provider I referenced in the setup steps.
• Browse the full tutorial index for more scraping guides across other marketplaces.

Written by Xavier Fok

disclosure: this article may contain affiliate links. if you buy through them we may earn a commission at no extra cost to you. verdicts are independent of payouts. last reviewed by Xavier Fok on 2026-07-14.