How to scrape Lazada at scale in 2026 with proxies that work

Lazada is the biggest marketplace in Southeast Africa’s opposite corner of the map, six country storefronts (Singapore, Malaysia, Thailand, Indonesia, the Philippines, Vietnam) run on one Alibaba-built stack. If you do price monitoring, seller research or catalogue enrichment for SEA, you end up needing its data. The trouble is that a script that works for 50 product pages on your laptop starts hitting slider captchas and empty JSON by page 300.

I run proxy infrastructure out of Singapore, and Lazada is one of the targets I test against most often. This tutorial is for operators who already know basic Python and HTTP and want a setup that keeps working past the first afternoon. I will cover which proxies hold up, how I structure the scraper, and what changes as volume grows.

The outcome: a Playwright-based collector that pulls product listings and detail pages from a chosen Lazada country site, rotates sessions sensibly, backs off when it gets challenged, and writes clean JSON lines you can load into a database. I am not going to promise any business result from the data. That part is on you.

what you need

python 3.11 or newer, with playwright, httpx and selectolax installed
chromium via playwright install chromium
a proxy pool in the same country as the storefront you target. For lazada.sg that means Singapore IPs, for lazada.co.th Thai IPs, and so on
proxy type: I get the best survival rate from mobile (4G/5G) IPs, then residential, and datacenter IPs last. Datacenter ranges get challenged almost immediately on Lazada in my testing
a place to store results: sqlite is fine to start, Postgres once you pass a few million rows
a small VPS or home server to run the job. Any 2 vCPU, 4 GB box handles 3 to 5 concurrent browser contexts
budget: proxies are the main cost. Check your vendor’s current price list, because mobile and residential bandwidth pricing moves a lot. Plan for the proxy bill to be larger than the server bill by a wide margin
time to read the target’s rules. Lazada publishes a robots.txt per storefront, and it tells you which paths they ask crawlers to stay out of

If you are still deciding between proxy types, I compared them for marketplace work in residential vs mobile proxies for ecommerce scraping.

A note on legality: this is not legal advice. Scraping public product data is a grey area that depends on your country, the site’s terms and what you do with the data. If you collect anything that identifies a person (seller names, reviewer profiles), look at Singapore’s PDPC guidance or your local equivalent first.

step by step

1. pick the storefront and match the proxy country

Decide which country site you are scraping and buy proxies that exit in that country. Lazada serves different prices, currency, language and stock per storefront, and it redirects or challenges traffic that looks foreign.

Test your proxy before anything else:

curl -x http://USER:[email protected]:8000 https://ipinfo.io/json

Expected output: JSON with "country": "SG" (or your target country) and an ISP name that looks like a carrier or ISP, not a hosting company.

If it breaks: if the country is wrong, your vendor’s geo-targeting parameter is not being applied. Check their docs for the username suffix or port that selects the country. If the org shows a hosting provider, you bought datacenter IPs and should expect early blocks.

2. load one search page in a real browser

Plain HTTP requests to Lazada’s HTML often return a shell page plus a challenge. Start with Playwright, which accepts a proxy directly (see the Playwright proxy docs).

import asyncio, json
from playwright.async_api import async_playwright

PROXY = {"server": "http://proxy.example.com:8000",
         "username": "USER", "password": "PASS"}

async def main():
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=True, proxy=PROXY)
        ctx = await browser.new_context(
            locale="en-SG", timezone_id="Asia/Singapore",
            viewport={"width": 1366, "height": 800})
        page = await ctx.new_page()
        await page.goto("https://www.lazada.sg/catalog/?q=usb%20c%20cable",
                        wait_until="domcontentloaded")
        await page.wait_for_timeout(3000)
        print(await page.title())
        await browser.close()

asyncio.run(main())

Expected output: a page title containing your search term.

If it breaks: if the title is something like a verification or “punish” page, the IP is flagged. Swap the IP and retry before touching any code. If it works on a mobile IP and fails on your datacenter one, you have your answer about proxy type.

3. find where the product data lives

Lazada’s search pages embed the listing data as JSON that the front end hydrates, and the page also calls internal JSON endpoints as you scroll. Open DevTools, go to the network tab, filter on XHR/fetch, and look for responses carrying the item list (names, prices, ratings, item ids). Rather than parse rendered HTML, I let Playwright capture those responses:

items = []

async def on_response(resp):
    if "catalog" in resp.url and resp.request.resource_type in ("xhr", "fetch"):
        try:
            data = await resp.json()
        except Exception:
            return
        for it in data.get("mods", {}).get("listItems", []):
            items.append({"id": it.get("itemId"), "name": it.get("name"),
                          "price": it.get("price"), "url": it.get("itemUrl")})

page.on("response", on_response)

The exact keys change from time to time, so treat the above as a pattern and confirm against what you see in your own network tab.

Expected output: a list of 40 or so dicts per search page.

If it breaks: if items stays empty, print every XHR URL once and find the new endpoint. Lazada changes field names without notice. Keep your parsing in one function so a fix is one edit.

4. add session rotation, one IP per browser context

Do not rotate the IP on every request. A real shopper keeps one IP for a session, and Lazada ties its challenge cookies to that IP. I use one browser context per IP and retire it after a fixed number of pages.

MAX_PAGES_PER_SESSION = 25

async def run_session(browser_factory, urls):
    ctx = await browser_factory()   # new context, new proxy session
    for n, url in enumerate(urls):
        if n >= MAX_PAGES_PER_SESSION:
            break
        await fetch_page(ctx, url)
    await ctx.close()

With mobile proxies you can trigger a fresh IP by hitting the rotation URL your provider gives you (on ProxySmart-based setups this is the reset link per port). Wait for the new IP to come up before starting the next session.

Expected output: each session completes up to 25 pages, then the next one starts from a different IP.

If it breaks: if sessions die at page 3 or 4, lower your speed (step 5) before you lower the session length.

5. throttle like a person

Randomise delays and keep concurrency low per IP.

import random
await asyncio.sleep(random.uniform(2.5, 7.0))

I run one page at a time per IP, with several IPs in parallel. Fast bursts on a single IP are the most reliable way to get a captcha.

Expected output: a steady rate that does not trigger challenges. Mine ends up around 8 to 15 pages a minute per IP.

If it breaks: if you see challenges only after long runs, add a longer pause (30 to 90 seconds) every 10 pages.

6. detect challenges and back off

Never let your parser treat a captcha page as an empty product. Check for it explicitly and handle it.

def is_challenged(title: str, html: str) -> bool:
    markers = ("punish", "captcha", "slide to verify", "unusual traffic")
    t = (title + html[:4000]).lower()
    return any(m in t for m in markers)

On a hit: mark the IP as burnt, close the context, rotate, and requeue the URL with a retry counter. HTTP 429 is the standard “too many requests” signal (RFC 6585), so if you see it, honour any Retry-After header.

Expected output: challenged URLs reappear in the queue and succeed on a fresh IP.

If it breaks: if the same URL fails three times on three IPs, park it in a dead-letter file and move on. Some pages are just broken or removed.

7. fetch detail pages

For each item id from search, load the product page in the same session and capture price, stock, seller, rating count and shipping origin. Detail pages are heavier and more heavily guarded than search, so budget more time per page (I add roughly double the delay).

Expected output: one JSON object per item with at least id, title, price, currency, seller and URL.

If it breaks: if prices come back null, you probably read the page before the price block rendered. Wait for a selector rather than a fixed timeout.

8. store results and make the job resumable

Write each result to sqlite as you go, keyed on item id plus scrape date.

import sqlite3
db = sqlite3.connect("lazada.db")
db.execute("""create table if not exists items(
  item_id text, day text, name text, price text, seller text,
  primary key(item_id, day))""")
db.execute("insert or replace into items values(?,?,?,?,?)",
           (iid, today, name, price, seller))
db.commit()

Expected output: rerunning the job the same day does not create duplicates.

If it breaks: if the DB locks under concurrency, funnel all writes through a single writer task or move to Postgres.

9. schedule it and watch the success rate

Run it from cron or a systemd timer, and log three numbers per run: pages attempted, pages succeeded, IPs burnt. The ratio of burnt IPs to successful pages is your health metric. When it drifts up, something changed on the site or with your proxy pool.

Expected output: a one-line summary per run you can graph.

If it breaks: if success drops suddenly across all IPs, the site changed. If it drops on one provider only, the provider’s pool got flagged.

common pitfalls

using datacenter proxies and blaming the code. Switch the proxy type first, because it is the cheapest experiment
mismatched geography. A Malaysian IP on lazada.sg, or a browser locale that disagrees with the IP, gets you redirects and odd prices
rotating the IP every request. It looks less human, not more, and it burns through your pool and bandwidth
parsing challenge pages as data. You end up with a database full of nulls that looks like “no products found”
ignoring bandwidth costs. A full browser pulls images, fonts and trackers. Block images and media in Playwright with request routing and your per-page transfer drops a lot, which matters when you pay per GB

scaling this

At 10x (a few thousand pages a day): the script above is enough. One small server, a handful of IPs, sqlite. Your main job is tuning delays and watching the burnt-IP ratio.

At 100x (hundreds of thousands of pages a day): move to a queue (Redis or Postgres with skip locked), run workers on several machines, and separate the proxy pool management from the scraper so workers request an IP lease instead of owning one. Cache aggressively: do not rescrape detail pages whose price rarely moves. Switch to Postgres, and track per-IP stats so you can retire poor performers. Bandwidth becomes the budget line to manage, so strip images and test whether you can get the same data from the JSON endpoints alone.

At 1000x (millions of pages a day): the problem becomes mostly one of proxy supply and operations. You need several providers so that one flagged pool does not stop you, your own mobile modems if volume justifies it, and monitoring that pages you when success rate drops. At this size it is worth asking whether you need all the pages. Sampling categories and tracking only items that moved is usually cheaper than a full crawl. Also expect to spend ongoing time on maintenance, because every few weeks something in the page structure or challenge logic shifts.

If you run several accounts or browser profiles alongside scraping, the fingerprinting side matters too. I cover tools for that over at antidetectreview.org.

where to go next

how to scrape Shopee with mobile proxies: the other big SEA marketplace, with a different anti-bot setup
residential vs mobile proxies for ecommerce scraping: the cost and survival trade-offs in more detail
the full article index: everything else on the site, including proxy reviews and other target walkthroughs

Written by Xavier Fok

disclosure: this article may contain affiliate links. if you buy through them we may earn a commission at no extra cost to you. verdicts are independent of payouts. last reviewed by Xavier Fok on 2026-10-15.