How to scrape Pinterest at scale in 2026 with proxies that work

Pinterest is one of the more annoying public sites to collect data from. The pages are heavy, the feed loads through infinite scroll, and the same IP will start getting empty responses or login walls long before it gets a clean block. I have run Pinterest collection jobs from Singapore for trend research and image-metadata projects, and most of the failures I see come from proxy choice and pacing, not from the parsing code.

This tutorial is for operators who already know basic Python and want a pipeline that collects public pin and board metadata (titles, descriptions, image URLs, outbound links, save counts where visible) without falling over at a few thousand requests. It is not for logged-in scraping, account automation, or anything behind a private board. I only cover public pages.

By the end you will have a working Playwright and requests setup that routes through rotating proxies, a queue with retries, and a rough idea of what breaks as you go from a hundred pages to a million. This is not legal advice. Read Pinterest’s terms of service and decide for yourself whether your use case is acceptable.

What you need

python 3.11 or newer, with playwright, httpx and selectolax installed
a proxy source: residential or mobile rotating proxies. datacenter IPs get flagged fast on Pinterest, in my tests within a few dozen requests
a budget. residential bandwidth typically runs a few dollars per GB depending on vendor and volume, and Pinterest pages are image-heavy, so block images in the browser or you will burn bandwidth for nothing. check your vendor’s current price list before you commit
a place to store results: SQLite is fine up to a few hundred thousand rows, Postgres after that
a small queue: a plain table of URLs with a status column is enough to start
2 to 4 hours for the first working version

If you are choosing between proxy types, my comparison is in residential vs mobile proxies for scraping. Short version: residential is the default for Pinterest, mobile is worth paying for only when residential pools keep getting soft-blocked.

First, check whether you need to scrape at all. Pinterest has an official API v5 for pins, boards and your own account data. If your project only needs data from accounts you control, use the API and skip the rest of this article.

Step by step

1. Set up the project and check robots.txt

Create a virtual environment and install the tools.

python -m venv .venv
.venv\Scripts\activate
pip install playwright httpx selectolax
playwright install chromium

Then read the target’s crawl rules. Pinterest publishes them at pinterest.com/robots.txt. The robots exclusion protocol is documented in RFC 9309. It is a convention, not a law, but I treat disallowed paths as off limits and stick to public pin and board pages.

Expected output: Chromium downloads once, and you have a note of which paths you will not touch.

If it breaks: on Windows, if playwright install fails behind a corporate proxy, set HTTPS_PROXY in the shell first and retry.

2. Test your proxy before you scrape anything

Never debug a scraper and a proxy at the same time. Confirm the exit IP first.

import httpx

PROXY = "http://USER:[email protected]:8000"

with httpx.Client(proxy=PROXY, timeout=20) as c:
    r = c.get("https://api.ipify.org?format=json")
    print(r.json())

Expected output: a JSON object with an IP that is not yours. Run it five times. With a rotating gateway you should see different IPs, with a sticky session you should see the same one.

If it breaks: a 407 means bad credentials. A timeout usually means the gateway port or protocol is wrong (http vs socks5). Check your vendor’s dashboard.

3. Fetch one public pin with a plain HTTP request

Before reaching for a browser, see how far plain requests go. Pinterest pages often embed JSON in the initial HTML, and that is far cheaper than rendering.

import httpx
from selectolax.parser import HTMLParser

HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                  "(KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36",
    "Accept-Language": "en-US,en;q=0.9",
}

def fetch(url):
    with httpx.Client(proxy=PROXY, headers=HEADERS, timeout=25,
                      follow_redirects=True) as c:
        return c.get(url)

r = fetch("https://www.pinterest.com/pin/EXAMPLE_PIN_ID/")
tree = HTMLParser(r.text)
print(r.status_code, tree.css_first("title").text())

Expected output: status 200 and a title that matches the pin. Look for <script id="__PWS_INITIAL_PROPS__"> or similar embedded JSON blocks in the HTML. The exact script ids change over time, so inspect the page source rather than trusting me.

If it breaks: a 200 with a login prompt or a near-empty body means the IP or fingerprint is flagged. Rotate the IP and retry. If it still fails on clean residential IPs, move to step 4.

4. Switch to Playwright for pages that need rendering

Board pages and search results load pins as you scroll. For those you need a browser. Route it through the proxy and block images and media to save bandwidth.

from playwright.sync_api import sync_playwright

def scrape_board(url, scrolls=6):
    with sync_playwright() as p:
        browser = p.chromium.launch(proxy={
            "server": "http://gate.example-proxy.com:8000",
            "username": "USER",
            "password": "PASS",
        })
        ctx = browser.new_context(locale="en-US",
                                  viewport={"width": 1366, "height": 900})
        page = ctx.new_page()
        page.route("**/*", lambda r: r.abort()
                   if r.request.resource_type in ("image", "media", "font")
                   else r.continue_())
        page.goto(url, wait_until="domcontentloaded")
        for _ in range(scrolls):
            page.mouse.wheel(0, 2500)
            page.wait_for_timeout(1200)
        html = page.content()
        browser.close()
        return html

The proxy options are described in the Playwright docs under network and proxies.

Expected output: an HTML string containing dozens of pin links after six scrolls. Blocking images typically cuts bandwidth per page by a large margin, and you do not need the pixels to get the image URLs.

If it breaks: if the page stays empty, try wait_until="networkidle" once to see if content is just slow. If you get a challenge page, you are on a bad IP. Do not add more scrolls, change the exit.

5. Parse pins into structured rows

Pull the pin links from the rendered board, then store the fields you need. Keep parsing separate from fetching so you can re-parse saved HTML without paying for proxy traffic again.

import re, sqlite3

def pin_ids(html):
    return sorted(set(re.findall(r'/pin/(\d+)/', html)))

db = sqlite3.connect("pins.db")
db.execute("""create table if not exists pins(
  pin_id text primary key, title text, image text, link text,
  fetched_at text default current_timestamp)""")

Expected output: a list of numeric pin IDs per board, de-duplicated. I store the raw HTML on disk, gzipped, next to the database. It costs almost nothing and has saved me from re-scraping many times when a field changed.

If it breaks: if your regex returns zero IDs on a page that looks fine in a browser, the markup changed. Open the saved HTML and find the new pattern. This will happen a few times a year.

6. Add a queue with retries and backoff

A scraper is only useful at scale if it survives failure. Use a status column (pending, done, failed, retry) and a retry counter.

import random, time

def run(urls, max_retries=3):
    for url in urls:
        for attempt in range(max_retries):
            r = fetch(url)
            if r.status_code == 200 and len(r.text) > 20000:
                save(url, r.text)
                break
            time.sleep((2 ** attempt) + random.uniform(0.5, 2))
        else:
            mark_failed(url)
        time.sleep(random.uniform(1.5, 4))

Expected output: most URLs finish on the first try, a small share on retry, and a few land in failed for manual review. The length check matters, because Pinterest will happily return a 200 with a stub page.

If it breaks: if retries are high across the board, slow down before you buy more proxies. See rotating proxies in Python requests for rotation patterns.

7. Control concurrency per exit IP

Do not hammer one IP. I start at 2 to 3 concurrent workers and raise it slowly while watching the success rate.

python run.py --workers 3 --delay-min 1.5 --delay-max 4

Expected output: steady throughput with a success rate above roughly 90 percent. If it drops below that, back off. More speed at 70 percent success is usually slower in real terms, because failed requests still cost proxy bandwidth.

If it breaks: if one worker fails far more than others, its sticky session got a bad IP. Force a new session id for each worker on every retry.

8. Log everything you will need later

Record the status code, response length, exit IP, elapsed time and proxy session for every request. Once a week, look at success rate by hour and by session type. You will find patterns, such as certain pools degrading at peak hours in their region.

Expected output: a CSV or table you can group by hour. This is boring and it is the thing that separates a scraper that quietly decays from one you can trust.

If it breaks: if logging slows the workers, write logs through a queue to a single writer process.

Common pitfalls

using datacenter proxies because they are cheap. on Pinterest the savings disappear in retries and blocks
loading images in the browser. you pay for bandwidth on every pixel and learn nothing from it
treating a 200 as success. check body length and expected markers, or you will fill your database with stub pages
reusing one browser fingerprint across thousands of sessions. a Chromium build with a stock viewport and identical headers on every request is easy to cluster. if fingerprint consistency becomes your bottleneck, the antidetect browser reviews cover what each tool does and does not hide
scraping logged-in or private content. it is a different risk class, and I do not recommend it

Scaling this

At 10x (a few thousand pages a day), everything above works on one machine. A single residential plan is enough, SQLite is fine, and you can run it from a cron job. Watch your bandwidth bill and success rate weekly.

At 100x (hundreds of thousands of pages), the problems change. Bandwidth becomes the main cost, so prefer the plain HTTP path from step 3 wherever it works and keep Playwright for pages that truly need rendering. Move to Postgres, run workers as separate processes, and keep a per-session success metric so you can drop bad pools. You will also want two proxy vendors, so one outage does not stop the job.

At 1000x (millions of pages), you are running infrastructure, not a script. You need a proper job queue (Redis or a Postgres-backed queue), dedupe across runs, change detection so you do not re-fetch unchanged pins, and alerts when success rate falls. Proxy cost will dominate, and negotiating volume pricing starts to matter. I would also reconsider whether you need every page: sampling and incremental updates are usually cheaper than full recrawls. If your work involves many sessions or identities across platforms, multiaccountops.com covers the operational side of running that kind of setup.

One more point on pacing. Scale is not a reason to ignore the target’s capacity. Keep your request rate modest, identify yourself honestly where you can, and stop if the site tells you to.

Where to go next

Written by Xavier Fok

disclosure: this article may contain affiliate links. if you buy through them we may earn a commission at no extra cost to you. verdicts are independent of payouts. last reviewed by Xavier Fok on 2026-10-16.