eBay Price Monitoring with Mobile Proxies

eBay exposes more data publicly than Amazon. The useful signals for pricing and competitive research:

• →
  Active listings — Buy-It-Now prices, auction asks, shipping.
• →
  Sold-listing history — the LH_Sold=1&LH_Complete=1 filter exposes up to 90 days of sold prices.
• →
  Auction bid history — visible via the bid-history sub-page for each listing.
• →
  Seller feedback — 12-month and lifetime stats, Top Rated Seller status.
• →
  Watchers count — a live signal of demand on high-interest listings.
• →
  Best Offer activity — accepted/declined counts when sellers enable it.

eBay is noticeably less aggressive than Amazon or Walmart. There's no third-party bot-manager fronting the site — detection relies primarily on:

• →
  IP reputation (datacenter ASNs throttled, residential/mobile fine)
• →
  Per-IP rate limits, especially on search result pages
• →
  Interstitial "pardon our interruption" page when patterns look automated
• →
  Session cookies (dp1, s, nonsession) — persist them

A real mobile IP + sane pacing + persistent session is usually enough. You rarely need a headless browser for read-only scraping.

Scrape an eBay search result page for a keyword with a mobile proxy:

import requests
from bs4 import BeautifulSoup
from urllib.parse import quote_plus

headers = {
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15",
    "Accept-Language": "en-US,en;q=0.9",
}
proxies = {
    "http": "http://USER:PASS@hostname:http_port",
    "https": "http://USER:PASS@hostname:http_port",
}

def scrape_ebay_search(keyword, sold=False, pages=1):
    results = []
    base = "https://www.ebay.com/sch/i.html"
    for page in range(1, pages + 1):
        params = (
            f"?_nkw={quote_plus(keyword)}"
            f"&_pgn={page}"
            + ("&LH_Sold=1&LH_Complete=1" if sold else "")
        )
        r = requests.get(base + params, headers=headers, proxies=proxies, timeout=30)
        soup = BeautifulSoup(r.text, "html.parser")
        for item in soup.select(".s-item"):
            title = item.select_one(".s-item__title")
            price = item.select_one(".s-item__price")
            link = item.select_one(".s-item__link")
            if title and price and link:
                results.append({
                    "title": title.get_text(strip=True),
                    "price": price.get_text(strip=True),
                    "url": link.get("href"),
                })
    return results

Toggle sold=Trueto hit the sold-listing archive — this is the most useful data for repricing and market-value research.

Auctions have time-sensitive fields that look different on each visit:

• →
  End time is rendered as a relative countdown ("2h 14m") — parse the absolute datetime attribute on the time element instead.
• →
  Current bid and bid count change between visits; poll on a schedule that fits the auction length (hourly for multi-day, every few minutes in the last hour).
• →
  Bid history lives at /bids/ViewBids?item=<id> and lists anonymized bidders.
• →
  Pagination on search is capped. Narrow filters (price, condition, category) to avoid the 10,000-result cap.

eBay offers a well-documented public API (Browse, Finding, Trading). When should you use it vs scraping?

eBay is forgiving enough that you can get away with longer sticky sessions than on Amazon. A good default:

• →
  Rotate the mobile proxy every 80–120 requests on catalog scans.
• →
  Keep the same IP when paginating a single search result set.
• →
  Use the POST /api/v1/proxies/{slotId}/switch endpoint for rotation, wait ~10s for the modem.
• →
  Persist the requests.Session() so cookies stick across calls.