Modern ecommerce platforms — Shopify, Walmart, WooCommerce, BigCommerce, Magento and the long tail of marketplaces — all sit behind anti-bot edges that filter datacenter IPs in milliseconds. Tier-1 residential IPs sit beneath that filter because they read as ordinary shoppers from ordinary ISPs. Unlimited bandwidth keeps catalogue-scale sweeps affordable when per-GB metering would dominate the bill.
Five years ago a Scrapy job from a cloud VM worked against most ecommerce sites. In 2026 it does not — the anti-bot edge ate the cloud VM. Here is the matrix of what changed.
Shopify's edge layer challenges datacenter IPs on product, collection and checkout endpoints. Residential rotation drops every request onto a fresh consumer IP that the edge has no reason to challenge.
Walmart filters datacenter ASNs at the edge and serves a JavaScript challenge to suspicious sessions. US-ASN residential routing passes consistently; mobile passes even more reliably for product-page scraping.
Most WooCommerce stores sit behind Cloudflare's bot-management layer. Residential IPs from real consumer ISPs do not trip the same heuristics datacenter ASNs do.
Shopify regional storefronts, Walmart Mexico, Walmart Canada and ecommerce marketplaces in Europe and Asia all serve geo-conditioned content. Country and city targeting reproduce regional inventory and pricing accurately.
A nightly 500k-product sweep with full HTML payloads runs into the terabytes per month. Unlimited residential bandwidth on Budget Unlimited caps the bill regardless of how many products you track.
Category pagination, infinite scroll and faceted search break if the IP rotates mid-cursor. Sticky sessions up to 60 hours hold one IP for the full category crawl.
Default for spot-accurate, geo-aware ecommerce data. Tier-1 ISP residential with city, state and ASN US targeting. The only plan that reliably reproduces metro-specific pricing, stock and regional storefronts.
Default for catalogue-scale nightly sweeps. Unlimited bandwidth, port-based pricing, 70+ countries with selectable per-port routing. Caps the bill when nightly sweeps cross terabyte volume.
Higher-throughput unlimited pool for ecommerce data platforms that need 1 Gbps+ aggregate across a multi-tenant catalogue.
Use when a specific store shows extra friction or when mobile-only product views differ from desktop. Real 4G/5G carrier IPs across 113 countries.
# Python — Scrapy spider against a Shopify catalogue via Platinum import scrapy class ProductSpider(scrapy.Spider): name = "products" custom_settings = { "DOWNLOADER_MIDDLEWARES": { "scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware": 543, }, } def start_requests(self): proxy = "http://YOUR_USER-cc-us-state-tx-session-cat_001:[email protected]:20228" urls = ["https://store.example.com/collections/all"] for u in urls: yield scrapy.Request(u, meta={"proxy": proxy}, callback=self.parse) def parse(self, response): for product in response.css("a.product-link::attr(href)").getall(): yield response.follow(product, callback=self.parse_product)
Shopify Bot Protection, Walmart's Akamai layer, Cloudflare in front of WooCommerce — all filter datacenter IPs and serve a degraded page or a challenge. Real residential IPs from a 90M+ rotating pool sit beneath that threshold because each IP looks like a normal shopper.
Yes. Product pages, collections, JSON product endpoints and checkout-flow scraping all work. For high-anti-bot stores, sticky residential or mobile sessions are recommended.
Yes. Walmart's anti-bot layer filters datacenter ASNs but Tier-1 US residential on Platinum (with optional city/ASN targeting) passes consistently. Target and Best Buy behave similarly. Mobile is an alternative when extra friction appears.
Per-GB Platinum at $3/GB is most cost-effective until a few hundred GBs/month. Above that, Budget Unlimited from $51.99/mo wins. Many catalogue teams run both — Platinum for spot captures and Budget for nightly bulk sweeps.
Yes. Standard HTTP/HTTPS/SOCKS5 with user:pass auth. Works with Scrapy, Playwright, Puppeteer, Crawlee, Apify actors, Firecrawl, requests and httpx. SOCKS5 is recommended for browser automation needing DNS-through-proxy.
Yes. Country targeting on every plan across 150+ countries. Platinum adds state and city-level US targeting for cases where pricing or stock differs by metro.
ZIP-level Buy Box and ASIN tracking.
Google search data for category and keyword discovery.
Limited-edition drops and Shopify-hosted releases.
$5 trial credit. Pay-as-you-go after. Volume pricing on Platinum down to $1.70/GB at 1,000 GB. Budget Unlimited from $51.99/mo flat.