Error reference HTTP status

Error 520: Cloudflare's catch-all, decoded

Error 520 is Cloudflare's catch-all: its edge connected to the origin server, sent the request, and got back something it couldn't interpret — an empty reply, a reset connection, or malformed headers. It's mostly an origin problem, but scrapers can trigger it themselves with bloated cookies and oversized headers. Here's how to tell which case you're in and fix it.

Try clean residential routes All error codes

520 Web Server Returned an Unknown Error

What error 520 means

HTTP 520 isn't in any RFC — it's a Cloudflare-specific status. When a site sits behind Cloudflare, the edge proxies your request to the site's real origin server. A 520 means that hand-off went wrong in an unclassifiable way: the origin returned an empty response, reset or closed the connection mid-exchange, replied with malformed or illegal headers, or sent nothing before disconnecting. Cloudflare files everything it can't categorize under 520, which is why it's labeled an unknown error.

The usual culprits live on the origin: a crashing application process, a web server that resets connections under load, restrictive origin-side security software dropping requests it dislikes, or keepalive and timeout mismatches between Cloudflare and the origin.

One cause sits squarely on the client side, and it matters for scrapers: oversized request headers. Cloudflare documents header payloads over roughly 32 KB — almost always runaway cookies accumulated over a long session — as a direct 520 trigger. If your crawler reuses one cookie jar for hours, rule this out first.

520 Web Server Returned an Unknown Error

How to fix error 520

First separate 'the origin is misbehaving for everyone' from 'my requests are triggering it.' These checks are ordered to make that split fast.

Retry after a short delay. 520s are frequently intermittent — an origin process restarting or briefly resetting connections — and clear within seconds to minutes.
Check the site from another network or device. If everyone gets a 520, the origin is down or misconfigured and only the site's operator can fix it; schedule retries and move on.
Clear your cookie jar and retry. Header payloads over roughly 32 KB — nearly always accumulated cookies — are a documented 520 trigger, and long-running scrape sessions hit this silently.
Audit your headers for anything malformed: duplicate Host headers, illegal characters, hand-rolled raw requests missing required fields. Origins can reset on requests that violate expectations.
Reduce concurrency. Origins that reset connections under load produce 520s in bursts; halving parallel connections often stops them immediately.
Compare a plain request from your machine with your scraper's exact request — same URL, headers, and cookies. If only the scraper's version gets a 520, diff the two until you find the trigger.
If it's your own site: check origin error logs at the failure timestamps, confirm the application process isn't crash-looping, and verify origin security tooling isn't dropping Cloudflare's forwarded requests.
If the 520 follows one exit IP, rotate to a fresh IP and a clean session — origin-side security sometimes resets connections from addresses it has scored badly.

520 Web Server Returned an Unknown Error

Where proxies fit into a 520

A 520 is generated between Cloudflare and the origin — a leg of the journey your proxy never touches — so a proxy can't directly cause one. What your proxy and session setup do control is the request state and reputation that origin-side security evaluates. When the origin's own firewall or application resets connections for traffic it distrusts, Cloudflare reports the reset as a 520 — and it happens selectively, to you, while browsers sail through.

That's why 520s on scraping infrastructure so often trace back to session hygiene: one eternal cookie jar shared across thousands of requests, headers accumulated through redirects, or an exit IP that origin security has already flagged. Fresh sessions, bounded cookies, and reputable residential exits remove those triggers.

Reset state per session, deliberately

ProxyOmega sticky sessions via -session-<id> pin an exit IP for up to 24 hours; pair each session ID with its own cookie jar and discard both together. Rotating the session ID gives you a genuinely clean slate — new IP, new cookies — which clears the accumulated-state class of 520s.

Use exits that origin security trusts

Origin-side tooling that resets suspect connections keys heavily on IP reputation. Residential exits from Budget Unlimited's 1.5M+ pool score as real consumer connections; Platinum's Tier-1 ISP-quality addresses and Mobile's real 4G/5G carrier IPs are the tiers to reach for on targets that reset anything datacenter-shaped.

Back off in bursts, not per-request

520s from an overloaded or flapping origin arrive in clusters. Treat a cluster as one event: pause the whole job for a minute, then resume through fresh rotation, rather than retrying each failed URL in place and prolonging the storm.

520 Web Server Returned an Unknown Error

A retry pattern with fresh sessions

This Python snippet retries 520s with a brand-new sticky session per attempt — fresh exit IP, fresh cookie jar — and backs off exponentially, which addresses both the intermittent-origin and accumulated-state causes at once.

import time
import uuid
import requests

PASSWORD = "password"  # your dashboard API key
HEADERS = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"}

def fetch(url, max_attempts=4):
    for attempt in range(max_attempts):
        sid = uuid.uuid4().hex[:8]
        user = f"username-session-{sid}"
        proxy = f"http://{user}:{PASSWORD}@residential.proxyomega.com:10000"
        with requests.Session() as s:   # fresh cookie jar every attempt
            s.proxies = {"http": proxy, "https": proxy}
            s.headers.update(HEADERS)
            resp = s.get(url, timeout=30)
        if resp.status_code != 520:
            return resp
        time.sleep(2 ** attempt)        # 1s, 2s, 4s, 8s
    raise RuntimeError(f"520 persisted for {url}")

print(fetch("https://example.com/").status_code)

FAQ

Frequently asked questions

Is error 520 my fault or the website's?

Usually the website's: its origin server returned something Cloudflare couldn't parse, or reset the connection. But two client-side triggers are worth ruling out first — request headers over roughly 32 KB (almost always accumulated cookies) and malformed hand-built requests. If other networks load the site fine while your scraper gets 520s, start with your own request state.

Can a proxy cause error 520?

Not directly — the failure happens between Cloudflare and the origin, a hop your proxy never touches. Indirectly, yes: if origin-side security resets connections from exit IPs it distrusts, Cloudflare surfaces that reset as a 520. Cleaner sessions and residential or mobile exit IPs remove that trigger; the proxy protocol itself is irrelevant.

What's the difference between error 520 and 521?

521 is specific: Cloudflare tried to open a TCP connection to the origin and was refused — the server is down or blocking. 520 is the catch-all for everything after a successful connection: empty replies, resets mid-request, malformed headers. With 521 nothing got through; with 520 something got through and made no sense.

How long do 520 errors usually last?

Intermittent 520s from a flapping or overloaded origin typically clear in seconds to a few minutes — retry with backoff and they resolve themselves. Persistent 520s on every request mean an origin misconfiguration or crash that only the site operator can repair; poll occasionally at low frequency rather than burning requests against it.

Clean sessions, on demand. Start routing today.

Spin up a fresh residential IP and session for every retry — ProxyOmega sticky sessions hold for up to 24 hours when you need them to.

Get started