Headless browsers and proxies: a setup that holds up

Wiring a proxy into a headless browser is deceptively easy. You pass one flag, the traffic routes, and the first request goes out through a new address. The demo works. Then you scale it up, and the setup starts leaking small tells that a person clicking around would never produce, and the run degrades in ways that are hard to trace back to the proxy layer.

This post is about the difference between a browser that merely routes through a proxy and one that presents as a coherent client. The gap between those two is where most headless setups quietly fail.

Put the proxy on the context, not the process

The first decision is where the proxy lives. Both Playwright and Puppeteer let you set a proxy at launch, which applies it to the whole browser process. That is fine for a single-job script. It stops being fine the moment you want two jobs, from two addresses, running side by side.

Prefer per-context proxying. In Playwright, a browser context is the natural unit of isolation: it has its own cookies, its own storage, its own session. Give each context its own proxy and you get clean separation without spinning up a browser per job.

const context = await browser.newContext({
  proxy: {
    server: "http://proxy.example:PORT",
    username: "user",
    password: "pass",
  },
});

Now the address, the cookie jar, and the fingerprint travel together. When you tear the context down, you tear all three down at once. That coherence is the whole game.

Rotating versus persistent contexts

There are two shapes of work, and they want opposite things from the address.

Some jobs are stateful. You log in, you hold a session, you move through pages that expect the same visitor across many requests. Here you want a persistent context on a stable address — the same identity for the life of the task. A dedicated address is the cleanest fit, because nothing else shares it and it does not change under you mid-session. This is what Static ISP is for: a single-tenant address that stays put, so a long-lived login does not suddenly appear to teleport.

Other jobs are stateless and broad. You are fetching many independent pages where no single request depends on the last, and looking like one visitor doing thousands of things is the tell you want to avoid. Here you want fresh contexts on a rotating pool — a new context, a new address, no shared history. Premium Unlimited fits this shape: a wide residential pool the run can draw from continuously.

The question that decides your architecture: does the target expect to see the same visitor across these requests, or would that consistency itself look wrong?

Answer that first. Get it wrong in either direction and you either burn a stable identity on work that should have been anonymous, or you scatter a session that needed to stay whole.

Close the leaks

A proxy routes your HTTP traffic. It does not, by itself, guarantee that every signal the browser emits agrees with the address it is routing through. Two leaks matter most.

WebRTC. A browser can reveal a local or true public address through WebRTC’s connection candidates, entirely outside the proxied HTTP path. If you do not need media features in an automation run, the safe move is to disable WebRTC in the browser launch configuration, or block the relevant APIs, so there is no second channel to leak through.

DNS. If name resolution happens on the machine running the browser rather than through the proxy, your resolver location can disagree with your apparent one. Route DNS through the same path as the rest of the traffic. With a SOCKS proxy, prefer the mode that resolves names remotely rather than locally.

Neither of these is exotic. They are just easy to forget, because the browser keeps working when they leak — it only stops looking consistent.

Make the browser agree with the address

This is the part people skip, and it is the part that separates a setup that holds up from one that draws attention.

An address carries an implied context. It sits in a place, in a timezone, in a language region. If your browser reports a timezone from one part of the world while the address sits in another, you have produced a client that does not exist. Same with locale, and with the Accept-Language header.

So match them. When you build the context, set the timezone and locale to fit the address you are routing through.

const context = await browser.newContext({
  proxy: { server: "http://proxy.example:PORT", username, password },
  timezoneId: "America/New_York",
  locale: "en-US",
});

The principle generalizes past those two fields. The address, the timezone, the language, and the headers should tell one story. You are not trying to trick anyone — you are trying to stop your own tooling from contradicting itself. A real visitor’s browser and network agree with each other by default. An automated one only agrees if you make it.

Let the product carry the shape

Notice how much of the browser-side work follows from one upstream choice: what kind of address you are on.

A rotating residential pool wants fresh contexts and per-request variety, and it wants you to keep the locale sensible for whatever region you land in. A dedicated static address wants one persistent context and a fixed, matching locale that never moves. If you know the shape of the work, the address type is usually obvious, and once the address type is set, most of the context configuration writes itself.

Keep the automation honest, too. Pull public data, respect the terms and robots directives of the sites you touch, and pace yourself so the run behaves like traffic a site can absorb rather than a burst it has to defend against. A setup that holds up technically and a setup that behaves responsibly tend to be the same setup.

The takeaway

A working proxy connection is the start, not the finish. The setups that last are the ones where the whole client agrees with itself: the proxy on the right unit of isolation, the context shape matched to the job, the leaks closed, and the timezone and locale in step with the address.

Get those four right and the proxy stops being a thing you bolted on. It becomes part of a client that simply looks like what it is.