Giving an AI agent a proxy: environment variables, Playwright MCP and browser agents

The three problems agents create#

• Unpredictable targets. The agent decides where to go; you cannot pre-select datacenter for one site and residential for another. A residential exit is the safe default.
• Many hops per task. Search, open, click, read, repeat: a task is a session, and should stay on one exit while it lasts.
• Unbounded spending. An agent in a loop can fetch gigabytes. Give it a ceiling it cannot exceed.

Environment variables for the whole toolchain#

Most Python HTTP stacks (requests, httpx, aiohttp, urllib) and many tools honour HTTPS_PROXY, HTTP_PROXY and NO_PROXY. Setting them in the agent's process proxies every tool that fetches pages, without touching their code.

export HTTPS_PROXY="http://AGENTUSER-cc-us-sid-task-4f2a-ttl-2h:[email protected]:9000"
export HTTP_PROXY="$HTTPS_PROXY"
export NO_PROXY="localhost,127.0.0.1,api.openai.com,api.anthropic.com"
python agent.py

Exclude the model provider's API hosts with NO_PROXY: the LLM calls do not need a residential exit and would only cost traffic.

Node.js agents#

Node's fetch (undici) ignores the environment by default. Install a global dispatcher once at startup and every fetch in the agent, including third-party tools, follows it.

import { EnvHttpProxyAgent, setGlobalDispatcher } from "undici";
setGlobalDispatcher(new EnvHttpProxyAgent()); // reads HTTPS_PROXY / NO_PROXY

Playwright MCP server#

The Playwright MCP server, used by Claude, ChatGPT-style tools and IDE agents to drive a browser, accepts a proxy at launch. Pass the gateway with a sticky username and the bypass list for local hosts.

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": [
        "@playwright/mcp@latest",
        "--proxy-server=http://res.proxshift.com:9000",
        "--proxy-bypass=localhost,127.0.0.1"
      ]
    }
  }
}

Chromium prompts for proxy credentials, which an MCP session cannot answer; whitelist the machine's public IP in the dashboard so no prompt appears, and use a sub-user if you want that machine on its own traffic ceiling. Where the server exposes username and password options, pass a sticky username instead.

Browser agents built on Playwright#

Frameworks such as browser-use expose Playwright's proxy settings (server, username, password) in their browser configuration. Give each agent run a fresh session id so parallel runs do not share an exit, and set the context locale and time zone to the exit's country.

proxy = {
    "server": "http://res.proxshift.com:9000",
    "username": f"AGENTUSER-cc-us-sid-{run_id}-ttl-1h",
    "password": "PASS",
}
# pass `proxy` where the framework accepts Playwright proxy settings

Ceilings: sub-users#

Create a sub-user per agent or per project with a traffic limit in GB. The agent gets its own username and password, its usage is reported separately, and it cannot spend beyond the ceiling whatever loop it falls into.

Behavioural hygiene#

• Cap the agent's requests per domain and per minute; rotation does not excuse hammering.
• Block images, media and fonts in the browser context; agents read text.
• Log the session id with each task so a blocked run can be traced to its exit and replayed on a new one.

Questions people ask#