[Bug]: HTTP crawler puts proxy credentials into the URL unencoded #2326

crawl4ai version

0.9.4 (same code on develop at 1f68e5b)

Expected Behavior

AsyncHTTPCrawlerStrategy works through a proxy whose username or password contains #, /, ? or %, as the browser path already does (it passes username and password to Playwright as separate fields).

Current Behavior

_format_proxy_url (async_crawler_strategy.py:2634) builds f"{protocol}://{username}:{password}@{rest}" without percent-encoding. Results with aiohttp 3.14.3 against a local proxy:

The InvalidURL message is the whole URL, and the except aiohttp.ClientError branch re-raises it as HTTPCrawlerError(f"HTTP client error: {str(e)}"), so the password ends up in the error text.

Is this reproducible?

Yes

Inputs Causing the Bug

ProxyConfig(server="http://host:port", username="user", password="p#ss") with AsyncHTTPCrawlerStrategy.

Steps to Reproduce

Pass the output of _format_proxy_url as proxy= to aiohttp.ClientSession.get, which is what the crawler does at async_crawler_strategy.py:2737.

Code snippets

from urllib.parse import quote

return f"{protocol}://{quote(username, safe='')}:{quote(password, safe='')}@{rest}"

(both branches). Happy to send it as a PR to develop with a test.

OS

Windows 10

Python version

3.11