I'm trying to use playwright in Python to scrape a portal in my company internal network, and I'm getting a useless error message (NS_ERROR_UNKNOWN) when I try to set some preferences in Firefox (according to playwrights documentation)
I'm using Ubuntu 22.04.4 LTS with WSL and Python 3.10.12. The portal only works properly in Firefox, and is only served through HTTPS. The company MITMs all SSL connections (including connections to the internal network), but I have the company root CA installed in WSL.
Here's a sample code:
from playwright.sync_api import sync_playwrightwith sync_playwright() as pwobj: browser = pwobj.firefox.launch( headless = False, firefox_user_prefs = {"security.enterprise_roots.enabled": "true" } ) # page with untrusted certificate for testing purposes page = browser.new_page().goto("https://untrusted-root.badssl.com/")When I run this code, I get the following error message:
Traceback (most recent call last): File "/home/lmnice/sw/projects/mapebot/Rascunhos/playwright_error.py", line 4, in <module> browser = pwobj.firefox.launch( File "/home/lmnice/.venvs/mapebot/lib/python3.10/site-packages/playwright/sync_api/_generated.py", line 13991, in launch self._sync( File "/home/lmnice/.venvs/mapebot/lib/python3.10/site-packages/playwright/_impl/_sync_base.py", line 115, in _sync return task.result() File "/home/lmnice/.venvs/mapebot/lib/python3.10/site-packages/playwright/_impl/_browser_type.py", line 94, in launch Browser, from_channel(await self._channel.send("launch", params)) File "/home/lmnice/.venvs/mapebot/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 59, in send return await self._connection.wrap_api_call( File "/home/lmnice/.venvs/mapebot/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 514, in wrap_api_call raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from Noneplaywright._impl._errors.Error: BrowserType.launch: Protocol error (Browser.enable): Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIPrefBranch.setStringPref]Call log:<launching> /home/lmnice/.cache/ms-playwright/firefox-1449/firefox/firefox -no-remote -wait-for-browser -foreground -profile /tmp/playwright_firefoxdev_profile-XXXXXXFpc6uf -juggler-pipe -silent - <launched> pid=24470 - [pid=24470][err] JavaScript warning: resource://services-settings/Utils.sys.mjs, line 114: unreachable code after return statement - [pid=24470][out] console.warn: services.settings: Ignoring preference override of remote settings server - [pid=24470][out] console.warn: services.settings: Allow by setting MOZ_REMOTE_SETTINGS_DEVTOOLS=1 in the environment - [pid=24470][out] console.error: ({}) - [pid=24470][out] - [pid=24470][out] Juggler listening to the pipe - [pid=24470][out] console.error: "Warning: unrecognized command line flag" "-wait-for-browser" - [pid=24470][out] - [pid=24470][out] ERROR: Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIPrefBranch.setStringPref] Browser.enable@chrome://juggler/content/protocol/BrowserHandler.js:40:24 - [pid=24470][out]If I run the same code without the firefox_user_prefs setting, I get the following error mesage:
Traceback (most recent call last): File "/home/lmnice/sw/projects/mapebot/Rascunhos/playwright_error.py", line 15, in <module> page = browser.new_page().goto("https://untrusted-root.badssl.com/") File "/home/lmnice/.venvs/error_playwright/lib/python3.10/site-packages/playwright/sync_api/_generated.py", line 8686, in goto self._sync( File "/home/lmnice/.venvs/error_playwright/lib/python3.10/site-packages/playwright/_impl/_sync_base.py", line 115, in _sync return task.result() File "/home/lmnice/.venvs/error_playwright/lib/python3.10/site-packages/playwright/_impl/_page.py", line 519, in goto return await self._main_frame.goto(**locals_to_params(locals())) File "/home/lmnice/.venvs/error_playwright/lib/python3.10/site-packages/playwright/_impl/_frame.py", line 145, in goto await self._channel.send("goto", locals_to_params(locals())) File "/home/lmnice/.venvs/error_playwright/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 59, in send return await self._connection.wrap_api_call( File "/home/lmnice/.venvs/error_playwright/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 514, in wrap_api_call raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from Noneplaywright._impl._errors.Error: Page.goto: SEC_ERROR_UNKNOWN_ISSUERCall log:navigating to "https://untrusted-root.badssl.com/", waiting until "load"I get the same error message as immediately above (SEC_ERROR_UNKOWN_ISSUER) if I run the code with some other random Firefox setting, say:
from playwright.sync_api import sync_playwrightwith sync_playwright() as pwobj: browser = pwobj.firefox.launch( headless = False, firefox_user_prefs = { # random option just to check if other options also cause this error"browser.backspace_action": 0 } ) # page with broken certificate for testing purposes page = browser.new_page().goto("https://untrusted-root.badssl.com/")The error appears to stem from the specific setting security.enterprise_roots.enabled, but other than that I'm stymied.
I would greatly appreciate it if someone would help me getting the playwright version of Firefox to work with custom CAs.