[py][bidi]: add enable_webextensions option for chromium-based browsers

[navin772]

User description

🔗 Related Issues

More info - #15749 (comment)

💥 What does this PR do?

Chrome requires 2 flags for BiDi webextensions to work:

1. --remote-debugging-pipe
2. --enable-unsafe-extension-debugging

This PR adds a chrome option to enable these flags easily - enable_webextensions
User can either use the above chrome option or add the flags manually.

The webExtension.install method will return an exception if the user forgets to add these flags.

Usage:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.enable_webextensions = True
driver = webdriver.Chrome(options=options)

OR directly pass the flags

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--remote-debugging-pipe")
options.add_argument("--enable-unsafe-extension-debugging")

driver = webdriver.Chrome(options=options)

🔧 Implementation Notes

💡 Additional Considerations

🔄 Types of changes

• New feature (non-breaking change which adds functionality and tests!)

---

PR Type

Enhancement, Tests

---

Description

• Adds enable_webextensions option to Chrome for BiDi webextensions

  • Automatically manages required Chrome flags for webextension support
  • Allows enabling/disabling via property or manual flags

• Improves error handling in webExtension.install for missing flags

  • Raises descriptive exception if required flags are missing

• Updates and documents BiDi webextension test for Chrome

  • Explains usage of new option and required flags

• Updates test configuration to enable webextensions for Chrome BiDi

---

Changes walkthrough 📝

	Relevant files
Enhancement	options.py Add `enable_webextensions` property to Chrome options        py/selenium/webdriver/chrome/options.py Adds enable_webextensions property with getter/setter Automatically adds/removes required Chrome flags for webextension support Updates docstrings and internal logic for flag management +37/-0
Bug fix	webextension.py Improve error handling for Chrome webextension install      py/selenium/webdriver/common/bidi/webextension.py Adds error handling for missing Chrome webextension flags Raises descriptive exception if required flags are not set +12/-2
Tests	bidi_webextension_tests.py Update and document BiDi webextension test for Chrome        py/test/selenium/webdriver/common/bidi_webextension_tests.py Updates test to document Chrome webextension enabling Adds logic to handle Chrome-specific extension id behavior Provides usage examples for new option and flags +26/-3
Configuration changes	conftest.py Enable Chrome webextensions in BiDi test configuration      py/conftest.py Enables webextensions for Chrome when BiDi is enabled in test config Checks for presence of new option before setting +4/-0

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-merge-pro]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ❌ 1234 - Not compliant Non-compliant requirements: • Fix issue where JavaScript in link's href is not triggered on click() in Firefox 42.0 Requires further human verification: • Need to verify if the PR changes affect Firefox click() behavior for JavaScript in href attributes 5678 - Not compliant Non-compliant requirements: • Fix "Error: ConnectFailure (Connection refused)" when instantiating ChromeDriver multiple times Requires further human verification: • Need to verify if the new Chrome BiDi webextension implementation affects ChromeDriver connectivity issues

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Flag Management The implementation removes flags when disabling webextensions, but doesn't check if they were manually added by the user. This could lead to unexpected behavior if a user manually added these flags and then toggled the property. else: # Remove webextension flags if disabling flags_to_remove = ["--enable-unsafe-extension-debugging", "--remote-debugging-pipe"] for flag in flags_to_remove: if flag in self._arguments: self._arguments.remove(flag) Exception Handling The error handling only checks for "Method not available" in the exception message, which might be fragile if error messages change in the future. Consider a more robust way to detect missing Chrome flags. if "Method not available" in str(e): raise WebDriverException( f"{str(e)}. If you are using Chrome, add '--enable-unsafe-extension-debugging' " "and '--remote-debugging-pipe' arguments or set options.enable_webextensions = True" ) from e

[qodo-merge-pro]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Learned best practice	Add parameter validation check Add validation at the beginning of the method to check if at least one of the parameters is provided. This prevents potential errors when all parameters are None, which would lead to an undefined extension_data variable. py/selenium/webdriver/common/bidi/webextension.py [50-58] def install(self, path=None, archive_path=None, base64_value=None) -> Dict: """Installs a web extension from the remote end. :Args: - path: Path to the extension directory - archive_path: Path to the extension archive - base64_value: Base64 encoded extension archive :Returns: A dictionary containing the extension ID and other details """ + if path is None and archive_path is None and base64_value is None: + raise ValueError("At least one of path, archive_path, or base64_value must be provided") + if path is not None: extension_data = {"type": "path", "path": path} elif archive_path is not None: extension_data = {"type": "archivePath", "path": archive_path} elif base64_value is not None: extension_data = {"type": "base64", "value": base64_value} params = {"extensionData": extension_data} [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 6 __ Why: Relevant best practice - Add null checks for parameters and properties before using them to prevent exceptions. Validate input parameters at the beginning of methods.	Low
General	Preserve exception traceback The exception handling creates a new WebDriverException that loses the original exception's traceback. Use raise ... from e syntax for all exception paths to preserve the original traceback information, which helps with debugging. py/selenium/webdriver/common/bidi/webextension.py [59-68] try: result = self.conn.execute(command_builder("webExtension.install", params)) return result except WebDriverException as e: if "Method not available" in str(e): raise WebDriverException( f"{str(e)}. If you are using Chrome, add '--enable-unsafe-extension-debugging' " "and '--remote-debugging-pipe' arguments or set options.enable_webextensions = True" ) from e - raise + raise e from e Apply / Chat Suggestion importance[1-10]: 3 __ Why: The suggestion to use raise e from e in the generic exception path is unnecessary, as a bare raise already preserves the traceback. The current code already uses raise ... from e for the custom message, so this change offers only marginal clarity and does not impact functionality or correctness.	Low
Update

[navin772]

Is there a better way to directly use the enable_webextensions chrome option in the test instead of modifying the conftest.py file?

[navin772]

@oliverdunk can you have a look at this PR?

[cgoldberg]

Is there a better way to directly use the enable_webextensions chrome option in the test instead of modifying the conftest.py file?

I guess the issue is that you don't want to add enable_webextensions to the driver fixture because it will affect most tests?

You could create the driver in the test and not use the fixture. We do that in a few tests. You could add a new fixture similar to the firefox_options fixture in conftest.py. Then use this fixture in your test, and set enable_webextensions from inside the test.

It's not a great solution, but I think it will help.

(let me know if that's not a good explanation)

[navin772]

@cgoldberg I have added a new pytest marker webextension and a check in conftest.py to set enable_webextensions to True only when this pytest marker is used (thus other tests won't be affected).

[shbenzer]

Is there a better way to directly use the enable_webextensions chrome option in the test instead of modifying the conftest.py file?

I guess the issue is that you don't want to add enable_webextensions to the driver fixture because it will affect most tests?

You could create the driver in the test and not use the fixture. We do that in a few tests. You could add a new fixture similar to the firefox_options fixture in conftest.py. Then use this fixture in your test, and set enable_webextensions from inside the test.

It's not a great solution, but I think it will help.

(let me know if that's not a good explanation)

I agree with this - I don't think the new marker or conftest changes are necessary. We can create a driver specifically for the webextension tests, and then put a xfail_firefox marker (e.g. with reason "not implemented") on those tests for now. You can wrap the webextension tests in a test class to make this easier.

[navin772]

We can create a driver specifically for the webextension tests

Will the new driver be created inside the webextension test file or conftest.py? I am trying the former one with a class based approach (different class for Firefox and chrome/edge) and getting errors like the extension won't load (the install method won't fail but I can't see the extension being loaded in the browser)

Edit: Extension is loaded but pages.load("blank.html") is opening the page in the window that doesn't have the extension installed.

[cgoldberg]

Will the new driver be created inside the webextension test file or conftest.py

The solution I'm suggesting would create it within the test.

It's going to be difficult to have a common test that supports all browsers. I would write a test for Firefox (that's skipped by other browsers) and one for Chrome/Edge (that's skipped by other browsers). For an example, look at ./py/test/selenium/webdriver/firefox/firefox_sizing_tests.py.

[navin772]

@cgoldberg yes, I am trying it as you described above. I noticed one issue right now which results in my tests getting failures, while launching the driver, it opens 2 browser windows, extension installation is successful and when I do pages.load("blank.html"), it opens the blank.html page in the window that doesn't have the extension installed (the other tab has it installed).
If I do driver.get("https://www.webpagetest.org/blank.html") instead of pages.load("blank.html") it works fine.

I am trying to fix this.

[navin772]

I think pages.load uses the normal driver and not the chromium_driver I added. For now, only this is working:

chromium_driver.get("https://www.webpagetest.org/blank.html")

@cgoldberg @shbenzer can you please have a look and see if something can be done so that pages.load work as expected.

[shbenzer]

I think pages.load uses the normal driver and not the chromium_driver I added. For now, only this is working:

chromium_driver.get("https://www.webpagetest.org/blank.html")

the pages fixture is defined in conftest.py as the following:

@pytest.fixture
def pages(driver, webserver):
    class Pages:
        def url(self, name, localhost=False):
            return webserver.where_is(name, localhost)

        def load(self, name):
            driver.get(self.url(name))

    return Pages()

You're correct. The problem is this uses the initial driver - you'll need to create your own Pages class if you wish to use this functionality

[navin772]

you'll need to create your own Pages class

@shbenzer I guess that will be a lot of changes to do just for a single test file, also we have an issue in running this multi driver approach in CI-RBE which is giving a user data dir error (maybe due to these 2 drivers), although browser tests on github runner are working.

[shbenzer]

you'll need to create your own Pages class

@shbenzer I guess that will be a lot of changes to do just for a single test file, also we have an issue in running this multi driver approach in CI-RBE which is giving a user data dir error (maybe due to these 2 drivers), although browser tests on github runner are working.

I don't think it's a big deal adding the class in if you'll use it, but do what you think is best.

With regards to the error you're seeing in RBE:

selenium.common.exceptions.SessionNotCreatedException: Message: session not created: probably user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir

You gotta love when an error message uses "probably"... You could try creating a temporary directory then setting it in chromium_driver's options with something like options.add_argument(f"--user-data-dir={temp_dir}")?

[cgoldberg]

@navin772
few things...

That SessionNotCreatedException is a super generic error that just means it couldn't launch the browser. It likely has nothing to do with -user-data-dir like the message says.

In your tests, you are instantiating ChromeOptions and EdgeOptions, but they don't include some of the default options we need to set from outside of the test (like the ability to add the --headless arg). Instead of creating them in the test file, you probably want to add a fixture to conftest.py for setting options, then create the driver in your test using the options fixture. Take a look at the firefox_options fixture. You could create a similar ones for chrome/edge, or combine them into one fixture that can handle options for any driver.

I don't think the pages_chromium fixture you added is going to work quite right, because it still uses the pages fixture which will launch a driver. You could use the webserver fixture instead, and then just navigate to the page in your test.

[navin772]

I don't think the pages_chromium fixture you added is going to work quite right, because it still uses the pages fixture which will launch a driver.

It does work because chromium_driver.get((self.url(https://github.com/name)) will launch the page using the chromium_driver not the pages driver. But still it can be simplified more as you said.

I will try to move the fixture to conftest.py.

[diemol]

Posting the same comment I posted on Slack:

Actually, ChatGPT helped me understand why this happens.
Since we are setting the location of the binary in those tests, this is ChatGPT’s explanation:

This test overrides the Chrome binary location, and probably (via getBinaryPaths) pulls a shared browserPath and driverPath that are used across tests.
Even if you’re not explicitly setting --user-data-dir, Chrome internally defaults to a shared user data dir unless told otherwise. And reusing a binary manually can cause odd behavior — particularly if Chrome launches before the profile lock is released.
That would explain:
Only this test failing (because it’s the only one using a custom binary),
Only on Ubuntu CI (Chrome on Linux is stricter about .lock files in the profile).

This is how I fixed it in our docs examples.
SeleniumHQ/seleniumhq.github.io@ffbf252

[cgoldberg]

LGTM 👍

is this ready to be merged? Let me know and I'll pull the trigger.