Screenshot for setup and teardown failure

[smanenti]

Hi!

While SeleniumBase is properly taking screenshot when a test is failing, it is not taking a screenshot when a fixture used for setup and teardown - which is called by the test - is failing.
We are using SB inside fixture for example to create user prior a test, and then delete that user after the test.
This is super useful, as if the test is failing due to an error in the fixture, we could then have a screenshot to help troubleshooting.

We wrote a workaround by defining a hook in pytest make report function, and by checking if report.when is "setup" or "teardown" + has failed. Then, we are calling SB to take a screenshot.

While this is working fine now, I was wondering if this is done on purpose (screenshot only for tests), or if we are missing a configuration of SB.

I've tried looking into GitHub for existing topic, but I've found nothing.

Thanks a lot!

[mdmintz]

If using SeleniumBase with pytest fixtures, make sure to use one of the fixture formats:

• https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/syntax_formats.md#sb_sf_03
• https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/syntax_formats.md#sb_sf_04

Eg:

# "sb" pytest fixture test in a method with no class
def test_sb_fixture_with_no_class(sb):
    sb.open("seleniumbase.io/help_docs/install/")
    sb.type('input[aria-label="Search"]', "GUI Commander")
    sb.click('mark:contains("Commander")')
    sb.assert_title_contains("GUI / Commander")

# "sb" pytest fixture test in a method inside a class
class Test_SB_Fixture:
    def test_sb_fixture_inside_class(self, sb):
        sb.open("seleniumbase.io/help_docs/install/")
        sb.type('input[aria-label="Search"]', "GUI Commander")
        sb.click('mark:contains("Commander")')
        sb.assert_title_contains("GUI / Commander")

Screenshots are saved for failing tests (when the failure occurs inside a test). If you are adding additional fixtures and there's a failure inside one of them, that would be something that falls outside the test because fixtures do work before and after a test, similar to a Python context manager.

[smanenti]

Thanks for your answer, I confirm we are using SB fixture in the proper way (inside test class).

Is there any reason why we don't want to take screenshot for failing fixture? IMHO, if there is a way to implement this in SeleniumBase, even if it requires explicit activation, that would be nice.

Thank you very much for the support.

[mdmintz]

This example demonstrates a test failing due to a fixture issue.
(The failure screenshot is saved to the latest_logs/ folder.)

import pytest

@pytest.fixture()
def external_fixture():
    raise Exception("Failed!")

# "sb" pytest fixture test in a method with no class
def test_sb_fixture_with_no_class(sb, request):
    sb.open("example.com")
    e_f = request.getfixturevalue("external_fixture")

As in the example above, in order to get the failure screenshot for a failing fixture, the fixture failure must occur within the test, meaning after setUp() has completed, but before tearDown() starts. The request fixture lets you load other fixtures within your test.

Also see the formats that use the request fixture with sb:

• https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/syntax_formats.md#sb_sf_07
• https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/syntax_formats.md#sb_sf_08