13

I see from here that I can pick out tests based on their mark like so:

pytest -v -m webtest

Let's say I have a test decorated like so:

@pytest.mark.parametrize('platform,configuration', (
    pytest.param('win', 'release')
    pytest.param('win', 'debug')))
def test_Foo(self):

I'd like to do something like the following:

pytest -v -m parameterize.configuration.release

That way I run test_Foo with the parameter of 'release' but not the parameter of 'debug'. Is there a way to do this? I think I can do this by writing a wrapper test and then passing only the desired parameter down, but I want to avoid doing so because we already have a large number of tests parameterized as describe, and I want to avoid writing a large number of wrapper tests.

4
  • 1
    -m is for exact matching of markers so it's not applicable here, but you can use -k instead, for example pytest -k win-release or pytest -k release.
    – hoefling
    Oct 12, 2018 at 20:16
  • 1
    if that's not enough, you can always add your own logic that e.g. selects only tests having a parameter by name or name and value etc.
    – hoefling
    Oct 12, 2018 at 20:20
  • This seems to do exactly what I need, thanks! I'm still testing it but so far no nasty edge cases. If you put it as a separate answer I'll accept it. Oct 16, 2018 at 22:26
  • Glad I could help! I've also added an example of adding custom filters as mentioned in the comment, should you be interested in further customization.
    – hoefling
    Oct 16, 2018 at 23:06

2 Answers 2

10

You can use -k for expression-based filtering:

$ pytest -k win-release

will run only tests containing win-release in their names. You can list all names without executing the tests by issuing

$ pytest --collect-only -q

Should an expression be not enough, you can always extend pytest by adding your custom filtering logic, for example by passing parameter name and value via command line args and selecting only tests that are parametrized accordingly:

# conftest.py

def pytest_addoption(parser):
    parser.addoption('--param-name', action='store',  help='parameter name')
    parser.addoption('--param-value', action='store',  help='parameter value')


def pytest_collection_modifyitems(session, config, items):
    param_name = config.getoption('--param-name')
    param_value = config.getoption('--param-value')
    if param_name and param_value:
        items[:] = [item for item in items
                    if hasattr(item, 'callspec')
                    and param_name in item.callspec.params
                    and item.callspec.params[param_name] == param_value]

Now you can e.g. call

$ pytest --param-name=platform --param-value=win

and only tests parametrized with platform=win will be executed.

2
  • 1
    pytest --param-name=platform --param-name=win should be pytest --param-name=platform --param-value=win Apr 19, 2022 at 9:49
  • @SamuelVerschelde ah good catch! Updated the answer.
    – hoefling
    Apr 25, 2022 at 7:54
2

An alternative to the official answer by hoefling is to create a special marker with pytest-pilot and to apply it:

conftest.py:

from pytest_pilot import EasyMarker

mymark = EasyMarker('mymark', has_arg=False, mode='hard_filter')

test_so.py:

import pytest
from .conftest import mymark

@pytest.mark.parametrize('platform,configuration', (
        mymark.param('win', 'release'),
        pytest.param('win', 'debug')
))
def test_foo(platform, configuration):
    pass

You can now run pytest --mymark, it correctly runs only the test with the mark

test_so\test_so.py::test_foo[win-release] PASSED                         [ 50%]
test_so\test_so.py::test_foo[win-debug] SKIPPED                          [100%]

Of course it might not be relevant in all cases, as it requires code modification ; however for advanced filtering patterns, or if the filtering is here to stay and you wish to have some CLI shortcuts to perform it, it might be interesting. Note: I'm the author of this lib ;)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.