Grouping tests in PyTest -Substrings, and Markers

As promised, in one of my previous blogs, here I’m to discuss that powerful feature of grouping tests in PyTest. Grouping of tests is an effective way to perform testing because it helps us to execute or skip certain tests based on criteria. If you are new to PyTest then refer to Getting started with PyTest before this.


Before diving in, quick environment check Python version used : Python 3.7 in PyCharm Install PyTest : pip install pytest Confirm the installation: pytest -h, which will display the help.

We can perform grouping in 2 ways:

  •  Matching Substring

  • Applying Marker


Matching Substring:


We can group a subset of tests just by mentioning a common substring from different tests in different files. For this, naming the tests accordingly is the key.


For example, we have multiple modules test_numbers_one.py and test_numbers_two.py with multiple tests that have test names with sub-strings “arith” and “compare”.


test_numbers_one.py

def test_arithadd():
    a = 6
    b = 3
    assert a + b == 9, "Expected: {}, Result: {}".format(a + b, 9)

def test_arithsubtract():
    a = 6
    b = 3
    assert a - b == 3, "Expected: {}, Result: {}".format(a - b, 3)

def test_compare_small(numbers):
    assert numbers[0] < numbers[1],\
        "{} not less than {}".format(numbers[0],numbers[1])

def test_compare_greater(numbers):
    assert numbers[0] > numbers[1],\
        "{} not greater than {}".format(numbers[0],numbers[1])

 test_numbers_two.py

def test_arith_multiplytest():
    a = 6
    b = 3
    assert a * b == 18 , "Expected: {}, Result: {}".format(a * b, 18)

def test_arith_divide():
    a = 6
    b = 3
    assert a / b == 2, "Expected: {}, Result: {}".format(a / b, 2)

def test_compare_equalto(numbers):
    assert numbers[0] == numbers[1],\
        "{} not equal to {}".format(numbers[0],numbers[1])

To execute tests with a specific substring, use option -k followed by substring. pytest -k arith -v, here “arith” is the substring. 


Below when executed, it collected 7 items / 3 deselected / 4 selected which have “arith” as substring.


C:\Users\PyTest_Project\UserApp>pytest -k arith -v  
=========== test session starts ====================================================================
platform win32 — Python 3.7.7, pytest-5.4.3, py-1.8.2, pluggy-0.13.1 — c:\programs\python\python37\python.exe
cachedir: .pytest_cache
metadata: {‘Python’:3.7.7, ‘Platform’: ‘Windows-1010.0.19041-SP0, ‘Packages’: {‘pytest’:5.4.3, ‘py’:1.8.2, ‘pluggy’:0.13.1}, ‘Plugins’: {‘django’:3.9.0, ‘forked’:1.1.3, ‘html’:2.1.1, ‘metadata’:1.10.0, ‘xdist’:1.32.0},JAVA_HOME:C:\\Pr
ogram Files\\Java\\jdk1.8.0_191’}
rootdir: C:\Users\PyTest_Project\UserApp, inifile: pytest.ini
plugins: django-3.9.0, forked-1.1.3, html-2.1.1, metadata-1.10.0, xdist-1.32.0
collected 7 items / 3 deselected / 4 selected
test_numbers_one.py::test_arithadd PASSED [ 25%]
test_numbers_one.py::test_arithsubtract PASSED [ 50%]
test_numbers_two.py::test_arith_multiplytest PASSED [ 75%]
test_numbers_two.py::test_arith_divide PASSED [100%]
==================================================================== 4 passed, 3 deselected in 0.11s ====================================================================

Now let’s try with “compare” substring.


To execute tests with compare substring, I have used a fixture numbers to initialize numbers. You can place fixture in the same file or in conftest.py. For more details on fixtures refer to this.


Fixture used:

@pytest.fixture
def numbers():
    a=9
    b=3
    return a,b

and when executed, it collected 7 items / 4 deselected / 3 selected which have “compare” as a substring. Here, 2 tests failed for obvious reasons as 9 is not lesser than 3, and 9 is not equal to 3. 


C:\Users\PyTest_Project\UserApp>pytest -k compare -v
================== test session starts =======================================================
platform win32 -- Python 3.7.7, pytest-5.4.3, py-1.8.2, pluggy-0.13.1 -- c:\programs\python\python37\python.exe
cachedir: .pytest_cache
metadata: {'Python': '3.7.7', 'Platform': 'Windows-10-10.0.19041-SP0', 'Packages': {'pytest': '5.4.3', 'py': '1.8.2', 'pluggy': '0.13.1'}, 'Plugins': {'django': '3.9.0', 'forked': '1.1.3', 'html': '2.1.1', 'metadata': '1.10.0', 'xdist': '1.32.0'}, 'JAVA_HOME': 'C:\\Pr
ogram Files\\Java\\jdk1.8.0_191'}
rootdir: C:\Users\PyTest_Project\UserApp, inifile: pytest.ini
plugins: django-3.9.0, forked-1.1.3, html-2.1.1, metadata-1.10.0, xdist-1.32.0
collected 7 items / 4 deselected / 3 selected
test_numbers_one.py::test_compare_small FAILED                                                                                                                                                                                                                       [ 33%]
test_numbers_one.py::test_compare_greater PASSED                                                                                                                                                                                                                     [ 66%]
test_numbers_two.py::test_compare_equalto FAILED                                                                                                                                                                                                                     [100%]
============================================================ FAILURES =============================================================
_______________________________________________________ test_compare_small ________________________________________________________
numbers = (9, 3)
def test_compare_small(numbers):
>       assert numbers[0] < numbers[1],\
            "{} not less than {}".format(numbers[0],numbers[1])
E       AssertionError: 9 not less than 3
E       assert 9 < 3
test_numbers_one.py:12: AssertionError
______________________________________________________ test_compare_equalto _______________________________________________________
numbers = (9, 3)
def