Insufficient output from unittest subTest elements under pytest

Question

I'm interested in using unittest's subTest for looping through some very similar tests. I found that, when I run tests written in this way under pytest (or nosetests), the output does not contain information about the individual failures. Taking the example from the docs:

import unittest

class NumbersTest(unittest.TestCase):
    def test_even(self):
        """
        Test that numbers between 0 and 5 are all even.
        """
        for i in range(0, 6):
            with self.subTest(i=i):
                self.assertEqual(i % 2, 0)

if __name__ == '__main__':
    unittest.main()

If I run python test_even.py, it clearly shows three failures, as expected:

======================================================================
FAIL: test_even (__main__.NumbersTest) (i=1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_even.py", line 10, in test_even
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

======================================================================
FAIL: test_even (__main__.NumbersTest) (i=3)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_even.py", line 10, in test_even
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

======================================================================
FAIL: test_even (__main__.NumbersTest) (i=5)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_even.py", line 10, in test_even
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

----------------------------------------------------------------------
Ran 1 test in 0.002s

FAILED (failures=3)

However, if I run pytest -v test_even.py, it only tells me there was a failure in this test. I can't see which elements failed:

test_even.py::NumbersTest::test_even FAILED                                                                      [100%]

======================================================= FAILURES =======================================================
________________________________________________ NumbersTest.test_even _________________________________________________

self = <test_even.NumbersTest testMethod=test_even>

    def test_even(self):
        """
        Test that numbers between 0 and 5 are all even.
        """
        for i in range(0, 6):
            with self.subTest(i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_even.py:10: AssertionError
=============================================== 1 failed in 0.15 seconds ===============================================

Is there a way to show up the individual failures? Ideally, I'd also like some sort of output for the ones that passed, just to reassure myself that the test discovery is working properly!

Answer 1

It seems that pytest does not yet support subTest. One solution might be to ditch unittest altogether and write native pytest tests:

import pytest

@pytest.mark.parametrize("test_input", range(0, 6))
def test_even(test_input):
    assert test_input % 2 == 0

if __name__ == '__main__':
    pytest.main([__file__])

Answer 2

Once pytest-subtests is added to the environment, this just works with the script from the original question:

$ pytest test_even.py
============================= test session starts ==============================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
rootdir: /net/home/h04/hadru/python
plugins: subtests-0.7.0
collected 1 item                                                               

test_even.py .                                                        [100%]

=================================== FAILURES ===================================
_________________________ NumbersTest.test_even (i=1) __________________________

self = <test_even.NumbersTest testMethod=test_even>

    def test_even(self):
        """
        Test that numbers between 0 and 5 are all even.
        """
        for i in range(0, 6):
            with self.subTest(i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_even.py:10: AssertionError
_________________________ NumbersTest.test_even (i=3) __________________________

self = <test_even.NumbersTest testMethod=test_even>

    def test_even(self):
        """
        Test that numbers between 0 and 5 are all even.
        """
        for i in range(0, 6):
            with self.subTest(i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_even.py:10: AssertionError
_________________________ NumbersTest.test_even (i=5) __________________________

self = <test_even.NumbersTest testMethod=test_even>

    def test_even(self):
        """
        Test that numbers between 0 and 5 are all even.
        """
        for i in range(0, 6):
            with self.subTest(i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_even.py:10: AssertionError
=========================== short test summary info ============================
SUBFAIL test_even.py::NumbersTest::test_even - AssertionError: 1 != 0
SUBFAIL test_even.py::NumbersTest::test_even - AssertionError: 1 != 0
SUBFAIL test_even.py::NumbersTest::test_even - AssertionError: 1 != 0
========================= 3 failed, 1 passed in 0.10s ==========================