70

Is it possible to create an abstract TestCase, that will have some test_* methods, but this TestCase won't be called and those methods will only be used in subclasses? I think I am going to have one abstract TestCase in my test suite and it will be subclassed for a few different implementation of a single interface. This is why all test methods are the some, only one, internal method changes. How can I do it in elegant way?

3

14 Answers 14

82

I didn't quite understand what do you plan to do -- the rule of thumb is "not to be smart with tests" - just have them there, plain written.

But to achieve what you want, if you inherit from unittest.TestCase, whenever you call unittest.main() your "abstract" class will be executed - I think this is the situation you want to avoid.

Just do this: Create your "abstract" class inheriting from "object", not from TestCase. And for the actual "concrete" implementations, just use multiple inheritance: inherit from both unittest.TestCase and from your abstract class.

import unittest

class Abstract(object):
    def test_a(self):
        print "Running for class", self.__class__

class Test(Abstract, unittest.TestCase):
    pass

unittest.main()

update: reversed the inheritance order - Abstract first so that its defintions are not overriden by TestCase defaults, as well pointed in the comments bellow.

11
  • 3
    This is how I wanted to do this in the first place. I want to be smart, because I am running my test against different database. The interface for accessing the database is always the same, so I just need to instantise my connection to the database and the always the same tests are run. I don't know, if that's too smart for tests, but I just don't like to type a lot ;-)
    – gruszczy
    Dec 30, 2010 at 23:16
  • 1
    Wow, years of madness with multiple inheritance in C++ scared me off MI (in my defence, I inherited the codebase - the evil predated my involvement). Now I see an actual valid use.
    – LRE
    Feb 1, 2014 at 23:35
  • 11
    Take note of a common pitfall: in case you override TestCase method in Abstract (overriding "setUp" is the common case), you want to reverse the order of base classes - class Test(Abstract, unittest.TestCase). Otherwise the empty setUp method in unittest.TestCase takes precedence over the implemented one in Abstract.
    – Yonatan
    May 15, 2014 at 7:13
  • 3
    @Yonatan makes a good point, and reversing the inheritance order seems like a more sensible default. Why would you ever want your custom base class overridden by TestCase? Mar 31, 2015 at 23:07
  • 1
    This is often called SomethingMixin. Jun 16, 2015 at 22:39
26

There's a very simple way that everyone has missed so far. And unlike several of the answers, it works with all test drivers, rather than failing the minute you switch between them.

Simply use inheritence as usual, then add:

del AbstractTestCase

at the end of the module. This will prevent the Abstract class from being executed.

Example:

import unittest
from abc import abstractmethod, ABC

class AbstractSomethingTest(ABC, unittest.TestCase):
   @abstractmethod
   def implement_me(self):
      pass

   def test_implemented_method_should_return_inri(self):
      self.assertEqual("INRI", self.implement_me())


class ImplementedSomethingTest(AbstractSomethingTest):
   def implement_me(self):
      return "INRI"

class AnotherClassImplementedSomethingTest(AbstractSomethingTest):
   def implement_me(self):
      return "INRI"

del AbstractSomethingTest
7
  • Are there any drawbacks?
    – Max Malysh
    Aug 29, 2017 at 4:03
  • @MaxMalysh You can't use old-style calls to the superclass. But super still works, and everyone should really be using that.
    – o11c
    Aug 29, 2017 at 4:13
  • 3
    From all the solutions I found I like this one the most. Mixins either lack support for asserts or if you use them they may be available during runtime but they do raise warnings in the IDE. The del statement at the end of a module may be easily overlooked and the intention is not that obvious but still I think this is the cleanest of all options when one really just has like a few variables that change and all the tests stay the same. Or did you come up with a better solution in the meantime?
    – Viktor
    Jul 2, 2019 at 14:23
  • 3
    Please expand on this answer - I almost missed it. Explain how this solves: TypeError: Can't instantiate abstract class AbstractTestCase with abstract methods my_help_method,... Explain where to put the del (in the modules that inherit the base) and what to name the module that contains the base so it is ignored.
    – David
    Jul 25, 2019 at 18:12
  • 6
    This doesn't work if the abstract base class is in a common module that other test modules want to import.
    – Pi Delport
    Jan 12, 2021 at 12:31
19

If you really want to use inheritance instead of mixins, a simple solution is to nest the abstract test in another class.

It avoids issues with test runner discovery and you can still import the abstract test from another module.

import unittest

class AbstractTests(object):
    class AbstractTest(unittest.TestCase)
        def test_a(self):
            print "Running for class", self.__class__

class Test(AbstractTests.AbstractTest):
    pass
4
  • You were reading my mind! I just found Vadim's solution on this other thread and agree completely that this is more elegant bc it doesn't require any modification to TestLoader. I have undone my TestLoader changes and tested the nested class trick. Thanks! stackoverflow.com/a/25695512/532621
    – timblaktu
    Oct 19, 2018 at 19:34
  • Done. Also edited my answer about the nested class being better. Thanks.
    – timblaktu
    Oct 19, 2018 at 19:38
  • This really seems like the cleanest solution. When using multi-inheritance I had some problems with mypy for the multi-inheriting classes. The other solution with deleting the abstract class only works if you have the abstract and implemented class defined in the same file one after another Mar 22, 2023 at 8:58
  • The best answer. Unlike multiple inheritance this method can define unittest.TestCase class which 1) implements all functionality including all super methods like self.assertRaises, ... 2) it is not invoked during unittests Sep 14, 2023 at 10:35
13

Multiple inheritance isn't a great option here, chiefly for the two following reasons:

  1. None of the methods in TestCase use super() so you'd have to list your class first for methods like setUp() and tearDown() to work.
  2. pylint will warn that the base class uses self.assertEquals() etc which aren't defined on self at that point.

Here's the kludge I came up with: turn run() into a no-op for the base class only.

class TestBase( unittest.TestCase ):

  def __init__( self, *args, **kwargs ):
    super( TestBase, self ).__init__( *args, **kwargs )
    self.helper = None
    # Kludge alert: We want this class to carry test cases without being run
    # by the unit test framework, so the `run' method is overridden to do
    # nothing.  But in order for sub-classes to be able to do something when
    # run is invoked, the constructor will rebind `run' from TestCase.
    if self.__class__ != TestBase:
      # Rebind `run' from the parent class.
      self.run = unittest.TestCase.run.__get__( self, self.__class__ )                          
    else:
      self.run = lambda self, *args, **kwargs: None

  def newHelper( self ):
    raise NotImplementedError()

  def setUp( self ):
    print "shared for all subclasses"
    self.helper = self.newHelper()

  def testFoo( self ):
    print "shared for all subclasses"
    # test something with self.helper

class Test1( TestBase ):
  def newHelper( self ):
    return HelperObject1()

class Test2( TestBase ):
  def newHelper( self ):
    return HelperObject2()
1
  • 3
    (1) is not really a problem - the obvious thing to do is to have your class come first and use super (My answer had this inverted - I just corrected it). (2) The very word you use says it - you have to use a "kludge" not to use a clean mechanism built into Python 's OOP working from the very beggining just because a 3rd party linting tool can't introspect properly. That is your choice of tools.
    – jsbueno
    Jul 7, 2015 at 15:09
9

Just to put in my two-cents, although it likely goes against some convention, you could define your abstract test case as a protected member to prevent its execution. I've implemented the following in Django and works as required. See example below.

from django.test import TestCase


class _AbstractTestCase(TestCase):

    """
    Abstract test case - should not be instantiated by the test runner.
    """

    def test_1(self):
        raise NotImplementedError()

    def test_2(self):
        raise NotImplementedError()


class TestCase1(_AbstractTestCase):

    """
    This test case will pass and fail.
    """

    def test_1(self):
        self.assertEqual(1 + 1, 2)


class TestCase2(_AbstractTestCase):

    """
    This test case will pass successfully.
    """

    def test_1(self):
        self.assertEqual(2 + 2, 4)

    def test_2(self):
        self.assertEqual(12 * 12, 144)
2
  • 4
    This seems to work great and it's much simpler than the monkey-patch approach above.
    – Fasaxc
    May 11, 2015 at 18:35
  • 3
    It may work with django (and in my case, with nose), but the unittest discovery mechanism still seems to find these abstract test cases... Mar 8, 2017 at 6:37
7

Raise unittest.SkipTest in setUpClass()

Just another approach is to raise a unittest.SkipTest in setUpClass() of the base class and override setUpClass() in child classes:

class BaseTestCase(TestCase):
    @classmethod
    def setUpClass(cls):
        "Child classes must override this method and define cls.x and cls.y"
        raise unittest.SkipTest

    def test_x(self):
        self.assertEqual(self.x * 3, self.x)

    def test_y(self):
        self.assertEqual(self.y * 3, self.y + self.y + self.y)

    def test_z(self):
        self.assertEqual(self.x + self.y, self.y)


class IntegerTestCase(BaseTestCase):
    @classmethod
    def setUpClass(cls):
        cls.x = 0
        cls.y = 2


class StringTestCase(BaseTestCase):
    @classmethod
    def setUpClass(cls):
        cls.x = ''
        cls.y = 'zuzuka'

If you need to use custom TestCase that defines its own setUpClass() and you need to call super().setUpClass(), you can define you own method to "set up data" and raise SkipTest only inside that method:

class BaseTestCase(ThidPartyTestCase):
    @classmethod
    def setUpClass(cls):
        super().setUpClass()  # if ThirdPartyTestCase has own setUpClass()
        cls.setUpTestCaseData()

    @classmethod
    def setUpTestCaseData(cls):
        "Override and set up cls.x and cls.y here"
        raise unittest.SkipTest

    ...  # tests


class IntegerTestCase(BaseTestCase):
    @classmethod
    def setUpTestCaseData(cls):
        cls.x = 0
        cls.y = 2
1
  • I think that the extra setUpTestCaseData method here is a bit redundant. See my answer for a cleaner solution.
    – typeracer
    Jun 9, 2020 at 23:36
5

The unittest module provides several options for skipping tests.

My preferred solution is to override the setUpClass method in the "abstract" base class to raise a unittest.SkipTest exception if needed:

class BaseTestCase(unittest.TestCase):
  @classmethod
  def setUpClass(cls):
    if cls is BaseTestCase:
      raise unittest.SkipTest("%s is an abstract base class" % cls.__name__)
    else:
      super(BaseTestCase, cls).setUpClass()
1
  • So it will be instantiated and if it is inheriting from abc.ABC with abstract methods it will throw error. Jul 27, 2022 at 11:27
4

If you follow the convention of explicitly listing all test classes in run_unittest (see e.g. the Python test suite for many uses of that convention), then it will be straight-forward to not list a specific class.

If you want to continue using unittest.main, and if you can allow using unittest2 (e.g. from Python 2.7), you can use its load_tests protocol to specify which classes contain test cases). In earlier versions, you will have to subclass TestLoader, and override loadTestsFromModule.

1
  • Won't I lose then the ability to invoke those tests from the command line? I would like to by default run all tests (except the ones from the abstract test case), but in certain cases only some of them.
    – gruszczy
    Dec 30, 2010 at 23:08
4

Python unittest library has load_tests protocol, which can be used to achieve exactly what do you want:

# Add this function to module with AbstractTestCase class
def load_tests(loader, tests, _):
    result = []
    for test_case in tests:
        if type(test_case._tests[0]) is AbstractTestCase:
            continue
        result.append(test_case)
    return loader.suiteClass(result)
1
  • IndexError: list index out of range Jul 27, 2022 at 11:16
1

Another reason for wanting to do what the OP is doing is to create a highly-parameterized base class which implements much of a set of core tests which need to be reproduced in several environments/scenarios. What I'm describing is essentially creating a parameterized fixture, a la pytest, using unittest.

Assuming you (like me) decide to run away as fast as you can from any multiple-inheritance-based solutions, one might have the following problem with using load_tests() to filter out your base class from the loaded suite:

In the standard TestLoader, load_tests is called after the auto-loading-from-class is done. Because: * this auto-loading-from-class will attempt to construct instances from your base class with the standard signature init(self, name), and * you may want this base class to have a very different ctor signature, or * you may want to skip construction-then-removal of your base class instances for some other reason

.. you may want to completely prevent this auto-loading of test instances from base classes.

EDIT: Vadim's solution in this other thread is a more elegant, concise, and independent way to do this. I have implemented the "nested class trick" and confirmed it works beautifully for the purpose of preventing TestLoader from "finding" your TestCase bases.

I originally had done this by modifying TestLoader.loadTestsFromModule to simply skip any TestCase classes which serve as base classes for any other TestCase classes in the module:

for name in dir(module):
    obj = getattr(module, name)
    # skip TestCase classes:
    # 1. without any test methods defined
    # 2. that are base classes
    #    (we don't allow instantiating TestCase base classes, which allows test designers
    #     to implement actual test methods in highly-parametrized base classes.)
    if isinstance(obj, type) and issubclass(obj, unittest.TestCase) and \
            self.getTestCaseNames(obj) and not isbase(obj, module):
        loaded_suite = self.loadTestsFromTestCase(obj)
        # ignore empty suites
        if loaded_suite.countTestCases():
            tests.append(loaded_suite)

where:

def isbase(cls, module):
    '''Returns True if cls is base class to any classes in module, else False.'''
    for name in dir(module):
        obj = getattr(module, name)
        if obj is not cls and isinstance(obj, type) and issubclass(obj, cls):
            return True
    return False

The parametrization I spoke of above is implemented by having each child class define it's fixture details (the parameters) and pass them to the base class TestCase ctor so that all of its common impl methods (the "fixturey" ones setUp*/tearDown*/cleanup* and the test methods themselves) have all the info that defines the now very specific fixture that that child TestCase class is to operate on.

For me, this was a temporary solution for quickly implementing some parametrized fixtures in unittest, since I plan to move my team's tests to pytest asap.

1

If you set __test__ = False in the base test class it should disable its tests. Quoting from this link:

class MessageTestBase(unittest.TestCase):
    __test__ = False

    def setUp(self):
        self.status = 'running'

    def tearDown(self):
        self.status = 'not running'

    def test_common_test(self):
        self.assertEqual(self.status, 'running')


class TestSlack(MessageTestMixin):
    __test__ = True 

Notice the differences. Our mixin becomes a base class that inherits from TestCase. we include __test__ = False in the base class to prevent the test runner from executing tests in this class. Then the child class only inherits from MessageTestBase and includes __test__ = True to instruct the test runner to run our tests.

More details here: How does __test__ = False magic attribute work for test discovery

0

Here's a relatively simple approach that allows your common tests to inherit from TestCase (so type checking and IDE tooling stays happy), that uses only documented unittest features, and that avoids the "skip" test status:

import unittest

class CommonTestCases(unittest.TestCase):
    def __init__(self, methodName='runTest'):
        if self.__class__ is CommonTestCases:
            # don't run these tests on the abstract base implementation
            methodName = 'runNoTestsInBaseClass'
        super().__init__(methodName)

    def runNoTestsInBaseClass(self):
        print('not running tests in abstract base class')
        pass

    def test_common(self):
        # This will run *only* in subclasses. Presumably, this would 
        # be a test you need to repeat in several different contexts.
        self.assertEqual(2 + 2, 4)


class SomeTests(CommonTestCases):
    # inherited test_common *will* be run here

    def test_something(self):
        self.assertTrue(True)


# Also plays nicely with MRO, if needed:
class SomeOtherTests(CommonTestCases, django.test.SimpleTestCase):
    # inherited test_common *will* be run here

    def test_something_else(self):
        self.client.get('/')  # ...

How it works: per the unittest.TestCase documentation, "Each instance of TestCase will run a single base method: the method named methodName." The default "runTests" runs all the test* methods on the class—that's how TestCase instances normally work. But when running in the abstract base class itself, you can simply override that behavior with a method that does nothing.

A side effect is your test count will increase by one: the runNoTestsInBaseClass "test" gets counted as a successful test when it's run on CommonTestCases.

0

I know it's an old thread but it might help someone: I had a similar problem with my tests in a Django app that I had to upgrade from version 2.2 to version 4.2.

Previously, I’ve been using the django-nose library which has a nose based custom unit test runner, built on top of Django's own unit test runner. The feature asked by the OP was covered by the nose test runner. I think django-nose only run the tests of a test class if the name of the class starts as Test - hence I adopted a convention to name my abstract test classes as _Test.

Since django-nose (and nose) is in maintenance mode I had drop it but Django's built-in test runner (unittest2) collects and run all tests whose class is a subclass of unittest.TestCase.

So my solution was to subclass Django's DiscoverRunner the following way:

from unittest import TestSuite
from django.test.runner import DiscoverRunner


class CustomDiscoverRunner(DiscoverRunner):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.filtered_out = 0

    def build_suite(self, test_labels=None, extra_tests=None, **kwargs) -> TestSuite:
        print("Custom DiscoverRunner starts up...")
        suite = super().build_suite(test_labels=test_labels, extra_tests=extra_tests, **kwargs)

        # noinspection PyProtectedMember
        # pylint: disable=protected-access
        filtered_tests = [t for t in suite._tests if self._needed(t)]

        print(f"Running {len(filtered_tests)} tests -- filtered out: {self.filtered_out}")
        return TestSuite(tests=filtered_tests)

    def _needed(self, test_name) -> bool:
        """
        DiscoverRunner will collect all tests from all classes that are subclassing unittest.TestCase

        This custom runner should only keep those tests whose class name starts with "Test{ExampleClass}"
        This is to make sure that tests in abstract test classes will be filtered out
        As a guideline, such classes should follow the "_Test{AbstractTestClass}" naming pattern

        Tests with _Failed class name component are kept to make sure errors will be reported correctly:
        if there is an error at test load time, test gets the class assigned as "unittest.loader._FailedTest"
        This happens when using incorrect test labels: e.g.: "python manage.py test NoModuleLikeThis"
        Keeping such failures in the suite will make sure, they will "run" and the error of incorrect label is reported
        """
        if (
                test_name.__class__.__name__.startswith("Test") or
                test_name.__class__.__name__.startswith("_Failed")
        ):
            return True
        self.filtered_out += 1
        return False

After this the custom runner can be setup to be the default runner by setting the corresponding settings variable:

TEST_RUNNER = 'myproject.unittest_runner.CustomDiscoverRunner'
-1

I have done it following way, maybe it can inspire you:

class AbstractTest(TestCase):
    def setUp(self):
        pass

    def tearDown(self):
        pass

    def _test_1(self):
        # your test case here

class ConcreteTest(AbstractTest)

    def test_1(self):
        self._test_1()

Although it's not the most convenient solution, it lets you escape from multi inheritance. Also, the solution suggested by Dan Ward didn't work with Django testing in PyCharm.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.