Is it possible to create an abstract TestCase
, that will have some test_* methods, but this TestCase
won't be called and those methods will only be used in subclasses? I think I am going to have one abstract TestCase
in my test suite and it will be subclassed for a few different implementation of a single interface. This is why all test methods are the some, only one, internal method changes. How can I do it in elegant way?
15 Answers
I didn't quite understand what do you plan to do -- the rule of thumb is "not to be smart with tests" - just have them there, plain written.
But to achieve what you want, if you inherit from unittest.TestCase, whenever you call unittest.main() your "abstract" class will be executed - I think this is the situation you want to avoid.
Just do this: Create your "abstract" class inheriting from "object", not from TestCase. And for the actual "concrete" implementations, just use multiple inheritance: inherit from both unittest.TestCase and from your abstract class.
import unittest
class Abstract(object):
def test_a(self):
print "Running for class", self.__class__
class Test(Abstract, unittest.TestCase):
pass
unittest.main()
update: reversed the inheritance order - Abstract
first so that its defintions are not overriden by TestCase
defaults, as well pointed in the comments bellow.
-
3This is how I wanted to do this in the first place. I want to be smart, because I am running my test against different database. The interface for accessing the database is always the same, so I just need to instantise my connection to the database and the always the same tests are run. I don't know, if that's too smart for tests, but I just don't like to type a lot ;-)– gruszczyCommented Dec 30, 2010 at 23:16
-
1Wow, years of madness with multiple inheritance in C++ scared me off MI (in my defence, I inherited the codebase - the evil predated my involvement). Now I see an actual valid use.– LRECommented Feb 1, 2014 at 23:35
-
11Take note of a common pitfall: in case you override TestCase method in Abstract (overriding "setUp" is the common case), you want to reverse the order of base classes -
class Test(Abstract, unittest.TestCase)
. Otherwise the empty setUp method in unittest.TestCase takes precedence over the implemented one in Abstract.– YonatanCommented May 15, 2014 at 7:13 -
3@Yonatan makes a good point, and reversing the inheritance order seems like a more sensible default. Why would you ever want your custom base class overridden by
TestCase
? Commented Mar 31, 2015 at 23:07 -
1
There's a very simple way that everyone has missed so far. And unlike several of the answers, it works with all test drivers, rather than failing the minute you switch between them.
Simply use inheritence as usual, then add:
del AbstractTestCase
at the end of the module. This will prevent the Abstract class from being executed.
Example:
import unittest
from abc import abstractmethod, ABC
class AbstractSomethingTest(ABC, unittest.TestCase):
@abstractmethod
def implement_me(self):
pass
def test_implemented_method_should_return_inri(self):
self.assertEqual("INRI", self.implement_me())
class ImplementedSomethingTest(AbstractSomethingTest):
def implement_me(self):
return "INRI"
class AnotherClassImplementedSomethingTest(AbstractSomethingTest):
def implement_me(self):
return "INRI"
del AbstractSomethingTest
-
-
@MaxMalysh You can't use old-style calls to the superclass. But
super
still works, and everyone should really be using that.– o11cCommented Aug 29, 2017 at 4:13 -
3From all the solutions I found I like this one the most. Mixins either lack support for asserts or if you use them they may be available during runtime but they do raise warnings in the IDE. The del statement at the end of a module may be easily overlooked and the intention is not that obvious but still I think this is the cleanest of all options when one really just has like a few variables that change and all the tests stay the same. Or did you come up with a better solution in the meantime?– ViktorCommented Jul 2, 2019 at 14:23
-
3Please expand on this answer - I almost missed it. Explain how this solves:
TypeError: Can't instantiate abstract class AbstractTestCase with abstract methods my_help_method,...
Explain where to put thedel
(in the modules that inherit the base) and what to name the module that contains the base so it is ignored.– DavidCommented Jul 25, 2019 at 18:12 -
7This doesn't work if the abstract base class is in a common module that other test modules want to import. Commented Jan 12, 2021 at 12:31
If you really want to use inheritance instead of mixins, a simple solution is to nest the abstract test in another class.
It avoids issues with test runner discovery and you can still import the abstract test from another module.
import unittest
class AbstractTests(object):
class AbstractTest(unittest.TestCase)
def test_a(self):
print "Running for class", self.__class__
class Test(AbstractTests.AbstractTest):
pass
-
You were reading my mind! I just found Vadim's solution on this other thread and agree completely that this is more elegant bc it doesn't require any modification to TestLoader. I have undone my TestLoader changes and tested the nested class trick. Thanks! stackoverflow.com/a/25695512/532621 Commented Oct 19, 2018 at 19:34
-
Done. Also edited my answer about the nested class being better. Thanks. Commented Oct 19, 2018 at 19:38
-
This really seems like the cleanest solution. When using multi-inheritance I had some problems with mypy for the multi-inheriting classes. The other solution with deleting the abstract class only works if you have the abstract and implemented class defined in the same file one after another Commented Mar 22, 2023 at 8:58
-
The best answer. Unlike multiple inheritance this method can define unittest.TestCase class which 1) implements all functionality including all super methods like self.assertRaises, ... 2) it is not invoked during unittests Commented Sep 14, 2023 at 10:35
Multiple inheritance isn't a great option here, chiefly for the two following reasons:
- None of the methods in
TestCase
usesuper()
so you'd have to list your class first for methods likesetUp()
andtearDown()
to work. - pylint will warn that the base class uses
self.assertEquals()
etc which aren't defined onself
at that point.
Here's the kludge I came up with: turn run()
into a no-op for the base class only.
class TestBase( unittest.TestCase ):
def __init__( self, *args, **kwargs ):
super( TestBase, self ).__init__( *args, **kwargs )
self.helper = None
# Kludge alert: We want this class to carry test cases without being run
# by the unit test framework, so the `run' method is overridden to do
# nothing. But in order for sub-classes to be able to do something when
# run is invoked, the constructor will rebind `run' from TestCase.
if self.__class__ != TestBase:
# Rebind `run' from the parent class.
self.run = unittest.TestCase.run.__get__( self, self.__class__ )
else:
self.run = lambda self, *args, **kwargs: None
def newHelper( self ):
raise NotImplementedError()
def setUp( self ):
print "shared for all subclasses"
self.helper = self.newHelper()
def testFoo( self ):
print "shared for all subclasses"
# test something with self.helper
class Test1( TestBase ):
def newHelper( self ):
return HelperObject1()
class Test2( TestBase ):
def newHelper( self ):
return HelperObject2()
-
3(1) is not really a problem - the obvious thing to do is to have your class come first and use super (My answer had this inverted - I just corrected it). (2) The very word you use says it - you have to use a "kludge" not to use a clean mechanism built into Python 's OOP working from the very beggining just because a 3rd party linting tool can't introspect properly. That is your choice of tools.– jsbuenoCommented Jul 7, 2015 at 15:09
Just to put in my two-cents, although it likely goes against some convention, you could define your abstract test case as a protected member to prevent its execution. I've implemented the following in Django and works as required. See example below.
from django.test import TestCase
class _AbstractTestCase(TestCase):
"""
Abstract test case - should not be instantiated by the test runner.
"""
def test_1(self):
raise NotImplementedError()
def test_2(self):
raise NotImplementedError()
class TestCase1(_AbstractTestCase):
"""
This test case will pass and fail.
"""
def test_1(self):
self.assertEqual(1 + 1, 2)
class TestCase2(_AbstractTestCase):
"""
This test case will pass successfully.
"""
def test_1(self):
self.assertEqual(2 + 2, 4)
def test_2(self):
self.assertEqual(12 * 12, 144)
-
4This seems to work great and it's much simpler than the monkey-patch approach above.– FasaxcCommented May 11, 2015 at 18:35
-
3It may work with django (and in my case, with nose), but the unittest discovery mechanism still seems to find these abstract test cases... Commented Mar 8, 2017 at 6:37
Raise unittest.SkipTest
in setUpClass()
Just another approach is to raise a unittest.SkipTest
in setUpClass()
of the base class and override setUpClass()
in child classes:
class BaseTestCase(TestCase):
@classmethod
def setUpClass(cls):
"Child classes must override this method and define cls.x and cls.y"
raise unittest.SkipTest
def test_x(self):
self.assertEqual(self.x * 3, self.x)
def test_y(self):
self.assertEqual(self.y * 3, self.y + self.y + self.y)
def test_z(self):
self.assertEqual(self.x + self.y, self.y)
class IntegerTestCase(BaseTestCase):
@classmethod
def setUpClass(cls):
cls.x = 0
cls.y = 2
class StringTestCase(BaseTestCase):
@classmethod
def setUpClass(cls):
cls.x = ''
cls.y = 'zuzuka'
If you need to use custom TestCase that defines its own setUpClass()
and you need to call super().setUpClass()
, you can define you own method to "set up data" and raise SkipTest only inside that method:
class BaseTestCase(ThidPartyTestCase):
@classmethod
def setUpClass(cls):
super().setUpClass() # if ThirdPartyTestCase has own setUpClass()
cls.setUpTestCaseData()
@classmethod
def setUpTestCaseData(cls):
"Override and set up cls.x and cls.y here"
raise unittest.SkipTest
... # tests
class IntegerTestCase(BaseTestCase):
@classmethod
def setUpTestCaseData(cls):
cls.x = 0
cls.y = 2
The unittest
module provides several options for skipping tests.
My preferred solution is to override the setUpClass
method in the "abstract" base class to raise a unittest.SkipTest
exception if needed:
class BaseTestCase(unittest.TestCase):
@classmethod
def setUpClass(cls):
if cls is BaseTestCase:
raise unittest.SkipTest("%s is an abstract base class" % cls.__name__)
else:
super(BaseTestCase, cls).setUpClass()
-
So it will be instantiated and if it is inheriting from abc.ABC with abstract methods it will throw error. Commented Jul 27, 2022 at 11:27
If you follow the convention of explicitly listing all test classes in run_unittest (see e.g. the Python test suite for many uses of that convention), then it will be straight-forward to not list a specific class.
If you want to continue using unittest.main, and if you can allow using unittest2 (e.g. from Python 2.7), you can use its load_tests protocol to specify which classes contain test cases). In earlier versions, you will have to subclass TestLoader, and override loadTestsFromModule.
-
Won't I lose then the ability to invoke those tests from the command line? I would like to by default run all tests (except the ones from the abstract test case), but in certain cases only some of them.– gruszczyCommented Dec 30, 2010 at 23:08
Python unittest library has load_tests protocol, which can be used to achieve exactly what do you want:
# Add this function to module with AbstractTestCase class
def load_tests(loader, tests, _):
result = []
for test_case in tests:
if type(test_case._tests[0]) is AbstractTestCase:
continue
result.append(test_case)
return loader.suiteClass(result)
Another reason for wanting to do what the OP is doing is to create a highly-parameterized base class which implements much of a set of core tests which need to be reproduced in several environments/scenarios. What I'm describing is essentially creating a parameterized fixture, a la pytest, using unittest.
Assuming you (like me) decide to run away as fast as you can from any multiple-inheritance-based solutions, one might have the following problem with using load_tests() to filter out your base class from the loaded suite:
In the standard TestLoader, load_tests is called after the auto-loading-from-class is done. Because: * this auto-loading-from-class will attempt to construct instances from your base class with the standard signature init(self, name), and * you may want this base class to have a very different ctor signature, or * you may want to skip construction-then-removal of your base class instances for some other reason
.. you may want to completely prevent this auto-loading of test instances from base classes.
EDIT: Vadim's solution in this other thread is a more elegant, concise, and independent way to do this. I have implemented the "nested class trick" and confirmed it works beautifully for the purpose of preventing TestLoader from "finding" your TestCase bases.
I originally had done this by modifying TestLoader.loadTestsFromModule to simply skip any TestCase classes which serve as base classes for any other TestCase classes in the module:
for name in dir(module):
obj = getattr(module, name)
# skip TestCase classes:
# 1. without any test methods defined
# 2. that are base classes
# (we don't allow instantiating TestCase base classes, which allows test designers
# to implement actual test methods in highly-parametrized base classes.)
if isinstance(obj, type) and issubclass(obj, unittest.TestCase) and \
self.getTestCaseNames(obj) and not isbase(obj, module):
loaded_suite = self.loadTestsFromTestCase(obj)
# ignore empty suites
if loaded_suite.countTestCases():
tests.append(loaded_suite)
where:
def isbase(cls, module):
'''Returns True if cls is base class to any classes in module, else False.'''
for name in dir(module):
obj = getattr(module, name)
if obj is not cls and isinstance(obj, type) and issubclass(obj, cls):
return True
return False
The parametrization I spoke of above is implemented by having each child class define it's fixture details (the parameters) and pass them to the base class TestCase ctor so that all of its common impl methods (the "fixturey" ones setUp*/tearDown*/cleanup* and the test methods themselves) have all the info that defines the now very specific fixture that that child TestCase class is to operate on.
For me, this was a temporary solution for quickly implementing some parametrized fixtures in unittest, since I plan to move my team's tests to pytest asap.
If you set __test__ = False
in the base test class it should disable its tests. Quoting from this link:
class MessageTestBase(unittest.TestCase): __test__ = False def setUp(self): self.status = 'running' def tearDown(self): self.status = 'not running' def test_common_test(self): self.assertEqual(self.status, 'running') class TestSlack(MessageTestMixin): __test__ = True
Notice the differences. Our mixin becomes a base class that inherits from TestCase. we include
__test__ = False
in the base class to prevent the test runner from executing tests in this class. Then the child class only inherits from MessageTestBase and includes__test__ = True
to instruct the test runner to run our tests.
More details here: How does __test__ = False magic attribute work for test discovery
Here's a relatively simple approach that allows your common tests to inherit from TestCase (so type checking and IDE tooling stays happy), that uses only documented unittest features, and that avoids the "skip" test status:
import unittest
class CommonTestCases(unittest.TestCase):
def __init__(self, methodName='runTest'):
if self.__class__ is CommonTestCases:
# don't run these tests on the abstract base implementation
methodName = 'runNoTestsInBaseClass'
super().__init__(methodName)
def runNoTestsInBaseClass(self):
print('not running tests in abstract base class')
pass
def test_common(self):
# This will run *only* in subclasses. Presumably, this would
# be a test you need to repeat in several different contexts.
self.assertEqual(2 + 2, 4)
class SomeTests(CommonTestCases):
# inherited test_common *will* be run here
def test_something(self):
self.assertTrue(True)
# Also plays nicely with MRO, if needed:
class SomeOtherTests(CommonTestCases, django.test.SimpleTestCase):
# inherited test_common *will* be run here
def test_something_else(self):
self.client.get('/') # ...
How it works: per the unittest.TestCase
documentation, "Each instance of TestCase will run a single base method: the method named methodName." The default "runTests" runs all the test* methods on the class—that's how TestCase instances normally work. But when running in the abstract base class itself, you can simply override that behavior with a method that does nothing.
A side effect is your test count will increase by one: the runNoTestsInBaseClass "test" gets counted as a successful test when it's run on CommonTestCases.
I know it's an old thread but it might help someone: I had a similar problem with my tests in a Django app that I had to upgrade from version 2.2 to version 4.2.
Previously, I’ve been using the django-nose library which has a nose based custom unit test runner, built on top of Django's own unit test runner. The feature asked by the OP was covered by the nose test runner. I think django-nose only run the tests of a test class if the name of the class starts as Test - hence I adopted a convention to name my abstract test classes as _Test.
Since django-nose (and nose) is in maintenance mode I had drop it but Django's built-in test runner (unittest2) collects and run all tests whose class is a subclass of unittest.TestCase.
So my solution was to subclass Django's DiscoverRunner the following way:
from unittest import TestSuite
from django.test.runner import DiscoverRunner
class CustomDiscoverRunner(DiscoverRunner):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.filtered_out = 0
def build_suite(self, test_labels=None, extra_tests=None, **kwargs) -> TestSuite:
print("Custom DiscoverRunner starts up...")
suite = super().build_suite(test_labels=test_labels, extra_tests=extra_tests, **kwargs)
# noinspection PyProtectedMember
# pylint: disable=protected-access
filtered_tests = [t for t in suite._tests if self._needed(t)]
print(f"Running {len(filtered_tests)} tests -- filtered out: {self.filtered_out}")
return TestSuite(tests=filtered_tests)
def _needed(self, test_name) -> bool:
"""
DiscoverRunner will collect all tests from all classes that are subclassing unittest.TestCase
This custom runner should only keep those tests whose class name starts with "Test{ExampleClass}"
This is to make sure that tests in abstract test classes will be filtered out
As a guideline, such classes should follow the "_Test{AbstractTestClass}" naming pattern
Tests with _Failed class name component are kept to make sure errors will be reported correctly:
if there is an error at test load time, test gets the class assigned as "unittest.loader._FailedTest"
This happens when using incorrect test labels: e.g.: "python manage.py test NoModuleLikeThis"
Keeping such failures in the suite will make sure, they will "run" and the error of incorrect label is reported
"""
if (
test_name.__class__.__name__.startswith("Test") or
test_name.__class__.__name__.startswith("_Failed")
):
return True
self.filtered_out += 1
return False
After this the custom runner can be setup to be the default runner by setting the corresponding settings variable:
TEST_RUNNER = 'myproject.unittest_runner.CustomDiscoverRunner'
I agree with the answer of @tsuna and the comment of @interDist in this similar question.
A further improvement can be using a decorator to explicitly mark test classes which are meant to be base or abstract and their test methods must only be executed for the derived classes:
from unittest import TestCase
def abstract_test(cls):
def run(self, result=None):
return result if self.__class__ is cls else TestCase.run(self, result)
cls.run = run
return cls
@abstract_test
class Base(TestCase):
"""This test shall only be executed in subclasses."""
def test_b(self):
print(f"{Base.__doc__} {self.__class__.__name__}.test_b from {Base.__name__}")
@abstract_test
class Base2(Base):
"""This too."""
def test_b2(self):
print(f"{Base2.__doc__} {self.__class__.__name__}.test_b2 from {Base2.__name__}")
class Derived(Base2):
def test_c(self):
print(f"{self.__class__.__name__}.test_c")
The decorator replaces the run
method inherited from TestCase
instead of overriding it in a base test class. Using the decorator avoids the code repetition and makes it more obvious which class is a base or abstract test.
However, executing the same inherited test in derived test cases may indicate that the units (under test) are not well designed:
- The code tested by the inherited test should be in a separate unit;
- If this code fails, it breaks several tests instead of a single one;
- It lasts longer or consumes more resources to test the same logic multiple times.
I have done it following way, maybe it can inspire you:
class AbstractTest(TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def _test_1(self):
# your test case here
class ConcreteTest(AbstractTest)
def test_1(self):
self._test_1()
Although it's not the most convenient solution, it lets you escape from multi inheritance. Also, the solution suggested by Dan Ward didn't work with Django testing in PyCharm.
__test__=False
in your base class.