503

I want to write a test to establish that an Exception is not raised in a given circumstance.

It's straightforward to test if an Exception is raised ...

sInvalidPath=AlwaysSuppliesAnInvalidPath()
self.assertRaises(PathIsNotAValidOne, MyObject, sInvalidPath) 

... but how can you do the opposite.

Something like this i what I'm after ...

sValidPath=AlwaysSuppliesAValidPath()
self.assertNotRaises(PathIsNotAValidOne, MyObject, sValidPath) 
5
  • 9
    You could always simply do whatever ought to work in the test. If it raises an error, that will show up (counted as an error, rather than a failure). Of course, that assumes that it doesn't raise any error, rather than just a defined type of error. Other than that, I guess you'd have to write your own.
    – Thomas K
    Nov 30, 2010 at 23:40
  • 2
    possible duplicate of Python - test that succeeds when exception is not raised
    – brandizzi
    Jul 14, 2015 at 1:22
  • It turns out that you can in fact implement an assertNotRaises method that shares 90% of its code/behavior with assertRaises in about ~30-ish lines of code. See my answer below for details.
    – tel
    Mar 2, 2018 at 4:47
  • I want this so I can compare two functions with hypothesis to make sure they produce the same output for all kinds of input, while ignoring the cases where the original raises an exception. assume(func(a)) doesn't work because output can be an array with ambiguous truth value. So I just want to call a function and get True if it doesn't fail. assume(func(a) is not None) works I guess
    – endolith
    Apr 19, 2019 at 20:25
  • Does this answer your question? Pass a Python unittest if an exception isn't raised
    – Flimm
    Jul 6, 2020 at 9:39

11 Answers 11

557
def run_test(self):
    try:
        myFunc()
    except ExceptionType:
        self.fail("myFunc() raised ExceptionType unexpectedly!")
8
  • 50
    @hiwaylon - No, this is the correct solution in fact. The solution proposed by user9876 is conceptually flawed: if you test for the non-raising of say ValueError, but ValueError is instead raised, your test must exit with a failure condition, not an error one. On the other hand, if in running the same code you would raise a KeyError, that would be an error, not a failure. In python - differently than some other languages - Exceptions are routinely used for control flow, this is why we have the except <ExceptionName> syntax indeed. To that regard, user9876's solution is simply wrong.
    – mac
    Apr 18, 2013 at 11:52
  • 1
    This one has the unfortunate effect of showing < 100% coverage (the except will never happen) for tests.
    – Shay
    Nov 27, 2018 at 20:10
  • 11
    @Shay, IMO you should always exclude the test files themselves from the coverage reports (as they almost always run 100% by definition, you would be artificially inflating the reports) Oct 16, 2019 at 10:55
  • 1
    @original-bbq-sauce, wouldn't that leave me open to unintentionally skipped tests. E.g., typo in test name (ttst_function), wrong run configuration in pycharm, etc.?
    – Shay
    Oct 19, 2019 at 15:14
  • 4
    requiring 100% coverage on test files catches has caught more than a few mistakes for me that otherwise would likely not have been found. Feb 4, 2021 at 16:10
86

Hi - I want to write a test to establish that an Exception is not raised in a given circumstance.

That's the default assumption -- exceptions are not raised.

If you say nothing else, that's assumed in every single test.

You don't have to actually write an any assertion for that.

4
  • 11
    @IndradhanushGupta Well the accepted answer makes the test more pythonic than this one. Explicit is better than implicit.
    – 0xc0de
    Oct 29, 2015 at 7:14
  • 39
    No other commenter has pointed out why this answer is wrong, though it's the same reason that user9876's answer is wrong: failures and errors are different things in test code. If your function were to throw an exception during a test that doesn't assert, the test framework would treat that as an error, rather than a failure to not assert. Mar 18, 2016 at 22:02
  • 2
    @CoreDumpError I understand the difference between a failure and an error, but wouldn't this force you to surround every test with a try/exception construction? Or would you recommend to do that only for tests that explicitly raise an exception under some condition (which basically means that the exception is expected). Jan 9, 2018 at 13:23
  • 6
    @federicojasson You answered your own question quite well in that second sentence. Errors vs failures in tests can be succinctly described as "unexpected crashes" vs "unintended behavior", respectively. You want your tests to show an error when your function crashes, but not when an exception that you know it will throw given certain inputs gets thrown when given different inputs. Jan 9, 2018 at 18:41
58

Just call the function. If it raises an exception, the unit test framework will flag this as an error. You might like to add a comment, e.g.:

sValidPath=AlwaysSuppliesAValidPath()
# Check PathIsNotAValidOne not thrown
MyObject(sValidPath)

Edited to add clarifications from the comments:

  • Unit tests can have 3 results: Pass, Fail, Error. (Actually more if you count XPass/XFail/Skip...)
  • If you're testing a particular exception is not thrown, and it is thrown, then in theory that should be a Fail. But the code above makes it an Error, which is theoretically "wrong".
  • As a practical matter, with an Error your test runner will probably print the stack trace, which may be useful debugging the failure. With a Fail you probably won't see the stack trace.
  • As a practical matter, with a Fail you can mark the test as "Expected to Fail". With an Error you probably can't do that, although you can mark the test as "skip".
  • As a practical matter, making the test case report an Error requires additional code.
  • Whether the difference between "Error" and "Failure" matters depends on your processes. The way my team uses unit tests, they have to all pass. (Agile programming, with a continuous integration machine that runs all the unit tests). What actually matters to my team is "do all the unit tests pass?" (i.e. "is Jenkins green?"). So for my team, there's no practical difference between "Fail" and "Error".
  • Due to the advantages mentioned above (less code, seeing the stack trace), and the fact that Fail/Error are treated the same by my team, I use this approach.
  • You may have different requirements if you use your unit tests in a different way, especially if your processes treat "fail" and "error" differently, or if you want to be able to mark tests as "expected failure".
  • If you would rather have this test report an Error, use DGH's answer.
9
  • 43
    Failures and Errors are conceptually different. Furthermore, since in python Exceptions are routinely used for control flow, this will make very very difficult to understand at a glance (=without exploring the test code) if you have broken your logic or your code...
    – mac
    Nov 29, 2012 at 15:52
  • 3
    Either your test passes or it doesn't. If it doesn't pass, you're going to have to go fix it. Whether it's reported as a "failure" or an "error" is mostly irrelevant. There is one difference: with my answer you'll see the stack trace so you can see where PathIsNotAValidOne was thrown; with the accepted answer you won't have that information so debugging will be harder. (Assuming Py2; not sure if Py3 is better at this).
    – user9876
    Jul 30, 2014 at 11:08
  • 21
    @user9876 - Nope. The test exit conditions are 3 (pass/nopass/error), not 2 as you seem to erroneously believe. The difference between errors and failures is substantial and treating them as if they were the same is just poor programming. If you don't believe me, just look around to how test runners work and what decision trees they implement fro failures and errors. A good starting point for python is the xfail decorator in pytest.
    – mac
    Oct 21, 2014 at 12:29
  • 6
    I guess it depends on how you use unit tests. The way my team uses unit tests, they have to all pass. (Agile programming, with a continuous integration machine that runs all the unit tests). I know that the test case can report "pass", "fail" or "error". But at a high level what actually matters to my team is "do all the unit tests pass?" (i.e. "is Jenkins green?"). So for my team, there's no practical difference between "fail" and "error". You may have different requirements if you use your unit tests in a different way.
    – user9876
    Apr 1, 2016 at 0:18
  • 1
    @user9876 the difference between 'fail' and 'error' is the difference between "my assert failed" and "my test doesn't even get to the assert". That, to me, is a useful distinction during fixing tests, but I guess, as you say, not for everyone.
    – C S
    Sep 11, 2018 at 16:09
21

You can define assertNotRaises by reusing about 90% of the original implementation of assertRaises in the unittest module. With this approach, you end up with an assertNotRaises method that, aside from its reversed failure condition, behaves identically to assertRaises.

TLDR and live demo

It turns out to be surprisingly easy to add an assertNotRaises method to unittest.TestCase (it took me about 4 times as long to write this answer as it did the code). Here's a live demo of the assertNotRaises method in action. Just like assertRaises, you can either pass a callable and args to assertNotRaises, or you can use it in a with statement. The live demo includes a test cases that demonstrates that assertNotRaises works as intended.

Details

The implementation of assertRaises in unittest is fairly complicated, but with a little bit of clever subclassing you can override and reverse its failure condition.

assertRaises is a short method that basically just creates an instance of the unittest.case._AssertRaisesContext class and returns it (see its definition in the unittest.case module). You can define your own _AssertNotRaisesContext class by subclassing _AssertRaisesContext and overriding its __exit__ method:

import traceback
from unittest.case import _AssertRaisesContext

class _AssertNotRaisesContext(_AssertRaisesContext):
    def __exit__(self, exc_type, exc_value, tb):
        if exc_type is not None:
            self.exception = exc_value.with_traceback(None)

            try:
                exc_name = self.expected.__name__
            except AttributeError:
                exc_name = str(self.expected)

            if self.obj_name:
                self._raiseFailure("{} raised by {}".format(exc_name,
                    self.obj_name))
            else:
                self._raiseFailure("{} raised".format(exc_name))

        else:
            traceback.clear_frames(tb)

        return True

Normally you define test case classes by having them inherit from TestCase. If you instead inherit from a subclass MyTestCase:

class MyTestCase(unittest.TestCase):
    def assertNotRaises(self, expected_exception, *args, **kwargs):
        context = _AssertNotRaisesContext(expected_exception, self)
        try:
            return context.handle('assertNotRaises', args, kwargs)
        finally:
            context = None

all of your test cases will now have the assertNotRaises method available to them.

1
  • 7
    IMNSHO this should be added to the unittest library..
    – qneill
    Jul 30, 2020 at 19:02
18

I am the original poster and I accepted the above answer by DGH without having first used it in the code.

Once I did use I realised that it needed a little tweaking to actually do what I needed it to do (to be fair to DGH he/she did say "or something similar" !).

I thought it was worth posting the tweak here for the benefit of others:

    try:
        a = Application("abcdef", "")
    except pySourceAidExceptions.PathIsNotAValidOne:
        pass
    except:
        self.assertTrue(False)

What I was attempting to do here was to ensure that if an attempt was made to instantiate an Application object with a second argument of spaces the pySourceAidExceptions.PathIsNotAValidOne would be raised.

I believe that using the above code (based heavily on DGH's answer) will do that.

2
  • 2
    Since you are clarifying your question and not answering it you should have edited it (not answered it).Please see my answer below.
    – hiwaylon
    Aug 25, 2011 at 20:33
  • 18
    It seems to be quite the opposite to the original problem. self.assertRaises(PathIsNotAValidOne, MyObject, sInvalidPath) should do the job in this case. Dec 2, 2013 at 13:54
3

I've found it useful to monkey-patch unittest as follows:

def assertMayRaise(self, exception, expr):
  if exception is None:
    try:
      expr()
    except:
      info = sys.exc_info()
      self.fail('%s raised' % repr(info[0]))
  else:
    self.assertRaises(exception, expr)

unittest.TestCase.assertMayRaise = assertMayRaise

This clarifies intent when testing for the absence of an exception:

self.assertMayRaise(None, does_not_raise)

This also simplifies testing in a loop, which I often find myself doing:

# ValueError is raised only for op(x,x), op(y,y) and op(z,z).
for i,(a,b) in enumerate(itertools.product([x,y,z], [x,y,z])):
  self.assertMayRaise(None if i%4 else ValueError, lambda: op(a, b))
2
3
def _assertNotRaises(self, exception, obj, attr):                                                                                                                              
     try:                                                                                                                                                                       
         result = getattr(obj, attr)                                                                                                                                            
         if hasattr(result, '__call__'):                                                                                                                                        
             result()                                                                                                                                                           
     except Exception as e:                                                                                                                                                     
         if isinstance(e, exception):                                                                                                                                           
            raise AssertionError('{}.{} raises {}.'.format(obj, attr, exception)) 

could be modified if you need to accept parameters.

call like

self._assertNotRaises(IndexError, array, 'sort')
1

If you pass an Exception class to assertRaises(), a context manager is provided. This can improve the readability of your tests:

# raise exception if Application created with bad data
with self.assertRaises(pySourceAidExceptions.PathIsNotAValidOne):
    application = Application("abcdef", "")

This allows you to test error cases in your code.

In this case, you are testing the PathIsNotAValidOne is raised when you pass invalid parameters to the Application constructor.

3
  • 1
    No, this will only fail if the exception isn't raised within the context manager block. Can be readily tested by 'with self.assertRaises(TypeError): raise TypeError', which passes. Feb 22, 2012 at 5:33
  • @MatthewTrevor Good call. As I recall, rather than testing code executes correctly, i.e. doesn't raise, I was suggesting testing error cases. I've edited answer accordingly. Hopefully I can earn a +1 to get out of the red. :)
    – hiwaylon
    Feb 25, 2012 at 15:17
  • Note, this also is Python 2.7 and later: docs.python.org/2/library/…
    – qneill
    Jul 29, 2015 at 21:07
1

you can try like that. try: self.assertRaises(None,function,arg1, arg2) except: pass if you don't put code inside try block it will through exception' AssertionError: None not raised " and test case will be failed. Test case will be pass if put inside try block which is expected behaviour.

0

My solution - derived from accepted solution: Cache error on function. If there is no error, Assertion error will raise. In that case -> test Passed! Else, write error message.

Code:

    try:
        with self.assertRaises((PathIsNotAValidOne, AnotherError)) as err:
            AlwaysSuppliesAnInvalidPath()
        error = str(err.exception)
    except AssertionError:
        self.assertTrue(True)
    else:
        self.fail(error)

And:

    try:
        with self.assertRaises((PathIsNotAValidOne, AnotherError)) as err:
            AlwaysSuppliesAValidPath()
        error = str(err.exception)
    except AssertionError:
        self.assertTrue(True)
    else:
        self.fail(error)

Little twisted logic, but works as a charm!

-1

One straight forward way to ensure the object is initialized without any error is to test the object's type instance.

Here is an example :

p = SomeClass(param1=_param1_value)
self.assertTrue(isinstance(p, SomeClass))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.