53

You may know this recommendation from Microsoft about the use of exceptions in .NET:

Performance Considerations

...

Throw exceptions only for extraordinary conditions, ...

In addition, do not throw an exception when a return code is sufficient...

(See the whole text at http://msdn.microsoft.com/en-us/library/system.exception.aspx.)

As a point of comparison, would you recommend the same for Python code?

4
  • 9
    -1: Why are you trying to apply .Net framework library documentation to Python? What made you think this .Net advice applied to Python? Was there some note or hint that this was relevant to Python?
    – S.Lott
    Commented Jul 20, 2009 at 9:54
  • 46
    I think in fairness he is asking whether "do not throw an exception when a return code is sufficient" applies more generally outside the .Net framework too. It's still a reasonable question even if the answer was "no, it's a .Net thing"
    – Draemon
    Commented Jul 20, 2009 at 10:17
  • 12
    This is a very general recommendation which applies for .NET and c++. I just wanted if such a recommendation is also valid for python. I know that Python and .NET are different. but both of them have exception handling. It is a point of comparison
    – luc
    Commented Jul 20, 2009 at 10:18
  • 1
    @luc: It may be a point of comparison, but the .NET quote doesn't apply to Python at all. Perhaps the question should be fixed to clarify this.
    – S.Lott
    Commented Jul 20, 2009 at 14:51

6 Answers 6

42

The pythonic thing to do is to raise and handle exceptions. The excellent book "Python in a nutshell" discusses this in 'Error-Checking Strategies' in Chapter 6.

The book discusses EAFP ("it's easier to ask forgiveness than permission") vs. LBYL ("look before you leap").

So to answer your question:

No, I would not recommend the same for python code. I suggest you read chapter 6 of Python in a nutshell.

3
  • 3
    "in most cases"? Almost nothing returns status codes. If there are any status code returns, it is because of an underlying C-language thing that wasn't properly wrapped in an exception. I'd vote for dropping "in most cases."
    – S.Lott
    Commented Jul 20, 2009 at 9:52
  • 9
    This answer doesn't explain anything. Can it be expanded to exclude requirement to buy the book? Commented Jun 23, 2013 at 9:23
  • Citation is great but it would be better to present a few facts in the answer itself. Otherwise it's much less useful.
    – Nolan
    Commented Oct 11, 2023 at 10:52
16

The best way to understand exceptions is "if your method can't do what its name says it does, throw." My personal opinion is that this advice should be applied equally to both .NET and Python.

The key difference is where you have methods that frequently can't do what their name says they should do, for instance, parsing strings as integers or retrieving a record from a database. The C# style is to avoid an exception being thrown in the first place:

int i;
if (Int32.TryParse(myString, out i)) {
    doWhatever(i);
}
else {
    doWhatever(0);
}

whereas Python is much more at ease with this kind of thing:

try:
    i = int(myString)
except ValueError:
    i = 0
doWhatever(i);
10

Usually, Python is geared towards expressiveness.
I would apply the same principle here: usually, you expect a function to return a result (in line with its name!) and not an error code.
For this reason, it is usually better raising an exception than returning an error code.

However, what is stated in the MSDN article applies to Python as well, and it's not really connected to returning an error code instead of an exception.
In many cases, you can see exception handling used for normal flow control, and for handling expected situations. In certain environments, this has a huge impact on performance; in all environments it has a big impact on program expressiveness and maintainability.

Exceptions are for exceptional situations, that are outside of normal program flow; if you expect something will happen, then you should handle directly, and then raise anything that you cannot expect / handle.

Of course, this is not a recipe, but only an heuristic; the final decision is always up to the developer and onto the context and cannot be stated in a fixed set of guidelines - and this is much truer for exception handling.

9

In Python exceptions are not very expensive like they are in some other languages, so I wouldn't recommend trying to avoid exceptions. But if you do throw an exception you would usually want catch it somewhere in your code, the exception being if a fatal error occurs.

2
  • 3
    Unless it's fatal. //Best regards, the nitpicking comittee. Commented Jul 20, 2009 at 10:05
  • 1
    @Lennart Regebro, True indeed, updated my answer to please the nitpicking committee.
    – googletorp
    Commented Jul 20, 2009 at 10:21
5

I think whether to return an error code or throw an exception is something very valid to think about, and a cross-linguistic comparison may be helpful and informative. I guess the very generalized answer to this concern is simply the consideration: that the set of legal return values for any function should be made as small as possible, and as large as necessary.

Generally, this will mean that if a given method returns an integer number in a single test case, users can rightfully expect the method to always return an integer number or throw an exception. But, of course, the conceptually simplest way is not always the best way to handle things.

The return-value-of-least-surprise is usually None; and if you look into it, you’ll see that it’s the very semantics of None that license its usage across the board: it is a singleton, immutable value that, in a lot of cases, evaluates to False or prohibits further computation—no concatenation, no arithmetics. So if you chose to write a frob(x) method that returns a number for a string input, and None for non-numeric strings and any other input, and you use that inside an expression like a=42+frob('foo'), you still get an exception very close to the point where bogus things happened. Of course, if you stuff frob('foo') into a database column that has not been defined with NOT NULL, you might run into problems perhaps months later. This may or may not be justifiable.

So in most cases where you e.g. want to derive a number from a string, using somwething like a bare float(x) or int(x) is the way to go, as these built-ins will raise an exception when not given a digestable input. If that doesn’t suit your use case, consider returning None from a custom method; basically, this return value tells consumers that ‘Sorry, I was unable to understand your input.’. But you only want to do this if you positively know that going on in your program does make sense from that point onwards.

You see, I just found out how to turn each notice, warning, and error message into a potentially show-stopping exception in, uhm, PHP. It just drives me crazy that a typo in a variable name generates in the standard PHP configuration, nothing but a notice to the user. This is so bad. The program just goes on doing things with a program code that does not make sense at all! I can’t believe people find this a feature.

Likewise, one should view it like this: if, at any given point in time, it can be asserted with reasonable costs that the execution of a piece of code does no more make sense — since values are missing, are out of bounds, or are of an unexpected type, or when resources like a database connection have gone down — it is imperative, to minimize debugging headaches, to break execution and hand control up to any level in the code which feels entitled to handle the mishap.

Experience shows that refraining from early action and allowing bogus values to creep into your data is good for nothing but making your code harder to debug. So are many examples of over-zealous type-casting: allowing integers to be added to floats is reasonable. To allow a string with nothing but digits to be added to a number is a bogus practice that is likely to create strange, unlocalized errors that may pop up on any given line that happens to be processing that data.

2

I did a simple experiment to compare the performance of raising exceptions with the following code:

from functools import wraps
from time import time
import logging

def timed(foo):
    @wraps(foo)
    def bar(*a, **kw):
        s = time()
        foo(*a, **kw)
        e = time()
        print '%f sec' % (e - s)
    return bar

class SomeException(Exception):
    pass

def somefunc(_raise=False):
    if _raise:
        raise SomeException()
    else:
        return

@timed
def test1(_reps):
    for i in xrange(_reps):
        try:
            somefunc(True)
        except SomeException:
            pass

@timed
def test2(_reps):
    for i in xrange(_reps):
        somefunc(False)

def main():

    test1(1000000)
    test2(1000000)

    pass

if __name__ == '__main__':
    main()

With the following results:

  • Raising exceptions: 3.142000 sec
  • Using return: 0.383000 sec

Exceptions are about 8 times slower than using return.

6
  • 1
    I ran your test and found about a 4.4x factor, which is not different enough from your results to change any conclusions. But I would state this more precisely as raising and catching an exception is about 2.8 microseconds slower than returning a value and checking it, using your results. On a more modern system with faster CPU that I ran the test on, the difference was 0.34 microseconds. Point being, as you provide evidence for, there is a cost to raising and catching exceptions for large iteration counts, which is one aspect of deciding an exception strategy.
    – mklein9
    Commented Mar 15, 2016 at 21:42
  • Sorry for the second comment, but I ran out of time to edit the previous one. My results above are for Python 3.4; 2.7 shows a loop time difference for the exception path of 0.48 microseconds, more than 40% higher than 3.4's, and a ratio of 5.7x.
    – mklein9
    Commented Mar 15, 2016 at 21:59
  • 2
    And how much slower, not to mention less readable, is the client code because it has to actually test the result before using it?
    – holdenweb
    Commented Aug 2, 2016 at 13:05
  • 1
    One consideration is that one generally does not throw exceptions in a loop. This answer says that throwing is more expensive than returning, but I believe it misses the point that you rarely throw, and you do it when there is a problem. The successful path is not really affected by that. If you find yourself in a situation where you throw in a loop, then that's likely a design issue, indeed. Commented Jan 7, 2019 at 10:16
  • this is a faulty test — the exception path always constructs a new object, but the return path does not. if i return SomeException() instead, the difference in speed is cut in half on my machine.
    – Eevee
    Commented Mar 13, 2021 at 21:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.