62

I have always used:

r = requests.get(url)
if r.status_code == 200:
    # my passing code
else:
    # anything else, if this even exists

Now I was working on another issue and decided to allow for other errors and am instead now using:

try:
    r = requests.get(url)
    r.raise_for_status()
except requests.exceptions.ConnectionError as err:
    # eg, no internet
    raise SystemExit(err)
except requests.exceptions.HTTPError as err:
    # eg, url, server and other errors
    raise SystemExit(err)
# the rest of my code is going here

With the exception that various other errors could be tested for at this level, is one method any better than the other?

2
  • 2
    Second one may be more readable because of exception names instead of status codes, but I think first one is simpler and you can write comments about meaning of status codes. Apr 27, 2020 at 16:18
  • @EkremDİNÇEL I'm leaning that way because in this instance I'm using a system that doesn't always have an internet connection and needed to add the test in, and quite frankly I preferred the way this looked :)
    – Madivad
    Apr 27, 2020 at 16:20

4 Answers 4

66

Response.raise_for_status() is just a built-in method for checking status codes and does essentially the same thing as your first example.

There is no "better" here, just about personal preference with flow control. My preference is toward try/except blocks for catching errors in any call, as this informs the future programmer that these conditions are some sort of error. If/else doesn't necessarily indicate an error when scanning code.


Edit: Here's my quick-and-dirty pattern.

import time
from http import HTTPStatus

import requests
from requests.exceptions import HTTPError

url = "https://theurl.com"
retries = 3
retry_codes = [
    HTTPStatus.TOO_MANY_REQUESTS,
    HTTPStatus.INTERNAL_SERVER_ERROR,
    HTTPStatus.BAD_GATEWAY,
    HTTPStatus.SERVICE_UNAVAILABLE,
    HTTPStatus.GATEWAY_TIMEOUT,
]

for n in range(retries):
    try:
        response = requests.get(url)
        response.raise_for_status()

        break

    except HTTPError as exc:
        code = exc.response.status_code
        
        if code in retry_codes:
            # retry after n seconds
            time.sleep(n)
            continue

        raise
      

However, in most scenarios, I subclass requests.Session, make a custom HTTPAdapter that handles exponential backoffs, and the above lives in an overridden requests.Session.request method. An example of that can be seen here.

5
  • 3
    I kind of like the try/except blocks, I just find that they get messy when 2 or 3 (or more) become nested. It's bad enough for nested if blocks, but multiple try/except/except/else/finally blocks become quite unreadable. Having said that, I'm growing to like them :)
    – Madivad
    Apr 29, 2020 at 14:33
  • 2
    I agree. I go above-and-beyond to reduce nesting wherever possible. A mountain of my coding time goes into readability, "Code is read more often than it is written" and all. I've found that if nested or multiple (more than 2) try/except blocks, or really any deep nesting/large conditional blocks exist, this is almost always a trigger for refactoring.
    – Sam Morgan
    Apr 29, 2020 at 15:55
  • 4
    To clarify, OP's first example accepts only HTTP 200 as a successful condition while the second de facto accepts all HTTP codes < 400 (raise_for_status() throws an exception on codes 400-599), so it's very roughly "the same thing". Dec 29, 2021 at 8:48
  • 2
    (Requests library follows the HTTP 3xx redirects by default (except for HEAD), so 300-399 codes are processed the same way in both cases.) Dec 29, 2021 at 9:05
  • "does esentially the same thing as your first example" - strongly disagree. RaiseForStatus with 201 code for example, will do nothing, while his first example will prevent the code execution.
    – tikej
    Nov 3 at 14:40
30

Almost always, raise_for_status() is better.

The main reason is that there is a bit more to it than testing status_code == 200, and you should be making best use of tried-and-tested code rather than creating your own implementation.

For instance, did you know that there are actually five different 'success' codes defined by the HTTP standard? Four of those 'success' codes will be misinterpreted as failure by testing for status_code == 200.

2
  • 7
    Another perspective: One should know what they're developing against and not blindly accept any of the possible success codes. If the docs of an API say that HTTP 200 is the expected success code then that's what I'm checking. Any deviation from that is a breach of contract.
    – Tobias
    Mar 7 at 23:59
  • 2
    @Tobias Rejecting a response due to "breach of contract" may be the right thing to do, but other times it pays to be liberal in what you accept. As always, context is everything.
    – Ian Goldby
    Jun 30 at 9:41
16

If you are not sure, follow the Ian Goldby's answer.

...however please be aware that raise_for_status() is not some magical or exceptionally smart solution - it's a very simple function that decodes the response body and throws an exception for HTTP codes 400-599, distinguishing client-side and server-side errors (see its code here).

And especially the client-side error responses may contain valuable information in the response body that you may want to process. For example a HTTP 400 Bad Request response may contain the error reason.

In such a case it may be better to not use raise_for_status() but instead cover all the cases by yourself.

Example code

try:
    r = requests.get(url)

    # process the specific codes from the range 400-599
    # that you are interested in first
    if r.status_code == 400:
        invalid_request_reason = r.text
        print(f"Your request has failed because: {invalid_request_reason}")
        return
    # this will handle all other errors
    elif r.status_code > 400:
        print(f"Your request has failed with status code: {r.status_code}")
        return

except requests.exceptions.ConnectionError as err:
    # eg, no internet
    raise SystemExit(err)

# the rest of my code is going here

Real-world use case

PuppetDB's API using the Puppet Query Language (PQL) responds with a HTTP 400 Bad Request to a syntactically invalid query with a very precise info where is the error.

Request query:

nodes[certname] { certname == "bastion" }

Body of the HTTP 400 response:

PQL parse error at line 1, column 29:

nodes[certname] { certname == "bastion" }
                            ^

Expected one of:

[
false
true
#"[0-9]+"
-
'
"
#"\s+"

See my Pull Request to an app that uses this API to make it show this error message to a user here, but note that it doesn't exactly follow the example code above.

1
  • 3
    Yes, it's always a good idea to look inside the response to get more information about what went wrong and to make this available somewhere for troubleshooting. That same response is also available through the exception raised by raise_for_status() - ex.response.status_code, ex.response.text, etc.
    – Ian Goldby
    Mar 15, 2022 at 8:19
2

Better is somewhat subjective; both can get the job done. That said, as a relatively inexperienced programmer I prefer the Try / Except form. For me, the T / E reminds me that requests don't always give you what you expect (in a way that if / else doesn't - but that could just be me).

raise_for_status() also lets you easily implement as many or as few different actions for the different error types (.HTTPError, .ConnectionError) as you need. In my current project, I've settled on the form below, as I'm taking the same action regardless of cause, but am still interested to know the cause:

    try:
        ...
    except requests.exceptions.RequestException as e:
        raise SystemExit(e) from None

Toy implementation:

import requests

def http_bin_repsonse(status_code):
    sc = status_code
    try:
        url = "http://httpbin.org/status/" + str(sc)
        response = requests.post(url)
        response.raise_for_status()

        p = response.content

    except requests.exceptions.RequestException as e:
        print("placeholder for save file / clean-up")
        raise SystemExit(e) from None
    
    return response, p

response, p = http_bin_repsonse(403)
print(response.status_code)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.