104

I would like to add a retry mechanism to python request library, so scripts that are using it will retry for non fatal errors.

At this moment I do consider three kind of errors to be recoverable:

  • HTTP return codes 502, 503, 504
  • host not found (less important now)
  • request timeout

At the first stage I do want to retry specified 5xx requests every minute.

I want to be able to add this functionality transparently, without having to manually implement recovery for each HTTP call made from inside these scripts or libraries that are using python-requests.

5
  • 2
    Does this help? Apr 24, 2014 at 11:23
  • 3
    @thefourtheye: that only applies to transport-level errors; socket timeouts, SSL errors, and the likes. A server return code in the 500 range is not covered.
    – Martijn Pieters
    Apr 24, 2014 at 11:31
  • 2
    Does python-requests handle status code 429? tools.ietf.org/html/rfc6585 Sadly, most websites send inappropriate codes when rate limiting (like 503 and 404).
    – Nemo
    Mar 31, 2015 at 14:48
  • requests includes a copy of urllib3's Retry class (in requests.packages.util.retry.Retry), which will allow granular control, and includes a backoff mechanism for retry. For status-based retry, use parameter: status_forcelist which will force specific status code response to be retried according to the strategy chosen.
    – datashaman
    Feb 25, 2016 at 17:09
  • @datashaman I already implemented what would be called a ResilientSession which takes care of this in a transparent way. Look at my implementation from inside JIRA Python library.
    – sorin
    Feb 25, 2016 at 17:47

6 Answers 6

230

This snippet of code will make all HTTP requests from the same session retry for a total of 5 times, sleeping between retries with an increasing backoff of 0s, 2s, 4s, 8s, 16s (the first retry is done immediately). It will retry on basic connectivity issues (including DNS lookup failures), and HTTP status codes of 502, 503 and 504.

import logging
import requests

from requests.adapters import HTTPAdapter, Retry

logging.basicConfig(level=logging.DEBUG)

s = requests.Session()
retries = Retry(total=5, backoff_factor=1, status_forcelist=[ 502, 503, 504 ])
s.mount('http://', HTTPAdapter(max_retries=retries))

s.get("http://httpstat.us/503")

See Retry class for details.

9
  • 5
    First backoff is 0s, not 1s according to documentation. So it goes like 0s, 2s, 4s, 8s, 16s
    – epokhe
    Feb 23, 2019 at 9:54
  • 7
    What is the significance of the first argument to session.mount, http:// ? What purpose does it serve? Feb 27, 2019 at 18:56
  • 13
    @JamesWierzba it's a pattern to attach to the adapter. So in this case all URLs that start with http:// will use that adapter. You could have a specific adapter for https://cnn.com for example. All calls using that session will go via the custom HTTP adapter when accessing https://cnn.com and the default for everything else.
    – datashaman
    Feb 28, 2019 at 4:39
  • 6
    Look for the method_whitelist option in Retry class, by default it doesn't retry on http methods except for ['HEAD', 'TRACE', 'GET', 'PUT', 'OPTIONS', 'DELETE'] - these methods are expected to not have side effects Nov 18, 2019 at 12:05
  • 5
    Set method_whitelist=False to retry for POST API calls as well. E.g. If you are communicating with GraphQL endpoint all calls are POST. By default only ['HEAD', 'TRACE', 'GET', 'PUT', 'OPTIONS', 'DELETE'] are retried as they are deemed safe / idempotent operations.
    – amirathi
    Jun 22, 2020 at 10:04
25

This is a snippet of code I used to retry for the petitions made with urllib2. Maybe you could use it for your purposes:

retries = 1
success = False
while not success:
    try:
        response = urllib2.urlopen(request)
        success = True
    except Exception as e:
        wait = retries * 30;
        print 'Error! Waiting %s secs and re-trying...' % wait
        sys.stdout.flush()
        time.sleep(wait)
        retries += 1

The waiting time grows incrementally to avoid be banned from server.

2
  • 3
    I would rather explicitly use 'continue' in a loop and break by default. Just to avoid infinite loop by mistake.
    – ed22
    Jun 28, 2020 at 10:20
  • 4
    to limit the number of retries, I could use while not success and retries < 5: May 8, 2021 at 10:06
11

Possible solution using retrying package

from retrying import retry
import requests


def retry_if_connection_error(exception):
    """ Specify an exception you need. or just True"""
    #return True
    return isinstance(exception, ConnectionError)

# if exception retry with 2 second wait  
@retry(retry_on_exception=retry_if_connection_error, wait_fixed=2000)
def safe_request(url, **kwargs):
    return requests.get(url, **kwargs)

response = safe_request('test.com')
3
  • the retrying package above seems unmaintained. I've had success with the more updated fork: github.com/jd/tenacity
    – chaim
    Feb 20 at 13:12
  • @chaim will the code provided in the answer still work with a fork?
    – Danil
    Feb 20 at 15:30
  • @Danill Mashkin, not exactly the same code, the updated fork has changed few things, but quite similar.
    – chaim
    Feb 22 at 0:56
9
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry


MAX_RETRY = 2
MAX_RETRY_FOR_SESSION = 2
BACK_OFF_FACTOR = 0.3
TIME_BETWEEN_RETRIES = 1000
ERROR_CODES = (500, 502, 504)


def requests_retry_session(retries=MAX_RETRY_FOR_SESSION,
    back_off_factor=BACK_OFF_FACTOR,
    status_force_list=ERROR_CODES, 
    session=None):
       session = session  
       retry = Retry(total=retries, read=retries, connect=retries,
                     backoff_factor=back_off_factor,
                     status_forcelist=status_force_list,
                     method_whitelist=frozenset(['GET', 'POST']))
       adapter = HTTPAdapter(max_retries=retry)
       session.mount('http://', adapter)
       session.mount('https://', adapter)
       return session



class ConfigService:

   def __init__(self):
      self.session = requests_retry_session(session=requests.Session())

   def call_to_api():
      config_url = 'http://localhost:8080/predict/'
      headers = {
        "Content-Type": "application/json",
        "x-api-key": self.x_api_key
      } 
      response = self.session.get(config_url, headers=headers)
      return response
1
  • Does the self.session.get in call_to_api() need to be wrapped in a try block? Jul 21, 2020 at 9:05
-1

I was able to obtain the desired level of reliability by extending requests.Session class.

Here is the code https://bitbucket.org/bspeakmon/jira-python/src/a7fca855394402f58507ca4056de87ccdbd6a213/jira/resilientsession.py?at=master

EDIT That code was:

from requests import Session
from requests.exceptions import ConnectionError
import logging
import time


class ResilientSession(Session):

    """
    This class is supposed to retry requests that do return temporary errors.

    At this moment it supports: 502, 503, 504
    """

    def __recoverable(self, error, url, request, counter=1):
        if hasattr(error,'status_code'):
            if error.status_code in [502, 503, 504]:
                error = "HTTP %s" % error.status_code
            else:
                return False
        DELAY = 10 * counter
        logging.warn("Got recoverable error [%s] from %s %s, retry #%s in %ss" % (error, request, url, counter, DELAY))
        time.sleep(DELAY)
        return True


    def get(self, url, **kwargs):
        counter = 0
        while True:
            counter += 1
            try:
                r = super(ResilientSession, self).get(url, **kwargs)
            except ConnectionError as e:
                r = e.message
            if self.__recoverable(r, url, 'GET', counter):
                continue
            return r

    def post(self, url, **kwargs):
        counter = 0
        while True:
            counter += 1
            try:
                r = super(ResilientSession, self).post(url, **kwargs)
            except ConnectionError as e:
                r = e.message
            if self.__recoverable(r, url, 'POST', counter):
                continue
            return r

    def delete(self, url, **kwargs):
        counter = 0
        while True:
            counter += 1
            try:
                r = super(ResilientSession, self).delete(url, **kwargs)
            except ConnectionError as e:
                r = e.message
            if self.__recoverable(r, url, 'DELETE', counter):
                continue
            return r

    def put(self, url, **kwargs):
        counter = 0
        while True:
            counter += 1
            try:
                r = super(ResilientSession, self).put(url, **kwargs)
            except ConnectionError as e:
                r = e.message

            if self.__recoverable(r, url, 'PUT', counter):
                continue
            return r

    def head(self, url, **kwargs):
        counter = 0
        while True:
            counter += 1
            try:
                r = super(ResilientSession, self).head(url, **kwargs)
            except ConnectionError as e:
                r = e.message
            if self.__recoverable(r, url, 'HEAD', counter):
                continue
            return r

    def patch(self, url, **kwargs):
        counter = 0
        while True:
            counter += 1
            try:
                r = super(ResilientSession, self).patch(url, **kwargs)
            except ConnectionError as e:
                r = e.message

            if self.__recoverable(r, url, 'PATCH', counter):
                continue
            return r

    def options(self, url, **kwargs):
        counter = 0
        while True:
            counter += 1
            try:
                r = super(ResilientSession, self).options(url, **kwargs)
            except ConnectionError as e:
                r = e.message

            if self.__recoverable(r, url, 'OPTIONS', counter):
                continue
            return r
4
  • You should copy-paste that code here. You shouldn't answer a question with something that is offsite - link rot. Also, TBH, your code could be a lot shorter. I think restructure it to be a decorator and you'll have a much shorter, less duplicated solution.
    – datashaman
    Feb 26, 2016 at 4:54
  • 2
    Here's a basic rewrite of your resilient session to use decorators. Still has some duplication (the methods all have the same pattern), but it works exactly the same and you only have to change the logic in one place: gist.github.com/datashaman/fc02882d6be49d0b882f
    – datashaman
    Feb 26, 2016 at 5:20
  • A more succinct version that does the retry on the request method, removing all duplication from the code: gist.github.com/datashaman/a517da0ebfe7939c6b83
    – datashaman
    Feb 26, 2016 at 5:52
  • 3
    I recognize that this code is 5 years old, but it's quality is very low. The answer from @datashaman should be accepted so that this answer can receive less attention. Sep 3, 2019 at 17:08
-1

Method to retry certain logic if some exception has occured at time intervals t1=1 sec, t2=2 sec, t3=4 sec. We can increase/decrease the time interval as well.

MAX_RETRY = 3
retries = 0

try:

    call_to_api() // some business logic goes here.

except Exception as exception:

    retries += 1
    if retries <= MAX_RETRY:
        print("ERROR=Method failed. Retrying ... #%s", retries)
        time.sleep((1 << retries) * 1) // retry happens after time as a exponent of 2
        continue
    else:
        raise Exception(exception)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.