22

I'm running a Python script that uses the requests package for making web requests. However, the web requests go through a proxy with a self-signed cert. As such, requests raise the following Exception:

requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",)

I know that SSL validation can be disabled in my own code by passing verify=False, e.g.: requests.get("https://www.google.com", verify=False). I also know that if I had the certificate bundle, I could set the REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE environment variables to point to those files. However, I do not have the certificate bundle available.

How can I disable SSL validation for external modules without editing their code?

3 Answers 3

54

Note: This solution is a complete hack.

Short answer: Set the CURL_CA_BUNDLE environment variable to an empty string.

Before:

$ python
import requests
requests.get('http://www.google.com')
<Response [200]>

requests.get('https://www.google.com')
...
File "/usr/local/lib/python2.7/site-packages/requests-2.17.3-py2.7.egg/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",)

After:

$ CURL_CA_BUNDLE="" python
import requests
requests.get('http://www.google.com')
<Response [200]>

requests.get('https://www.google.com')
/usr/local/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py:852: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning)
<Response [200]>

How it works

This solution works because Python requests overwrites the default value for verify from the environment variables CURL_CA_BUNDLE and REQUESTS_CA_BUNDLE, as can be seen here:

if verify is True or verify is None:
verify = (os.environ.get('REQUESTS_CA_BUNDLE') or
    os.environ.get('CURL_CA_BUNDLE'))

The environment variables are meant to specify the path to the certificate file or CA_BUNDLE and are copied into verify. However, by setting CURL_CA_BUNDLE to an empty string, the empty string is copied into verify and in Python, an empty string evaluates to False.

Note that this hack only works with the CURL_CA_BUNDLE environment variable - it does not work with the REQUESTS_CA_BUNDLE. This is because verify is set with the following statement:

verify = (os.environ.get('REQUESTS_CA_BUNDLE') or os.environ.get('CURL_CA_BUNDLE'))

It only works with CURL_CA_BUNDLE because '' or None is not the same as None or '', as can be seen below:

print repr(None or "")
# Prints: ''
print repr("" or None )
# Prints: None
9
  • 4
    I've successfully used this technique on Unix, but cannot get it to work on Windows Apr 15, 2019 at 9:24
  • 1
    Also with set CURL_CA_BUNDLE="" not working on Windows for me Feb 17, 2020 at 10:50
  • 3
    This is amazing in its simplicity and hackiness :) Confirming this works as described on macOS 10.14 with Python 3.7/3.8.
    – BrianC
    Aug 6, 2020 at 22:11
  • 2
    @David: That's how variables work in Bash and similar shells. If you just do CURL_CA_BUNDLE="", that only sets the variable within the context of that specific shell or for the command that follows on the same line. If you use export CURL_CA_BUNDLE="", then it will also apply to any new processes started from that shell, like Python (but still not ones in completely separate processes). Oct 13, 2020 at 14:23
  • 6
    This no longer works with requests 2.28.0 or newer, since it was considered a bug, see github.com/psf/requests/issues/6071... Jun 9, 2022 at 17:36
4

I saw this hack only due to some trouble with my private CA.

The given hack with CURL_CA_BUNDLE='' will not longer work in 2022 with next minor release of requests (that means 2.28 ?).

Please refer to GH issue 6071 which was fixed 6 days before in Feb. 2022

3
  • 1
    Yeah... so are there any other hacks other than pinning the version of requests?
    – Sam Zhang
    Apr 10, 2022 at 10:27
  • 1
    Use requests 2.27.0 (pip install requests==2.27.0) Dec 7, 2022 at 10:51
  • pip install requests==2.27.1 because hvac 2.1.0 requires requests<3.0.0,>=2.27.1, but you have requests 2.27.0 which is incompatible.
    – dragonfly
    Apr 3 at 16:34
2

My problem was that a third party package sent the request, so that I had no chance to influence the verify = True.

What helped me:

pip install pip-system-certs

And then:

import pip_system_certs #import as first package
1
  • This is the only solution that worked for me in Windows.
    – NLV
    Feb 9 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.