55

I am running a Python code where I have to get some data from HTTPSConnectionPool(host='ssd.jpl.nasa.gov', port=443). But each time I try to run the code I get the following error. I am on MAC OS 12.1

raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='ssd.jpl.nasa.gov', port=443): Max retries exceeded with url: /api/horizons.api?format=text&EPHEM_TYPE=OBSERVER&QUANTITIES_[...]_ (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:997)')))

I really don't know how to bypass this issue. thank you for the help!

9 Answers 9

96

WARNING: When enabling Legacy Unsafe Renegotiation, SSL connections will be vulnerable to the Man-in-the-Middle prefix attack as described in CVE-2009-3555.

With the help of https://bugs.launchpad.net/bugs/1963834 and https://bugs.launchpad.net/ubuntu/+source/gnutls28/+bug/1856428

Beware that editing your system's openssl.conf is not recommended, because you might lose your changes once openssl is updated.

Create a custom openssl.cnf file in any directory with these contents:

openssl_conf = openssl_init

[openssl_init]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
Options = UnsafeLegacyRenegotiation

Before running your program, make sure your OPENSSL_CONF environment variable is set to your custom openssl.cnf full path when running the scraper like so:

OPENSSL_CONF=/path/to/custom/openssl.cnf python your_scraper.py

or like so:

export OPENSSL_CONF=/path/to/custom/openssl.cnf
python your_scraper.py

or, if you are using pipenv or systemd or docker, place this into your .env file

OPENSSL_CONF=/path/to/custom/openssl.cnf
3
  • 1
    This worked for me as well, although I don't fully don't understand what I'm doing. (just trying to get TD ameritrade stocks. Was working before upgrading to 22.04) The file to edit is Modify the existing openssl config file, path: /usr/lib/ssl/openssl.cnf
    – Chad
    May 18, 2022 at 15:18
  • @Chad It is not recommended to change system defaults because they might be overridden by package changes. Happens to everyone. There is a cleaner way, I'm editing the answer to publish a working example.
    – nurettin
    May 28, 2022 at 15:26
  • Worked for me ... I was facing problems using Ubuntu 22.04 (openssl 3.0.2) which is default python 3.10.6. I can not download a corpus using nltk.download() as source seems to use legacy functions ... I am getting this [nltk_data] Error loading stopwords: <urlopen error [SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED]
    – Rafael
    Jun 13 at 21:22
29

I hit the same error on Linux (it happens when the server doesn't support "RFC 5746 secure renegotiation" and the client is using OpenSSL 3, which enforces that standard by default).

Here is a solution (you may have to adjust it slightly).

  1. Import ssl and urllib3 in your Python code
  2. Create a custom HttpAdapter which uses a custom ssl Context
class CustomHttpAdapter (requests.adapters.HTTPAdapter):
    '''Transport adapter" that allows us to use custom ssl_context.'''

    def __init__(self, ssl_context=None, **kwargs):
        self.ssl_context = ssl_context
        super().__init__(**kwargs)

    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = urllib3.poolmanager.PoolManager(
            num_pools=connections, maxsize=maxsize,
            block=block, ssl_context=self.ssl_context)
  1. Set up an ssl context which enables OP_LEGACY_SERVER_CONNECT, and use it with your custom adapter.

ssl.OP_LEGACY_SERVER_CONNECT is not available in Python yet (https://bugs.python.org/issue44888). However it turns out that in OpenSSL its value is 0x4 in the bitfield. So we can do the following.

ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
ctx.options |= 0x4
session.mount('https://', CustomHttpAdapter(ctx))
8
  • Hey Harry! I tried to use your code but I get unresolved references for requests. and session. as they are not defined Mar 28, 2022 at 15:22
  • 1
    session is an instance of requests.Session(). My code snippets are incomplete as they were taken from a larger project. You will have to adjust them to fit your code. Mar 28, 2022 at 16:01
  • 1
    This leads to: ValueError: Cannot set verify_mode to CERT_NONE when check_hostname is enabled. simply add, then it should be running >>> ctx.check_hostname = False >>> ctx.verify_mode = ssl.CERT_NONE
    – laol
    Apr 21, 2022 at 8:26
  • This also worked for me, probably the best future proof alternative to downgrading Sep 9, 2022 at 11:30
  • 1
    This works, the question is: Am I vulnerable to the Man-in-the-Middle prefix attack as described in CVE-2009-3555 if I use this solution?
    – Nachengue
    Jun 7 at 0:16
23

Complete code snippets for Harry Mallon's answer:

Define a method for reuse:

import requests
import urllib3
import ssl


class CustomHttpAdapter (requests.adapters.HTTPAdapter):
    # "Transport adapter" that allows us to use custom ssl_context.

    def __init__(self, ssl_context=None, **kwargs):
        self.ssl_context = ssl_context
        super().__init__(**kwargs)

    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = urllib3.poolmanager.PoolManager(
            num_pools=connections, maxsize=maxsize,
            block=block, ssl_context=self.ssl_context)


def get_legacy_session():
    ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
    ctx.options |= 0x4  # OP_LEGACY_SERVER_CONNECT
    session = requests.session()
    session.mount('https://', CustomHttpAdapter(ctx))
    return session

Then use it in place of the requests call:

get_legacy_session().get("some-url")
2
  • 2
    Nice! I now simply used with (get_legacy_session() as s, s.get("some-url") as response) and it works. Very useful for deployment in google cloud (as I could not downgrade SSL here) Sep 9, 2022 at 11:31
  • does not work ''' r = get_legacy_session().post(url, params=params, data=payload) '''
    – Dean Chen
    Oct 24, 2022 at 7:04
14

This error comes up when using OpenSSL 3 to connect to a server which does not support it. The solution is to downgrade the cryptography package in python:

run pip install cryptography==36.0.2 in the used enviroment.

source: https://github.com/scrapy/scrapy/issues/5491

EDIT: Refer to Hally Mallon and ahmkara's answer for a fix without downgrading cryptography

4
  • 5
    Didn't worked for me and api.searchads.apple.com
    – mastisa
    Aug 17, 2022 at 10:14
  • 1
    I had same problem with python requests 2.28.1 via cryptography 37.0.2. I downgraded the cryptography to 36.0.2 and problem fixed. tnx a lot :X
    – ali reza
    Aug 18, 2022 at 3:05
  • 7
    Tried with python3.8, 3.9, 3.10 on ubuntu 22.04 - no luck Feb 20 at 15:47
  • 4
    This method didn't work for me either. Apr 9 at 11:20
6

This doesn't really answer the issue, but a coworker switched from Node 18 to 16 and stopped getting this error.

2

If you want to use urlopen, this snippet worked for me.

import ssl
import urllib.request

url = 'http://....'

# Set up SSL context to allow legacy TLS versions
ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
ctx.options |= 0x4  # OP_LEGACY_SERVER_CONNECT

# Use urllib to open the URL and read the content
response = urllib.request.urlopen(url, context=ctx)
1

To fix the same problem in ruby you can do below:

# Set OP_LEGACY_SERVER_CONNECT option
OpenSSL::SSL::SSLContext::DEFAULT_PARAMS[:options] |= OpenSSL::SSL::OP_LEGACY_SERVER_CONNECT

# Make a request
uri = URI('https://example.com')
res = Net::HTTP.post(uri, {}.to_json)

# Unset OP_LEGACY_SERVER_CONNECT option
OpenSSL::SSL::SSLContext::DEFAULT_PARAMS[:options] &= ~OpenSSL::SSL::OP_LEGACY_SERVER_CONNECT
0

If you are using conda, usually conda installs a new openssl executable with each environment. One easy fix is to downgrade your openssl to 1.0 by running the following with your environment.

conda install -n conda-env-name openssl=1

Or find where the openssl config is for your specific conda environment and follow Jack Lee's answer.

You'll have to closely monitor the SSL versions from this website to ensure you specify the correct channel. https://anaconda.org/conda-forge/openssl/labels

-1

For me, it worked when I downgraded python to v3.10.8.

(If you are facing the issue in docker container, read below)

In my docker image, I was using alpine-10 which was using v3.10.9. Since I couldn't get alpine with v3.10.8, I used 3.10.8-slim-bullseye.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.