2

I have been trying to perform an HTTPS request in Python 3 using requests and aggregating pretty much all the knowledge from the prior attempts documented on StackOverflow. I cannot for the life of me seem to get out of the sslv3 alert handshake failure rabbit hole.

This is my environment:

  • macOS 10.13.6
  • Python 3.7.0 (installed via Homebrew along with openssl)
  • OpenSSL 1.0.2p 14 Aug 2018 (output of print(ssl.OPENSSL_VERSION))
  • requests 2.19.1 (output of print(requests.__version__)) installed via pip install requests[security]
  • even cryptography 2.3.1 is installed

This is the bare-bone failing code:

>>> import requests
>>> requests.get('https://iris.nuigalway.ie')

And this is the output:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3-1.23-py3.7.egg/urllib3/contrib/pyopenssl.py", line 444, in wrap_socket
    cnx.do_handshake()
  File "/usr/local/lib/python3.7/site-packages/pyOpenSSL-18.0.0-py3.7.egg/OpenSSL/SSL.py", line 1907, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/local/lib/python3.7/site-packages/pyOpenSSL-18.0.0-py3.7.egg/OpenSSL/SSL.py", line 1639, in _raise_ssl_error
    _raise_current_error()
  File "/usr/local/lib/python3.7/site-packages/pyOpenSSL-18.0.0-py3.7.egg/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'ssl3_read_bytes', 'sslv3 alert handshake failure')]

Needless to say it works with cURL, browsers etc.

curl --verbose "https://iris.nuigalway.ie"

Here's a handshake snippet of the ouput:

* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.0 (IN), TLS handshake, Server hello (2):
* TLSv1.0 (IN), TLS handshake, Certificate (11):
* TLSv1.0 (IN), TLS handshake, Server finished (14):
* TLSv1.0 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.0 (OUT), TLS change cipher, Client hello (1):
* TLSv1.0 (OUT), TLS handshake, Finished (20):
* TLSv1.0 (IN), TLS change cipher, Client hello (1):
* TLSv1.0 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.0 / DES-CBC3-SHA
* ALPN, server did not agree to a protocol

Now the cipher used by cURL does not indeed seem to be among the default ciphers of urllib3 1.23 (seemingly used by requests) as per https://github.com/urllib3/urllib3/blob/1.23/urllib3/util/ssl_.py

So I tried adding it using the advice given at https://stackoverflow.com/a/40741362 like this:

>>> requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += 'DES-CBC3-SHA'

and even setting it to ALL. I even tried not to verify the certificate, all to no avail.

>>> requests.get('https://iris.nuigalway.ie', verify=False)

A check with s_client on the server:

$ openssl s_client -connect iris.nuigalway.ie:443

reveals the following TLS version and cipher:

New, TLSv1/SSLv3, Cipher is RC4-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-SHA

What options could I possibly have not tried yet?

Many Thanks

UPDATE

The values of ssl.OPENSSL_VERSION OpenSSL.SSL.SSLeay_version(0) revealed two different versions of OpenSSL used by ssl and pyOpenSSL respectively, the latter being a more recent OpenSSL 1.1.0i 14 Aug 2018 that has most likely dropped support for the DES-CBC3-SHA cipher.

Below is the temporary solution I have adopted:

  1. uninstall cryptography
  2. injecting only the required cipher, like this:

(Note that it is no longer a concatenation)

requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'DES-CBC3-SHA'

I appreciate that this solution may be sub-optimal and not applicable to many cases, but at least the lesson learnt is that different versions of OpenSSL may be at play from one package to another.

I will be happy to know of a more flexible solution if any.

5
  • You shouldn't have to mess with ciphers, and you certainly should not deactivate certificate validation. From your output it seems the server may speak only TLS 1.0 and not higher versions, so first try that. See "Example: Specific SSL Version" in docs.python-requests.org/en/master/user/advanced . Also make sure that you use the same CA bundle. Aug 23, 2018 at 18:37
  • * ALPN, server did not agree to a protocol. Maybe the server's SSL implementation doesn't know ALPN? Try building OpenSSL without ALPN support (if possible), or use an older version (e.g. 1.0.1 - which is no longer supported).
    – CristiFati
    Aug 23, 2018 at 21:45
  • @PatrickMevzek thanks, I tried something along those lines that I'm not sure is right though (and has led me to the same error as before): class Tls1HttpAdapter(HTTPAdapter): def init_poolmanager(self, connections, maxsize, block=False): self.poolmanager = PoolManager( num_pools=connections, maxsize=maxsize, block=block, ssl_version=ssl.PROTOCOL_TLSv1) then I mounted a requests.Session() with that Tls1HttpAdapter() and tried a get on that session, to no avail.
    – alexdma
    Aug 24, 2018 at 15:34
  • Just for paranoia's sake, can we see the output of print(requests.get('https://www.howsmyssl.com/a/check', verify=False).json()['tls_version']) just to make sure requests is trying a sane client? Aug 26, 2018 at 23:27
  • Apart from the insecure request warning, the output is 'TLS 1.2'
    – alexdma
    Aug 27, 2018 at 11:31

1 Answer 1

2

The server seems to be really broken. If you just add DES-CBC3-SHA to the list of ciphers it will not work, maybe because the server croaks because the client offers ciphers the server does not know or because of too much ciphers.

If one instead changes it to only allow this single cipher it works for me, i.e.

requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'DES-CBC3-SHA'
7
  • Thanks, though I get another error when trying that: OpenSSL.SSL.Error: [('SSL routines', 'SSL_CTX_set_cipher_list', 'no cipher match')] as if there was no such cipher in OpenSSL 1.0.2p. And yet it appears when I list them via /usr/local/opt/openssl/bin/openssl ciphers Was your environment otherwise the same as mine?
    – alexdma
    Aug 24, 2018 at 15:00
  • @alexdma: my environment was not the same as yours but I've used a openssl which has support for 3DES. Could it be that your /usr/local/opt/openssl/bin/openssl is not the same as the openssl library you use in Python? Aug 24, 2018 at 15:13
  • They are at least of the same version: /usr/local/opt/openssl/bin/openssl version and >>> ssl.OPENSSL_VERSION both print OpenSSL 1.0.2p 14 Aug 2018.
    – alexdma
    Aug 24, 2018 at 15:22
  • @alexdma: then I don't know - if you say that openssl version shows DES-CBC3-SHA as available you should also be able to set and use it as DEFAULT_CIPHERS as long as the same openssl library is used in both cases. Aug 24, 2018 at 16:27
  • 1
    @alexdma: maybe. Check OpenSSL.SSL.SSLeay_version(0) Aug 24, 2018 at 17:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.