6

I am trying to use Python 2.7 mechanize to log into Mint.com with the following code:

import mechanize
br = mechanize.Browser()
br.open("https://wwws.mint.com/login.event")

this works just fine on OSX, but it hangs on debian. The issue seems to be ssl-related; the traceback ends with

File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
    self._sslobj.do_handshake()

EDIT: the issue persists on Debian using urllib2. As suggested in the comments, it seems the issue is actually ssl related. Why would this be a problem on Debian and not OSX?

4
  • I think it's not related to mechanize itself, it's relevant to python stdlib - see this issue.
    – alecxe
    Jul 6, 2013 at 17:37
  • @alecxe thanks alot, that does seem applicable. However, according to that thread, this issue should be patched... which obviously it isn't. Any idea what I could do to get things to work? Also, if this bug were causing the problem, wouldn't it be a problem on OSX also, since they're both Python 2.7.3?
    – rickcnagy
    Jul 6, 2013 at 18:27
  • Well, it's hard to say. The code works for me on ubuntu 12.04 using python 2.7.3.
    – alecxe
    Jul 6, 2013 at 18:46
  • What's the specific version of the Python Debian packages involved?
    – ajd
    Aug 9, 2013 at 13:53

2 Answers 2

1

So things look ok on Fedora:

[bharrington@leviathan ~]$ python
Python 2.7.5 (default, Aug 22 2013, 09:31:58) 
[GCC 4.8.1 20130603 (Red Hat 4.8.1-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mechanize
>>> br = mechanize.Browser()
>>> br.open("https://wwws.mint.com/login.event")
<response_seek_wrapper at 0x29b6440 whose wrapped object = <closeable_response at 0x29b6320 whose fp = <socket._fileobject object at 0x298d150>>>
>>> br.title()
'Mint > Start Here'
>>> 

Which leads me to wonder about the SSL/OpenSSL libraries in use. Just to test it out can you run from the command line:

$ openssl s_client -connect wwws.mint.com:443

You should see the SSL certificate for mint displayed, along with the full certificate chain verification, and the final line: " Verify return code: 0 (ok)"

While I highly doubt it is an SSL issue directly I figure it's worth checking into. Additionally, verify the version of mechanize. Debian is notable for using stable versions of code (not new versions). The version I verified with of mechanize was 0.2.5

1

This is a manifestation of an incompatibility between recent versions of OpenSSL and certain Web servers. Apple's doing their best to eliminate OpenSSL on OS X, so they're only applying security patches (OpenSSL has been difficult to support as part of the OS, not to mention "minor" updates introducing issues like this one), whereas Debian is using a more recent OpenSSL 1.0.1.

@Brian Redbeard’s suggestion to check with the openssl command-line is a good one — it hung for me on wwws.mint.com when I just tried it.

This question on ServerFault finally provided the answer. The SSLLabs test linked there identifies long handshake intolerance as the issue, which affects OpenSSL 1.0.1 and later, and links to an OpenSSL bug with some potential workarounds.

Either using -no_tls1_2, as one of the OpenSSL developers recommends, or reducing the cipher list with the -cipher argument, causes OpenSSL 1.0.1 to successfully handshake with wwws.mint.com (as well as another server I was trying to contact).

For my purposes — a script that isn't going to be distributed — I monkeypatched ssl.wrap_socket as follows:

import ssl
old_wrap_socket = ssl.wrap_socket
def wrap_socket(sock, keyfile=None, certfile=None,
                server_side=False, cert_reqs=ssl.CERT_NONE,
                ssl_version=ssl.PROTOCOL_SSLv3, ca_certs=None,
                do_handshake_on_connect=True,
                suppress_ragged_eofs=True, ciphers=None):
    return old_wrap_socket(sock, keyfile, certfile,
                           server_side, cert_reqs, ssl_version,
                           ca_certs, do_handshake_on_connect,
                           suppress_ragged_eofs, ciphers)
ssl.wrap_socket = wrap_socket

import mechanize

The default value for ssl_version is ssl.PROTOCOL_SSLv23; by changing it to PROTOCOL_SSLv3 it successfully connects.

You could guard this patch using a test such as ssl.OPENSSL_VERSION_INFO[:3] >= (1, 0, 1).

This should likely be reported as a Debian OpenSSL bug if it hasn't been already.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.