24

Background

"In the Python runtime, we've added support for the Python SSL Library, so you can now open secure connections to remote services such as Apple's Push Notification service."

This quote is taken from a recent post on the Google App Engine blog.


Implementation

If you want to use native python ssl, you must enable it using the libraries configuration in your application's app.yaml file where you specify the library name "ssl" . . .

These instructions are provided for developers through the Google App Engine documentation.

The following lines have been added to the app.yaml file:

libraries:
- name: ssl
  version: latest

This much is in line with the advice provided through the Google App Engine documentation.


Problem

I have tried running my project in three different configurations. Two are working, and one is not.

Working ...

After I upload my application to Google App Engine, and run my project through the live server, everything works fine.

Working ...

When I run my project with manage.py runserver and include the Google App Engine SKD in my PYTHONPATH, everything works fine.

Not Working ...

However, when I run my project with dev_appserver.py, I get the following error:

ImportError at /
No module named _ssl
Request Method: GET
Request URL:    http://localhost:8080/
Django Version: 1.4.3
Exception Type: ImportError
Exception Value:    
No module named _ssl
Exception Location: /usr/local/lib/google_appengine_1.7.7/google/appengine/tools/devappserver2/python/sandbox.py in load_module, line 856
Python Executable:  /home/rbose85/Code/venvs/appserver/bin/python
Python Version: 2.7.3
Python Path:    
['/home/rbose85/Code/product/site',
 '/usr/local/lib/google_appengine_1.7.7',
 '/usr/local/lib/google_appengine_1.7.7/lib/protorpc',
 '/usr/local/lib/google_appengine_1.7.7',
 '/usr/local/lib/google_appengine_1.7.7',
 '/usr/local/lib/google_appengine_1.7.7/lib/protorpc',
 '/usr/local/lib/google_appengine_1.7.7',
 '/usr/local/lib/google_appengine_1.7.7/lib/protorpc',
 '/home/rbose85/Code/venvs/appserver/lib/python2.7',
 '/home/rbose85/Code/venvs/appserver/lib/python2.7/lib-dynload',
 '/usr/lib/python2.7',
 '/usr/local/lib/google_appengine',
 u'/usr/local/lib/google_appengine_1.7.7/lib/django-1.4',
 u'/usr/local/lib/google_appengine_1.7.7/lib/ssl-2.7',
 u'/usr/local/lib/google_appengine_1.7.7/lib/webapp2-2.3',
 u'/usr/local/lib/google_appengine_1.7.7/lib/webob-1.1.1',
 u'/usr/local/lib/google_appengine_1.7.7/lib/yaml-3.10']
Server time:    Wed, 24 Apr 2013 11:23:49 +0000

7 Answers 7

46

For the current GAE version (1.8.0 at least until 1.8.3), if you want to be able to debug SSL connections in your development environment, you will need to tweak a little bit the gae sandbox:

  • add "_ssl" and "_socket" keys to the dictionary _WHITE_LIST_C_MODULES in /path-to-gae-sdk/google/appengine/tools/devappserver2/python/sandbox.py
  • Replace the socket.py file provided by google in /path-to-gae-sdk/google/appengine/dis27 from the socket.py file from your Python framework.

IMPORTANT: Tweaking the sandbox environment might end up with functionality working on your local machine but not in production (for example, GAE only supports outbound sockets in production). I will recommend you to restore your sandbox when you are done developing that specific part of your app.

8
  • This fix works for blocking (ssl) sockets but it won't work for non-blocking sockets. GAE (and the dev-server) wrap the select module too and the module works only with GAE socket objects. Doing a select() with a builtin socket raises exceptions. I tried to make the dev server use the builtin select module bit didn't find a way to do it. Any suggestions? Oct 25, 2013 at 10:38
  • 3
    Just a note for those (like me) who don't know who to find where is path-to-gae. Run in your Terminal: ls -l `which dev_appserver.py` this will show you the real path. Do a cd on the google_appengine dir, and then cd google/appengine/tools/devappserver2/python/. Mar 5, 2014 at 7:25
  • Actually for me, from the main Google App Engine directory, you cd into google/appengine/dist27 and socket.py is right there... as jmg states in his answer. Mar 8, 2014 at 3:11
  • this hack may trick the sdk but resulted with production errors (at least in my case)
    – akiva
    Apr 24, 2014 at 0:51
  • I got this error when I replaced file socket.py file File"/opt/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 950, in load_module raise ImportError('No module named %s' % fullname) ImportError: No module named _socket Any ideas how to fix it? I had found only windows answers.
    – alexche8
    Nov 10, 2015 at 16:00
16

The solution by jmg works, but instead of changing the sdk files, you could monkey patch the relevant modules.

Just put something like this on the beginning of your project setup.

# Just taking flask as an example
app = Flask('myapp')

if environment == 'DEV':
    import sys

    from google.appengine.tools.devappserver2.python import sandbox
    sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']

    from lib import copy_of_stdlib_socket.py as patched_socket

    sys.modules['socket'] = patched_socket
    socket = patched_socket
6
  • +1 for code only solution and no need to muster with the appengine files.
    – MeLight
    Aug 18, 2014 at 9:57
  • If you get here and use this solution and you then get an issue about non-blocking sockets (and using select). Adding these two lines to the fix, right below import sys will disable non-blocking sockets entirely: import select and then del sys.modules['select'].select.
    – hjc1710
    Mar 9, 2015 at 22:52
  • 3
    from lib import copy_of_stdlib_socket.py as patched_socket What's this line about? PyCharm says "module not found."
    – brandones
    Aug 29, 2015 at 0:53
  • 1
    For those wondering about copy_of_stdlib_socket.py, the idea is to literally copy the Python core socket.py file to a new file in your project's directory tree, with a name like copy_of_stdlib_socket.py, then import that. There's something overriding the default socket module, and we want to override it back.
    – brandones
    Sep 9, 2016 at 10:41
  • 1
    You can accomplish this without even having to copy socket.py to a new file, per this answer: stackoverflow.com/a/36272309/1464495.
    – brandones
    Sep 9, 2016 at 11:16
6

I had to use a slightly different approach to get this working in CircleCI (unsure what peculiarity about their venv config caused this):

appengine_config.py

import os

if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    import imp
    import os.path
    import inspect
    from google.appengine.tools.devappserver2.python import sandbox

    sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']
    # Use the system socket.

    real_os_src_path = os.path.realpath(inspect.getsourcefile(os))
    psocket = os.path.join(os.path.dirname(real_os_src_path), 'socket.py')
    imp.load_source('socket', psocket)
5
  • 2
    This answer seems like the safest route to go and should be marked up. I'll add that it should be placed in a file called appengine_config.py in your base app directory so that it gets loaded on startup.
    – speedplane
    Jun 21, 2016 at 18:21
  • line 737, in HTTPConnection I got timeout=socket._GLOBAL_DEFAULT_TIMEOUT, source_address=None): AttributeError: 'module' object has no attribute '_GLOBAL_DEFAULT_TIMEOUT'
    – Serge
    Jul 12, 2017 at 19:15
  • Looks a perfect solution but in my case I am using virtualenv and it's not able to find socket.py on manually downloading and placing socket.py it's giving an error saying [Errno 22] Invalid argument. Nov 21, 2017 at 6:11
  • @vinitpayal unfortunately GAE doesn't work well with venv. vendoring is the best way to go - cloud.google.com/appengine/docs/standard/python/tools/… Nov 21, 2017 at 20:11
  • @SpainTrain Thanks for your reply after investing a lot of time I was able to make this work on venv. I think the one which worked was stackoverflow.com/questions/33822937/… Nov 22, 2017 at 3:25
6

I had this problem because I wasn't vendoring ssl in my app.yaml file. I know the OP did that, but for those landing here for the OP's error, it's worth making sure lines like the following are in your app.yaml file:

libraries:
- name: ssl
  version: latest
2
  • They really should have mentioned this in cloud.google.com/appengine/docs/standard/python/getting-started/…
    – trinth
    Feb 20, 2019 at 4:38
  • seriously. GAE started out as their flagship PaaS and in this transition to 3.7, they've been treating it as a side project (ie nearly no local dev tools for 3.7, not updating 2.7 documentation when they make changes, not migrating built-in libraries (ie NDB) to 3.7, etc). Here's hoping the rest of 2019 shows improvement here as Python 2.7 gets sunset.
    – hamx0r
    Feb 22, 2019 at 3:54
5

Stumbled upon this thread trying to work with Apples Push notification service and appengine... I was able to get this working without any monkey patching, by adding the SSL library in my app.yaml, as recommended in the official docs, hope that helps someone else :)

1

I added the code to appengine_config.py as listed by Spain Train, but had to also add the following code as well to get this to work:

phttplib = os.path.join(os.path.dirname(real_os_src_path), 'httplib.py')
imp.load_source('httplib', phttplib)
-1

You can test if ssl is available at your local system by opening a python shell and typing import ssl. If no error appears then the problem is something else, otherwise you don't have the relevant libraries installed on your system. If you are using a Linux operating system try sudo apt-get install openssl openssl-devel or the relevant instructions for your operating system to install them locally. If you are using windows, these are the instructions.

3
  • thank you for your answer. Very good information, but not what I need. I agree with you, if I open the Python shell and type import ssl I can very quickly test if an SSL module is available to my environment. However, I am not trying to work with SSL on my local Python. The problem I have is working with SSL in Google App Engine. I am able to run my project, without issue, through Django's manage.py runserver. The problem is when running my project through AppEngine's dev_appserver.py. When I use dev_appserver.py I get the above stated issue.
    – rbose85
    Apr 26, 2013 at 19:03
  • 2
    any solution for your problem? running into it too.
    – skurt
    Jun 4, 2013 at 12:46
  • I don't have a clue, I thought it was a more system wide installation issue.
    – topless
    Apr 6, 2016 at 18:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.