24

I'm trying to use the awesome Requests library on Google App Engine. I found a patch for urllib3, which requests relies on, that is compatible with App Engine. https://github.com/shazow/urllib3/issues/61

I can successfully

import requests

but then

response = requests.get('someurl')

fails with the following traceback. What's going on?

Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/admin/__init__.py", line 317, in post
    exec(compiled_code, globals())
  File "<string>", line 6, in <module>
  File "/Users/Rohan/Dropbox/MuktiTechnologiesINC/MuktiTechnologies/GAE/humanmictweet/GAE/libraries/requests/api.py", line 52, in get
    return request('get', url, **kwargs)
  File "/Users/Rohan/Dropbox/MuktiTechnologiesINC/MuktiTechnologies/GAE/humanmictweet/GAE/libraries/requests/api.py", line 40, in request
    return s.request(method=method, url=url, **kwargs)
  File "/Users/Rohan/Dropbox/MuktiTechnologiesINC/MuktiTechnologies/GAE/humanmictweet/GAE/libraries/requests/sessions.py", line 208, in request
    r.send(prefetch=prefetch)
  File "/Users/Rohan/Dropbox/MuktiTechnologiesINC/MuktiTechnologies/GAE/humanmictweet/GAE/libraries/requests/models.py", line 458, in send
    self.auth = get_netrc_auth(url)
  File "/Users/Rohan/Dropbox/MuktiTechnologiesINC/MuktiTechnologies/GAE/humanmictweet/GAE/libraries/requests/utils.py", line 43, in get_netrc_auth
    for loc in locations:
  File "/Users/Rohan/Dropbox/MuktiTechnologiesINC/MuktiTechnologies/GAE/humanmictweet/GAE/libraries/requests/utils.py", line 40, in <genexpr>
    locations = (os.path.expanduser('~/{0}'.format(f)) for f in NETRC_FILES)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 260, in expanduser
    userhome = pwd.getpwuid(os.getuid()).pw_dir
AttributeError: 'module' object has no attribute 'getuid'
1

3 Answers 3

14

As mentioned, master branch of standalone urllib3 supposedly supports AppEngine now (I'll do a proper PyPI release once someone confirms this fact), but Requests does not yet support AppEngine since it makes assumption about various filesystem things to load configuration files which don't exist on AppEngine. Specifically the error you ran into has to do with loading the ~/.netrc configuration file.

See Issue #493.

For what it's worth, the equivalent in urllib3 would be:

import urllib3
http = urllib3.PoolManager()
response = http.request('GET', 'someurl')

Update: urllib3 v1.3 was released yesterday which includes AppEngine support.

1
  • thank much, shazow. I'm using your patched urllib3 on App Engine successfully. Guess we'll have to see how difficult abstracting away some of this low-level filesystem stuff is.
    – rd108
    Mar 19, 2012 at 0:24
10

You can use the latest version of Requests on Google App Engine with the help of requests-toolbelt. This configures Requests to use urllib3's underlying support for App Engine's URLFetch service.

9

On Google Appengine (version 1.9.18) requests version 2.3.0 (only!) works IN PRODUCTION (but not on SDK) if you have billing enabled, which enables sockets support.

requests on the Appengine SDK fails with all https:// requests:

  ConnectionError: ('Connection aborted.', error(13, 'Permission denied'))

requests version 2.4.1 fails with:

  File "distlib/requests/adapters.py", line 407, in send
    raise ConnectionError(err, request=request)
  ConnectionError: ('Connection aborted.', error(13, 'Permission denied'))

requests version 2.5.1 fails with:

  File "distlib/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
  ConnectionError: ('Connection aborted.', error(13, 'Permission denied'))

Info on sockets support: https://cloud.google.com/appengine/docs/python/sockets/

PS: Replace awsome with very-painful if you intend to use requests on GAE.

See Also: Can Python Requests library be used on Google App Engine?

3
  • But it's possible to use urlfetch. Wonder if it's possible to make requests use it.
    – wlnirvana
    Feb 23, 2015 at 11:34
  • No. The maintainers of requests do not support appengine. In most cases you can replace/monkeypatch the calls from 3rd party libs to requests with urlfetch pretty easily, or you monkeypatch requests directly with compatible calls to urlfetch. We had a module called fake_requests.py emulating requests.
    – cat
    Mar 7, 2015 at 10:39
  • version 2.3.0 indeed did the trick for me. See stackoverflow.com/questions/29301863/…
    – Jimmy Kane
    Mar 28, 2015 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.