3

This question is still unsolved! Please answer if you know

Bug

I have filed a bug here

http://code.google.com/p/google-api-python-client/issues/detail?id=131&thanks=131&ts=1335708962

While working on my gdrive-cli project, I ran into this error attempting to upload a UTF-8 markdown file, using the "text/plain" mime-type. I also tried with "text/plain;charset=utf-8" and got the same result.

Here is the stacktrace I got:

Traceback (most recent call last):
  File "./gdrive-cli", line 155, in <module>
    handle_args(args)
  File "./gdrive-cli", line 92, in handle_args
    handle_insert(args.insert)
  File "./gdrive-cli", line 126, in handle_insert
    filename)
  File "/home/tom/Github/gdrive-cli/gdrive/gdrive.py", line 146, in insert_file
    media_body=media_body).execute()
  File "/usr/local/lib/python2.7/dist-packages/apiclient/http.py", line 393, in execute
    headers=self.headers)
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 401, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1544, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1294, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1231, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 955, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 989, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 951, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 809, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4518: ordinal not in range(128)

And the command I had to issue to generate it was:

gdrive-cli --insert README.md "readme file" none "text/plain" README.md

You can get the exact README.md file at the time this problem occurred here, http://tomdignan.com/files/README.md

The relevant code from the SDK examples follows. The parameters going in are in order:

a service instance, "README.md", "readme file", None (python keyword), "text/plain", and "README.md"

def insert_file(service, title, description, parent_id, mime_type, filename):
    """Insert new file.

    Args:
        service: Drive API service instance.
        title: Title of the file to insert, including the extension.
        description: Description of the file to insert.
        parent_id: Parent folder's ID.
        mime_type: MIME type of the file to insert.
        filename: Filename of the file to insert.
    Returns:
        Inserted file metadata if successful, None otherwise.
    """
    media_body = MediaFileUpload(filename, mimetype=mime_type)
    body = {
        'title': title,
        'description': description,
        'mimeType': mime_type
    }

    # Set the parent folder.
    if parent_id:
        body['parentsCollection'] = [{'id': parent_id}]


    try:
        file = service.files().insert(
                body=body,
                media_body=media_body).execute()

        # Uncomment the following line to print the File ID
        # print 'File ID: %s' % file['id']

        return file
    except errors.HttpError, error:
        print "TRACEBACK"
        print traceback.format_exc()
        print 'An error occured: %s' % error
        return None
2
  • One bug is enough. If you truly believe this is a bug, which it seems you do, then we will respond to the bug you filed. Posting here just duplicates the number of responses that need to be given to you.
    – Vic Fryzel
    Apr 30, 2012 at 2:58
  • @VicFryzel I thought about it and you are right. But the question is stuck. Tried to delete and failed. If it makes any difference, I posted the question before the bug. Apr 30, 2012 at 12:08

2 Answers 2

6

Traceback:

...
resp = pool.request(method, page, fields = fields)
  File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/request.py", line 79, in request
**urlopen_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/request.py", line 139, in request_encode_body
**urlopen_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/connectionpool.py", line 415, in urlopen
body=body, headers=headers)
  File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/connectionpool.py", line 267, in _make_request
conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python2.7/httplib.py", line 958, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 992, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 812, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 181: ordinal not in range(128)

urllib3

filepost.py:

def encode_multipart_formdata(fields, boundary=None):
    ...
    body = BytesIO()
    ...
    return body.getvalue(), content_type

request.py:

def request(self, method, url, fields=None, headers=None, **urlopen_kw):
    ...
    else:
        return self.request_encode_body(method, url, fields=fields,
                                        headers=headers,
                                        **urlopen_kw)

def request_encode_body(self, method, url, fields=None, headers=None,
                            encode_multipart=True, multipart_boundary=None,
                            **urlopen_kw):
    ...
    if encode_multipart:
        body, content_type = encode_multipart_formdata(fields or {},
                                                       boundary=multipart_boundary)
    ...
    headers = headers or {}
    headers.update({'Content-Type': content_type})

    return self.urlopen(method, url, body=body, headers=headers,
                        **urlopen_kw)

httplib

httplib.py:

def _send_request(self, method, url, body, headers):
    ...
    if body and ('content-length' not in header_names):
        self._set_content_length(body)
    ...

def _set_content_length(self, body):
    # Set the content-length based on the body.
    thelen = None
    try:
        thelen = str(len(body))
    except TypeError, te:
        # If this is a file-like object, try to
        # fstat its file descriptor
        try:
            thelen = str(os.fstat(body.fileno()).st_size)
        except (AttributeError, OSError):
            # Don't send a length if this failed
            if self.debuglevel > 0: print "Cannot stat!!"

    if thelen is not None:
        self.putheader('Content-Length', thelen)

    ...

def _send_output(self, message_body=None):
    ...
    if isinstance(message_body, str):
        msg += message_body
        message_body = None
    self.send(msg)
    if message_body is not None:
        #message_body was not a string (i.e. it is a file) and
        #we must run the risk of Nagle
        self.send(message_body)

    ...

def send(self, data):
    ...
    blocksize = 8192
    if hasattr(data,'read') and not isinstance(data, array):
        if self.debuglevel > 0: print "sendIng a read()able"
        datablock = data.read(blocksize)
        while datablock:
            self.sock.sendall(datablock)
            datablock = data.read(blocksize)
    else:
        self.sock.sendall(data)

My script.py:

#!/usr/bin/env python

from io import BytesIO

data = BytesIO()

print type(data.getvalue()

$./script.py

<type 'str'>

My solution:

#!/usr/bin/python

from urllib3 import HTTPConnectionPool, filepost
from io import BytesIO, SEEK_SET

pool = HTTPConnectionPool('web')

archive_name = '/var/archives/mock.tar.gz'

fields = {'uploadedfile' : (archive_name, open(archive_name).read())}

value, content_type = filepost.encode_multipart_formdata(fields)

headers = {'Content-Type' : content_type, 'Content-Length' : len(value)}

# or any temporary file. without Content-Length

data = BytesIO()

data.write(value)

data.seek(0, SEEK_SET)

resp = pool.urlopen('POST', '/upload/uploader.php', data, headers)

print resp.status

$./upload.py

200

/var/www/upload# ls

mock.tar.gz upload.html upload.php

2

Try encoding your message_body string:

msg += message_body.encode('utf-8')

This post really helped me when it comes to python strings and unicode.

1
  • This error is probably being caused by the line "Q. Why am I am getting "The authenticated user has not installed the app with client id†error?" in your readme. A cheap solution to fixing this would be to replace "idâ€".
    – garnertb
    Apr 29, 2012 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.