4
  • Django 1.6.1
  • Python 3.2.3
  • Apache 2.2.2
  • Ubuntu 12.04
  • Postgres 9.1.11

I'm getting this error when trying to import a serialized XML file. I exported it from one database that I was using for a development server, and moments later I import it to another database on a different development server (on the same machine). I'm very baffled as to why it's not handling the UTF-8 characters on import. I didn't do anything special. Plus, the same code worked when I last tested it. Here's the code I'm using to export & import, in an app's views.py...

from django.shortcuts import render, render_to_response, get_object_or_404
from django.http import HttpResponseRedirect, HttpResponse
from django.contrib.auth.decorators import user_passes_test
from django.core import serializers

@user_passes_test(lambda u: u.is_superuser)
def export_serial(request):
    '''
    This exports data using Django serializing.
    See: https://docs.djangoproject.com/en/dev/topics/serialization/
    '''
    XMLSerializer = serializers.get_serializer("xml")
    xml_serializer = XMLSerializer()
    xml_serializer.serialize(list(ThisModel.objects.all()) 
                        + list(ThatModel.objects.all())
                          + list(ThirdModel.objects.all()))
    data = xml_serializer.getvalue()

    response = HttpResponse(mimetype='text/xml')
    response['Content-Disposition'] = 'attachment; filename=my-backup.xml'
    response.write(data)
    return response


@user_passes_test(lambda u: u.is_superuser)
def import_serial(request):
    '''
    This is for importing data that was saved/backed up, as serialized.
    '''
    xmlfile = '/home/zamphatta/path/to/my-backup.xml'
    serialfile = open(xmlfile,'r')
    data = serialfile.read()
    serialfile.close()

    for deserialized_object in serializers.deserialize("xml", data):
        deserialized_object.save()

    template = loader.get_template('z-style/index.htm')
    context = RequestContext(request, {
        'title': data,
    })
    return HttpResponse(template.render(context))

I Googled & I checked out all StackOverflow questions that were related (but the answers don't clue me in to an answer for this), and still don't have a clue why this is throwing me this error. All I know is that non-ascii characters are making it puke, which doesn't make sense 'cause Python has no problem with UTF-8 characters and I'm not doing any weird stuff that would cause the code to suddenly treat it differently.

UPDATE: Here's a fuller version of the error message...

UnicodeDecodeError at /sdb/import/

'ascii' codec can't decode byte 0xe2 in position 5367: ordinal not in range(128)

Request Method: GET

Request URL:    http://coral/sdb/import/

Django Version: 1.6.1

Exception Type: UnicodeDecodeError

Exception Value:    
'ascii' codec can't decode byte 0xe2 in position 5367: ordinal not in range(128)

Exception Location: /usr/lib/python3.2/encodings/ascii.py in decode, line 26

Python Executable:  /usr/bin/python

Python Version: 3.2.3

Python Path:    
['/usr/local/lib/python3.2/dist-packages/distribute-0.6.49-py3.2.egg',
 '/usr/local/lib/python3.2/dist-packages/CherryPy-3.2.4-py3.2.egg',
 '/usr/lib/python3.2',
 '/usr/lib/python3.2/plat-linux2',
 '/usr/lib/python3.2/lib-dynload',
 '/usr/local/lib/python3.2/dist-packages',
 '/usr/local/lib/python3.2/dist-packages/setuptools-0.6c11-py3.2.egg-info',
 '/usr/lib/python3/dist-packages',
 '/home/dfy/code/zamphatta/',
 '/home/dfy/code/zamphatta/zamphatta/']
3
  • 2
    have you tried like this: your_string.encode('UTF-8') ?
    – mortymacs
    Jan 13, 2014 at 10:33
  • Could you provide full stacktrace of that error? Jan 13, 2014 at 10:35
  • Can you show us a traceback? Jan 13, 2014 at 10:44

1 Answer 1

0

Seems when the server was started up python3 manage.py runserver, the import would work just fine. It wasn't working when I was importing it through Django when Django was running on Apache via wsgi. So I'm guessing Apache was somehow interfering with it. It works now, as long as I import straight through Django.

6
  • 1
    All of my problems about ascii code errors , resolved with decode('UTF-8') or encode('UTF-8'). these can solve your problem.
    – mortymacs
    Jan 13, 2014 at 11:01
  • Where would I have put it though? It was just generated export & import that was handled by the same app.
    – Zamphatta
    Jan 13, 2014 at 11:03
  • please tell me exact line that you got error. data = serialfile.read() in import function?
    – mortymacs
    Jan 13, 2014 at 11:06
  • Don't know the exact line now, but it was coughing up on a lot of lines. Always due to characters like '♯', 'á', and '博客'.
    – Zamphatta
    Jan 13, 2014 at 11:13
  • please try it: serializers.deserialize("xml", data.encode('UTF-8')) instead of serializers.deserialize("xml", data). and tell me the results.
    – mortymacs
    Jan 13, 2014 at 11:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.