2

I am trying to return a request but it is giving me an error that there are non-unicode characters in the string. I am filtering them out but then it makes the string in unicode style which crashes the app with a badly formatted response.

Here is what I am trying to do

unfiltered_string = str({'location_id': location.pk, 'name': location.location_name,'address': location.address+', '+location.locality+', '+location.region+' '+location.postcode, 'distance': location.distance.mi, })
filtered_string = str(filter(lambda x: x in string.printable, unfiltered_string)).encode("utf-8")
locations.append(filtered_string)

The troubles is it appends a string that looks like

{'distance': 4.075068111513138, 'location_id': 1368, 'name': u'Stanford University', 'address': u'450 Serra Mall, Stanford, CA 94305'}

when I need the u'string' to just be 'string' like this

{'distance': 4.075068111513138, 'location_id': 1368, 'name': 'Stanford University', 'address': '450 Serra Mall, Stanford, CA 94305'}

if I try using string.encode('ascii','ignore') then I still get

"{'location_id': 1368, 'address': u'450 Serra Mall, Stanford, CA 94305', 'distance': 4.075068111513138, 'name': u'Stanford University'}"

and now I get extra quotations around the json

2
  • 2
    Please show us your full traceback and the code that produces the error. I'm sure that what you are doing is not correct for the problem you are facing.
    – Martijn Pieters
    Feb 21, 2015 at 18:54
  • “when I need the u'string' to just be 'string' like this”—why? You are producing repr output via dict.str() and this is valid Python literal syntax for Unicode strings. If you are trying to produce some format that is not a Python literal, for example JavaScript, then you shouldn't be using Python repr output (consider eg a JSON encoder instead).
    – bobince
    Feb 22, 2015 at 16:50

1 Answer 1

1

So, I'm going to go out on a limb here and say that your goal here is to ignore the unicode specific characters that you've got. I think it's really difficult to say anything definitive without a better explanation in your question, but if you're looking to get a "plain" string instead of a unicode one I would suggest using the ascii codec for encoding instead of utf-8.

<str>.encode('ascii')

If you want to remove the other characters, the encode function takes an optional second argument allowing you to ignore all characters that the specified codec can't handle:

<str>.encode('ascii', 'ignore')
3
  • 1
    There is no such thing as a type-cast in Python, using str() produces a new object of the str() type provided the unicode string can be encoded with the default codec. Using str() means you are relying on unicode_version to be encodable to bytes with the ASCII codec; you probably want to use an explicit codec instead.
    – Martijn Pieters
    Feb 21, 2015 at 18:52
  • @MartijnPieters that's totally fair, sorry, I think I jumped the gun on the answer a little bit before seeing that there was an explicit utf-8 encoding above, revising now. Feb 21, 2015 at 18:54
  • @TylerRice It's still very, very hard to give accurate feedback without a fuller picture of what you're trying to do. If you're just trying to get rid of the u, you can still do str(<u'text'>), but I have a feeling that there's more to it than that. Feb 21, 2015 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.