7

I learned from the websitethat I should add the code declaration in python when i wan't to input friendly unicode characters: http://www.python.org/dev/peps/pep-0263/, but I still feel confused about it.

Assume that i work in linux with vim, and i create a new py file and input codes as follows:

#!/usr/bin/python2.7
# -*- coding: utf8 -*-
s = u'ޔ'
print s

1. I tried to replace line 2 with codes as follows:

import sys
reload(sys)
sys.setdefaultencoding('utf8')

but it doesn't work, aren't they same?

2. I am not very familiar with linux, I really dont know why should i add _*_ at the beginning and end of code delcaration, and when i tried to replaced # -*- coding: utf8 -*- with # code=utf8 or # code: utf8, I got an error:

File "pythontest.py", line 3
SyntaxError: Non-ASCII character '\xde' in file pythontest.py on line 3, but no encoding declared; see     http://www.python.org/peps/pep-0263.html for details

but these code declaration is mentioned in the website http://www.python.org/dev/peps/pep-0263/!

and according to the documentation , the code declaration as follows is allowed:

# This Python file uses the following encoding: utf-8

Oops, what's this? I don't think it can be recognized by computer.what in the world should the code declared? I feel more and more confused.

Thanks for help.

4 Answers 4

3

The abstract of the PEP you link really says it all:

This PEP proposes to introduce a syntax to declare the encoding of a Python source file. The encoding information is then used by the Python parser to interpret the file using the given encoding. Most notably this enhances the interpretation of Unicode literals in the source code and makes it possible to write Unicode literals using e.g. UTF-8 directly in an Unicode aware editor.

(the emphasis is mine).

Even if what you wanted to do would have worked (replacing the encoding of the source file programmatically), it wouldn't have had any sense. Think about it: the code is static (doesn't change). It would make no sense to try to read it with different encoding: there is only one correct one (the one the author of the source edited the source in).

As for the syntax:

# This Python file uses the following encoding: utf-8

the PEP itself says that that syntax is "Without interpreter line, using plain text". It is placed there for humans. So that if you open a file in a text editor and find it full of gibberish, you can manually set the encode of the source in its menu.

EDIT: As for why you should put the encoding between # -*- and -*-... That's purely conventional. The first symbol, the hash sign, tells that that is a comment (so it won't be compiled to bytecode), then the _*_ is just a way to tell the parser that that specific comment is for him/her.

It is not any different than putting in your source:

# TODO: fix this nasty bug

in which the TODO: part tells the developer (and some IDE) that this is a message requiring an action. You could have really used whatever your wanted, including @MarkZar or WTF!... just convention!

HTH!

2
  • Oh,i am so appreciated.it's really very detailed. but I still have something not very clear.Do the format of the code declaration vary depending on systems and editors? And if sys.setdefaultencoding is non-useful here, what's the real functionality of it? Thanks a lot.
    – Searene
    Commented Nov 27, 2011 at 2:02
  • @MarkZar - Some might say it was a design flaw of Python, as it generates more confusion than it is helpful. :) Here is an explanation of why its use have always been discouraged, also offering a number of links to blog posts illustrating situations in which the use of sys.setdefaultencoding is problematic.
    – mac
    Commented Nov 27, 2011 at 11:41
3

The important part of python encoding declaration is coding: utf-8 and it should be in a comment before the first line of python code, and you can do whatever you want with the other part of the comment.

Here is the lines in the PEP described this behaviour:

More precisely, the first or second line must match the regular expression "coding[:=]\s*([-\w.]+)". The first group of this expression is then interpreted as encoding name. If the encoding is unknown to Python, an error is raised during compilation. There must not be any Python statement on the line that contains the encoding declaration.

0

You need the line since you need to tell the compiler which encoding the source code uses.

0

The encoding setting is searched using the regular expression coding[:=]\s*([-\w.]+) anywhere on the line. This means:

  • find the exact string coding= or coding: followed by zero or more white-space characters, followed by a run of at least one character that are alphanumeric, _ or -.

  • capture the run of at least one...

  • the captured part is used as the encoding.

That is, it is perfectly legal to use anything like

# This program was written for Python 3. Encoding that should be used for decoding: UTF-8!

because the string in required format can still be found there.


Python 3 source files default to UTF-8 as the encoding, so no # coding: utf-8 is necessary in Python 3 code for as long as you use UTF-8.

1
  • It appears that UTF-8 was explicitly mandated in 3.4, and that early versions of Python 3 also defaulted to ASCII.
    – tripleee
    Commented Jun 13, 2018 at 6:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.