19

On Windows 10 the logging module send this error (using scrapy)

# --- Logging error ---
...
# UnicodeEncodeError: 'charmap' codec can't encode characters in position 175-176: character maps to <undefined>

I have read that I should add encoding='utf-8' but I did not find how I could add it in the code below. EDIT: According to the tuto it isnt needed.

configure_logging(install_root_handler=False) #override default log settings
logging.basicConfig(
    filename='logfile.log',
    format='%(levelname)s: %(message)s',
    datefmt='%m-%d %H:%M',
    level=logging.INFO #CRITICAL ERROR WARNING  INFO    DEBUG    NOTSET 
)

I found many questions on the topics, but mostly on python 2 (or not related to the logging module). And the logging tutorial don't even talk about utf-8. (Notice that I can print UTF8 characters without any problem. The problem only occur with the logging module)

1 Answer 1

37

I'm not using Scrapy but I ran into the same issue with vanilla Python 3.6 logging of not being able to write UTF-8 characters via logging to a file (console worked just fine).

Based on this comment I added 'utf-8' to my FileHandler initialization. I'm configuring logging via an INI file so it looks like this:

[handler_file]
class = FileHandler
args = (r'log.txt', 'a', 'utf-8')
level = NOTSET
formatter = generic

To use logging.basicConfig() I think that you should be able to do the following:

configure_logging(install_root_handler=False) #override default log settings
logging.basicConfig(
    handlers=[logging.FileHandler('logfile.log', 'w', 'utf-8')],
    format='%(levelname)s: %(message)s',
    datefmt='%m-%d %H:%M',
    level=logging.INFO #CRITICAL ERROR WARNING  INFO    DEBUG    NOTSET 
)
6
  • The answer with logging.basicConfig() doesn't work for me.
    – Pixel78
    Commented Nov 3, 2020 at 10:48
  • @SomJura, can you be a little more specific? I just tried it on Python 3.8 and it seemed to work (I did skip the configure_logging() call).
    – smsearcy
    Commented Nov 6, 2020 at 4:28
  • It seems that it didn't work because I was using import logging, which is not necessay anymore with newer versions of Python.
    – Pixel78
    Commented Nov 8, 2020 at 12:54
  • Is it possible in windows? Facing the same issue, I have tried the above solution but it's not working. I'm using Python 3.8.10. I tried removing the import statement, 'import logging', but that's causing errors. 'NameError: name 'logging' is not defined' Commented Oct 26, 2021 at 6:32
  • @Abhi, I initially had this problem in Windows so it should work in Windows. What error are you having? Reviewing the second logging.basicConfig() example, I'm not sure what that first line with configure_logging() is doing there, what if you delete that? You will need import logging.
    – smsearcy
    Commented Oct 28, 2021 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.