344

For the following code:

logger.debug('message: {}'.format('test'))

pylint produces the following warning:

logging-format-interpolation (W1202):

Use % formatting in logging functions and pass the % parameters as arguments Used when a logging statement has a call form of “logging.(format_string.format(format_args...))”. Such calls should use % formatting instead, but leave interpolation to the logging function by passing the parameters as arguments.

I know I can turn off this warning, but I'd like to understand it. I assumed using format() is the preferred way to print out statements in Python 3. Why is this not true for logger statements?

2
  • 7
    Is this still up-to-date? Isn't the usage of string interpolation vs. f-strings generally deprecated? What is the drawback of writing: the_msg = ... [...] logger.debug(f'message: {the_msg}') besides the noise generated by the linter? Oct 26, 2022 at 13:42
  • f-strings produce warning W1203: "Use lazy % formatting in logging functions (logging-fstring-interpolation)", so I guess not. It's true, I think, that the preferred way of formatting strings is to use f-strings, but in this case it wastes CPU cycles.
    – Paul
    Dec 19, 2022 at 17:55

9 Answers 9

412

It is not true for logger statement because it relies on former "%" format like string to provide lazy interpolation of this string using extra arguments given to the logger call. For instance instead of doing:

logger.error('oops caused by %s' % exc)

you should do

logger.error('oops caused by %s', exc)

so the string will only be interpolated if the message is actually emitted.

You can't benefit of this functionality when using .format().


Per the Optimization section of the logging docs:

Formatting of message arguments is deferred until it cannot be avoided. However, computing the arguments passed to the logging method can also be expensive, and you may want to avoid doing it if the logger will just throw away your event.

14
  • 9
    @pfnuesel, .format() is expanded before the call to logger.error, while "lazy interpolation" means expansion is done only if needed (eg the message is actually displayed somewhere)
    – sthenault
    Jan 8, 2016 at 9:43
  • 19
    Is there a good reference for this lazy evaluation being preferred and making a difference? I couldn't find one in PEP282 or the logging library
    – culix
    May 6, 2016 at 14:48
  • 78
    But does this just mean that we will have maintenance problems with our code later? Will we later be "recommended" by pylint to move to the .format() style at some point because logging has upgraded? I ask because I'm more interested in maintainability than I am in bleeding-edge speed performance, at least for most tasks. Jun 20, 2018 at 17:42
  • 4
    @MikeWilliamson : I think this message is a warning because there are possible side effects, but you can safely ignore it. Nov 29, 2018 at 13:38
  • 18
    While much of the motivation behind the warning is around performance (ie: if a log statement is not emitted, then the interpolation cost is saved), it's worth noting that in many (arguably most) applications the performance cost is negligible. See: github.com/PyCQA/pylint/issues/2395 and github.com/PyCQA/pylint/issues/2354 Aug 15, 2019 at 22:01
94

Maybe this time differences can help you.

Following description is not the answer for your question, but it can help people.

If you want to use fstrings (Literal String Interpolation) for logging, then you can disable it from .pylintrc file with disable=logging-fstring-interpolation, see: related issue and comment.

Also you can disable logging-format-interpolation.


For pylint 2.4:

There are 3 options for logging style in the .pylintrc file: old, new, fstr

fstr option added in 2.4 and removed in 2.5

Description from .pylintrc file (v2.4):

[LOGGING]

# Format style used to check logging format string. `old` means using %
# formatting, `new` is for `{}` formatting,and `fstr` is for f-strings.
logging-format-style=old

for old (logging-format-style=old):

foo = "bar"
self.logger.info("foo: %s", foo)

for new (logging-format-style=new):

foo = "bar"
self.logger.info("foo: {}", foo)
# OR
self.logger.info("foo: {foo}", foo=foo)

Note: you can not use .format() even if you select new option.

pylint still gives the same warning for this code:

self.logger.info("foo: {}".format(foo))  # W1202
# OR
self.logger.info("foo: {foo}".format(foo=foo))  # W1202

for fstr (logging-format-style=fstr):

foo = "bar"
self.logger.info(f"foo: {foo}")

Personally, I prefer fstr option because of PEP-0498.

6
  • 9
    you can add "python.linting.pylintArgs": ["--logging-format-style=old"] to the vscode/settings.json file. docs
    – mustafagok
    Jan 7, 2020 at 19:15
  • 4
    in pylint 2.3.1: optparse.OptionValueError: option logging-format-style: invalid value: 'fstr', should be in ['old', 'new'] upgrading to latest pylint (2.4.4) fixed this. Jan 28, 2020 at 5:18
  • I am having following error: Try installing a more recent version of python-pylint, and please open a bug report if the issue persists in t\ he latest release. Thanks!
    – alper
    Jun 5, 2020 at 17:24
  • 14
    update: the fstr logging format option was removed, set disable=logging-fstring-interpolation instead in .pylintrc. github.com/PyCQA/pylint/issues/3570 Aug 14, 2020 at 5:34
  • 2
    A shorter use of f strings is self.logger.info(f"{foo=}"). No need to spell out the variable or expression twice.
    – Tom Ekberg
    Aug 16, 2021 at 20:52
39

In my experience a more compelling reason than optimization (for most use cases) for the lazy interpolation is that it plays nicely with log aggregators like Sentry.

Consider a 'user logged in' log message. If you interpolate the user into the format string, you have as many distinct log messages as there are users. If you use lazy interpolation like this, the log aggregator can more reasonably interpret this as the same log message with a bunch of different instances.

2
  • 6
    yes, this is the strongest argument for the lazy logging arguments formatting Jul 29, 2020 at 14:00
  • This isn't really the answer to the question. I could just as easily do a constant log message with a varying value, such as for JSON logging. @sthenault's answer is much clearer and makes much more sense given the context of the OP's question.
    – Arcsector
    Mar 8, 2023 at 0:41
10

Here is an example of why it's better to use %s instead of f-strings in logging.

>>> import logging
>>> logging.basicConfig(level=logging.INFO)
>>> logger = logging.getLogger('MyLogger')
>>>
>>> class MyClass:
...     def __init__(self, name: str) -> None:
...         self._name = name
...     def __str__(self) -> str:
...         print('GENERATING STRING')
...         return self._name
...
>>> c = MyClass('foo')
>>> logger.debug('Created: %s', c)
>>> logger.debug(f'Created: {c}')
GENERATING STRING

Inspired by Python 3.7 logging: f-strings vs %.

1
  • I put print("######") before the last line and got inspiration! May 24, 2023 at 19:55
2

maybe for someone the solution is disabling in settings.json in Visual Studio Code

 "python.linting.pylintArgs": [
    "--disable=C0111,C0114,C0115,C0116б,W1203",
    "--disable=missing-module-docstring",
    "--disable=missing-class-docstring",
    "--disable=missing-function-docstring",        
]
1

In my case I was getting this error with the PyLint extension in VS Code

Adding these lines to .vscode/settings.json solved the problem

    "pylint.args": [
        "--disable=logging-fstring-interpolation",
        "--disable=logging-format-interpolation",
    ]
0

format is indeed the preferred new style.

You should however not call format() directly, instead letting the interpolation of messages in logging do its job.

You can use the following code to enable using format style with lazy interpolation in python 3.2+

import logging

class BracketStyleRecord(logging.LogRecord):
    def getMessage(self):
        msg = str(self.msg) # see logging cookbook
        if self.args:
            try:
                msg = msg % self.args # retro-compability for 3rd party code
            except TypeError: # not all arguments converted during formatting
                msg = msg.format(*self.args)
        return msg

logging.setLogRecordFactory(BracketStyleRecord)
logging.basicConfig()
logging.error("The first number is %s", 1) # old-style
logging.error("The first number is {}", 1) # new-style

For details, caveats and references see this answer.

-1

"logging-format-interpolation (W1202)" is another one wrong recommendation from pylint (like many from pep8). F-string are described as slow vs %, but have you checked ? With 500_000 rotation of logging with f-string vs % -> f-string:23.01 sec. , %:25.43 sec.

So logging with f-string is faster than %. When you look at the logging source code : log.error() -> self.logger._log() -> self.makeRecord() -> self._logRecordFactory() -> class LogRecord() -> home made equivalent to format()

code :

import logging
import random
import time

loops = 500_000
r_fstr = 0.0
r_format = 0.0

def test_fstr():
    global loops, r_fstr
    for i in range(0, loops):
        r1 = time.time()
        logging.error(f'test {random.randint(0, 1000)}')
        r2 = time.time()
        r_fstr += r2 - r1

def test_format():
    global loops, r_format
    for i in range(0 ,loops):
        r1 = time.time()
        logging.error('test %d', random.randint(0, 1000))
        r2 = time.time()
        r_format += r2 - r1

test_fstr()
test_format()
print(f'Logging f-string:{round(r_fstr,2)} sec. , %:{round(r_format,2)} sec.')
2
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Nov 7, 2022 at 20:28
  • 2
    Pylint is correct. The reason is not performance-related (logging is the slow part, not string interpolation) but about the handling of errors. For example, if the formatting causes an exception when done within the logging code it will be caught and handled rather than causing an exception to be raised in your application code.
    – Tim
    Nov 9, 2022 at 4:58
-3

Might be several years after but having to deal with this the other day, I made simple; just formatted the string before logger.

message = 'message: {}'.format('test')
logger.debug(message)

That way there was no need to change any of the settings from log, if later on desire to change to a normal print there is no need to change the formatting or code.

2
  • 2
    The answer from Tristan Crockett is the best reason NOT to use your method, even if it seems more practical at first. Feb 22, 2022 at 12:26
  • 2
    Solve the question about how to remove the warning. is the logger that is not capable of parse and join the string properly. and should be keep in mind that the programmer who needs set what he wants to log.
    – pelos
    Feb 23, 2022 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.