79

I have the following lines of code that initialize logging. I comment one out and leave the other to be used. The problem that I'm facing is that the one that is meant to log to the file not logging to file. It is instead logging to the console. Please help.

For logging to Console:

logging.basicConfig(level=logging.INFO,
        format='%(asctime)s [%(levelname)s] (%(threadName)-10s) %(message)s',)

for file logging

logging.basicConfig(filename='server-soap.1.log',level=logging.INFO,
        format='%(asctime)s [%(levelname)s] (%(threadName)-10s) %(message)s')
2
  • your file logging example works fine for me. is the file writable? do you call your config and log calls in the right order?
    – Gryphius
    Commented Nov 27, 2013 at 11:01
  • Thanks Gryphius. Yes, the definition was ok. But I found that the problem was regarding the ordering of my imports and logging. I needed to define the logging before importing the various libraries.
    – Phil
    Commented Nov 29, 2013 at 7:39

7 Answers 7

134

I found out what the problem was. It was in the ordering of the imports and the logging definition.

The effect of the poor ordering was that the libraries that I imported before defining the logging using logging.basicConfig() defined the logging. This therefore took precedence to the logging that I was trying to define later using logging.basicConfig()

Below is how I needed to order it:

import logging
## for file logging
logging.basicConfig(filename='server-soap.1.log',
        level=logging.INFO,
        format='%(asctime)s %(levelname)s %(threadName)-10s %(message)s',)

from pysimplesoap.server import SoapDispatcher, SOAPHandler
from BaseHTTPServer import HTTPServer
import time,random,datetime,pytz,sys,threading
from datetime import timedelta
#DB
import psycopg2, psycopg2.extras
from psycopg2.pool import ThreadedConnectionPool

#ESB Call
from suds import WebFault
from suds.client import Client

But the faulty ordering that I initially had was:

from pysimplesoap.server import SoapDispatcher, SOAPHandler
from BaseHTTPServer import HTTPServer
import logging
import time,random,datetime,pytz,sys,threading
from datetime import timedelta
#DB
import psycopg2, psycopg2.extras
from psycopg2.pool import ThreadedConnectionPool

#ESB Call
from suds import WebFault
from suds.client import Client

## for file logging

logging.basicConfig(filename='server-soap.1.log',
        level=logging.INFO,
        format='%(asctime)s %(levelname)s %(threadName)-10s %(message)s',)
10
  • 9
    Tricky one. This drove me crazy!
    – Eike P.
    Commented May 28, 2016 at 13:19
  • 4
    OMG! dude I needed this. I've spent days trying to solve this problem. Thank you so much. Commented Dec 6, 2017 at 15:22
  • 1
    Drove me crazy too. I find some comment of logging.basicConfig, it says:This function does nothing if the root logger already has handlers configured. It is a convenience method intended for use by simple scripts to do one-shot configuration of the logging package.
    – g10guang
    Commented Mar 11, 2018 at 1:12
  • 2
    why is this the case tho.
    – kgui
    Commented May 24, 2018 at 14:28
  • 1
    @Phil I put a minus because the ordering of imports is not a problem here. The problem is that basicConfig should be called in your program not in libraries Commented Aug 15, 2019 at 10:36
46

"Changed in version 3.8: The force argument was added." I think it's a better choice for new version.

For older Version(< 3.8):

From the source code of logging I found the flows:

This function does nothing if the root logger already has handlers
configured. It is a convenience method intended for use by simple scripts
to do one-shot configuration of the logging package.

So, if some module we import called the basicConfig() method before us, our call will do nothing.

A solution I found can work is that you can reload logging before your own calling to basicConfig(), such as

def init_logger(*, fn=None):

    # !!! here
    from imp import reload # python 2.x don't need to import reload, use it directly
    reload(logging)

    logging_params = {
        'level': logging.INFO,
        'format': '%(asctime)s__[%(levelname)s, %(module)s.%(funcName)s](%(name)s)__[L%(lineno)d] %(message)s',
    }

    if fn is not None:
        logging_params['filename'] = fn

    logging.basicConfig(**logging_params)
    logging.error('init basic configure of logging success')
2
21

I got the same error, I fixed it by passing the following argument to the basic config.

logging.basicConfig(
    level="WARNING",
    format="%(asctime)s - %(name)s - [ %(message)s ]",
    datefmt='%d-%b-%y %H:%M:%S',
    force=True,
    handlers=[
        logging.FileHandler("debug.log"),
        logging.StreamHandler()
    ])

Here as you can see passing force=True overrides any other BasicConfigs

7
  • 3
    "Changed in version 3.8: The force argument was added." I think it's a better choice for new version
    – vacing
    Commented Sep 14, 2020 at 1:52
  • 1
    Finally, after much head-scratching, this worked! I was looking for a quicker/simpler solution than having to instantiate with logger = logging.getLogger('abc').
    – vk1011
    Commented Oct 28, 2021 at 21:12
  • 2
    This should be marked as the accepted answer. Thanks so much, adding force=True did the trick.
    – Parker
    Commented Jul 13, 2022 at 22:56
  • If you say force =True, how does this change affect other libraries that use the logging module? Commented Feb 16, 2023 at 9:37
  • 1
    @AlenPaulVarghese this means whatever those libraries will be logging will end up in my debug.log file. This process does not sit well with me Commented Mar 7, 2023 at 10:12
16

In case basicConfig() does not work:

logger = logging.getLogger('Spam Logger')
logger.setLevel(logging.DEBUG)
# create file handler which logs even debug messages
fh = logging.FileHandler('spam.log')
fh.setLevel(logging.DEBUG)
# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
fh.setFormatter(formatter)
# add the handlers to logger
logger.addHandler(ch)
logger.addHandler(fh)

# 'application' code
logger.debug('debug Spam message')
logging.debug('debug Spam message')
logger.info('info Ham message')
logger.warning('warn Eggs message')
logger.error('error Spam and Ham message')
logger.critical('critical Ham and Eggs message')

which gives me the following output:

2019-06-20 11:33:48,967 - Spam Logger - DEBUG - debug Spam message
2019-06-20 11:33:48,968 - Spam Logger - INFO - info Ham message
2019-06-20 11:33:48,968 - Spam Logger - WARNING - warn Eggs message
2019-06-20 11:33:48,968 - Spam Logger - ERROR - error Spam and Ham message
2019-06-20 11:33:48,968 - Spam Logger - CRITICAL - critical Ham and Eggs message

For the sake of reference, Python Logging Cookbook is readworthy.

3
  • 4
    Just want to emphasize that the logger.setLevel() is crucial, because this cost me an hour as I did not notice in the answer; ie even if your handlers have DEBUG level, the default for logger is WARN so your INFO and DEBUG won't come through without the logger.setLevel().
    – Oliver
    Commented Oct 12, 2019 at 21:12
  • Seems least "hackish" from provided answers.
    – Dolfa
    Commented Mar 20, 2020 at 11:31
  • ... which does not set the root logger while basicConfig() does.. Downvote.
    – Evan
    Commented Jul 15 at 11:34
1

Another solution that worked for me is instead of tracing down which module might be importing logging or even calling basicConfig before me is to just call setLevel after basicConfig again.

import os
import logging

RUNTIME_DEBUG_LEVEL = os.environ.get('RUNTIME_DEBUG_LEVEL').upper()
LOGGING_KWARGS = {
    'level': getattr(logging, RUNTIME_DEBUG_LEVEL)
}

logging.basicConfig(**LOGGING_KWARGS)
logging.setLevel(getattr(logging, RUNTIME_DEBUG_LEVEL))

Sort of crude, seems hacky, fixed my problem, worth a share.

2
0

Like vacing's answer mentioned, basicConfig has no effect if the root logger already has handlers configured.

I was using pytest which seems to set handlers which means the default logging setup with loglevel WARNING is active -- so it appears my app fails to log, but this only happens when executing unit tests with pytest. In a normal app run logs are produced as expected which is enough for my use case.

-1

IF YOU JUST WANT TO SET THE LOG LEVEL OF ALL LOGGERS


instead of ordering your imports after logging config:

just set level on the root level:

# Option 1:
logging.root.setLevel(logging.INFO)

# Option 2 - make it configurable:
# env variable + default value INFO
logging.root.setLevel(os.environ.get('LOG_LEVEL', logging.INFO))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.