How do I use dictConfig
? How should I specify its input config
dictionary?
6 Answers
How about here! The corresponding documentation reference is configuration-dictionary-schema
.
LOGGING_CONFIG = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'standard': {
'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
},
},
'handlers': {
'default': {
'level': 'INFO',
'formatter': 'standard',
'class': 'logging.StreamHandler',
'stream': 'ext://sys.stdout', # Default is stderr
},
},
'loggers': {
'': { # root logger
'handlers': ['default'],
'level': 'WARNING',
'propagate': False
},
'my.packg': {
'handlers': ['default'],
'level': 'INFO',
'propagate': False
},
'__main__': { # if __name__ == '__main__'
'handlers': ['default'],
'level': 'DEBUG',
'propagate': False
},
}
}
Usage:
import logging.config
# Run once at startup:
logging.config.dictConfig(LOGGING_CONFIG)
# Include in each module:
log = logging.getLogger(__name__)
log.debug("Logging is configured.")
In case you see too many logs from third-party packages, be sure to run this config using logging.config.dictConfig(LOGGING_CONFIG)
before the third-party packages are imported.
To add additional custom info to each log message using a logging filter, consider this answer.
-
15
-
3All those concise, beautiful YAML snippets in the python logging.config docs just can't be read directly. Bummer.– JimBMay 17, 2018 at 18:19
-
Isn't this django-specific? What if I'm using a different framework (Flask, Bottle, etc), or not even working on a web application? Dec 18, 2018 at 21:04
-
It feels like a cheat with
'disable_existing_loggers': False
as then you're maybe not configuring it whole cloth, but maybe reusing something that's already there.. If you set it toTrue
then I don't seem to get any output.– Nick TApr 12, 2019 at 23:19 -
1How about the recoverable errors? Errors should not be in the stdout. It's only for program output. May 19, 2020 at 13:42
The accepted answer is nice! But what if one could begin with something less complex? The logging module is very powerful thing and the documentation is kind of a little bit overwhelming especially for novice. But for the beginning you don't need to configure formatters and handlers. You can add it when you figure out what you want.
For example:
import logging.config
DEFAULT_LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'loggers': {
'': {
'level': 'INFO',
},
'another.module': {
'level': 'DEBUG',
},
}
}
logging.config.dictConfig(DEFAULT_LOGGING)
logging.info('Hello, log')
-
2This is the more relevant / useful example, at least in my case. It was the final
logging.info('Hello, log')
that made things click for me. The confusion in the documentation is that with dictConfig we no longer need to performgetLogger
or any of those actions. Jun 19, 2018 at 17:11 -
1@theotheo Can you explain the empty key
'': { 'level': 'INFO'...
and why it doesnt work without it (e.g. when changing the blank value to a valid value such asstandard
Oct 1, 2018 at 16:10 -
1@MikeWilliamson: It can be useful, though, to still call
getLogger()
if you want multiple loggers with different names. Each of these loggers inherits configuration from the root logger. Nov 12, 2018 at 7:40 -
4@MikeWilliamson
getLogger
is always optional. When using thelogging.info()
method directly the root logger is used, while withgetLogger()
you can have different loggers, with differents names and levels. Mar 5, 2019 at 10:54
Example with Stream Handler, File Handler, Rotating File Handler and SMTP Handler
from logging.config import dictConfig
LOGGING_CONFIG = {
'version': 1,
'loggers': {
'': { # root logger
'level': 'NOTSET',
'handlers': ['debug_console_handler', 'info_rotating_file_handler', 'error_file_handler', 'critical_mail_handler'],
},
'my.package': {
'level': 'WARNING',
'propagate': False,
'handlers': ['info_rotating_file_handler', 'error_file_handler' ],
},
},
'handlers': {
'debug_console_handler': {
'level': 'DEBUG',
'formatter': 'info',
'class': 'logging.StreamHandler',
'stream': 'ext://sys.stdout',
},
'info_rotating_file_handler': {
'level': 'INFO',
'formatter': 'info',
'class': 'logging.handlers.RotatingFileHandler',
'filename': 'info.log',
'mode': 'a',
'maxBytes': 1048576,
'backupCount': 10
},
'error_file_handler': {
'level': 'WARNING',
'formatter': 'error',
'class': 'logging.FileHandler',
'filename': 'error.log',
'mode': 'a',
},
'critical_mail_handler': {
'level': 'CRITICAL',
'formatter': 'error',
'class': 'logging.handlers.SMTPHandler',
'mailhost' : 'localhost',
'fromaddr': '[email protected]',
'toaddrs': ['[email protected]', '[email protected]'],
'subject': 'Critical error with application name'
}
},
'formatters': {
'info': {
'format': '%(asctime)s-%(levelname)s-%(name)s::%(module)s|%(lineno)s:: %(message)s'
},
'error': {
'format': '%(asctime)s-%(levelname)s-%(name)s-%(process)d::%(module)s|%(lineno)s:: %(message)s'
},
},
}
dictConfig(LOGGING_CONFIG)
There's an updated example of declaring a logging.config.dictConfig() dictionary schema buried in the logging cookbook examples. Scroll up from that cookbook link to see a use of dictConfig().
Here's an example use case for logging to both stdout and a "logs" subdirectory using a StreamHandler and RotatingFileHandler
with customized format
and datefmt
.
Imports modules and establish a cross-platform absolute path to the "logs" subdirectory
from os.path import abspath, dirname, join import logging from logging.config import dictConfig base_dir = abspath(dirname(__file__)) logs_target = join(base_dir + "\logs", "python_logs.log")
Establish the schema according to the dictionary schema documentation.
logging_schema = { # Always 1. Schema versioning may be added in a future release of logging "version": 1, # "Name of formatter" : {Formatter Config Dict} "formatters": { # Formatter Name "standard": { # class is always "logging.Formatter" "class": "logging.Formatter", # Optional: logging output format "format": "%(asctime)s\t%(levelname)s\t%(filename)s\t%(message)s", # Optional: asctime format "datefmt": "%d %b %y %H:%M:%S" } }, # Handlers use the formatter names declared above "handlers": { # Name of handler "console": { # The class of logger. A mixture of logging.config.dictConfig() and # logger class-specific keyword arguments (kwargs) are passed in here. "class": "logging.StreamHandler", # This is the formatter name declared above "formatter": "standard", "level": "INFO", # The default is stderr "stream": "ext://sys.stdout" }, # Same as the StreamHandler example above, but with different # handler-specific kwargs. "file": { "class": "logging.handlers.RotatingFileHandler", "formatter": "standard", "level": "INFO", "filename": logs_target, "mode": "a", "encoding": "utf-8", "maxBytes": 500000, "backupCount": 4 } }, # Loggers use the handler names declared above "loggers" : { "__main__": { # if __name__ == "__main__" # Use a list even if one handler is used "handlers": ["console", "file"], "level": "INFO", "propagate": False } }, # Just a standalone kwarg for the root logger "root" : { "level": "INFO", "handlers": ["file"] } }
Configure
logging
with the dictionary schemadictConfig(logging_schema)
Try some test cases to see if everything is working properly
if __name__ == "__main__": logging.info("testing an info log entry") logging.warning("testing a warning log entry")
[EDIT to answer @baxx's question]
To reuse this setting across your code base, instantiate a logger in the script you call dictConfig() and then import that logger elsewhere
# my_module/config/my_config.py dictConfig(logging_schema) my_logger = getLogger(__name__)
Then in another script
from my_module.config.my_config import my_logger as logger
logger.info("Hello world!")
-
1how would this work if one wanted to use the schema across multiple modules? Here it's declared in the module in which it's used I think ?– baxxAug 2, 2021 at 15:11
-
I may suggest minor conciseness edit:
from my_module.config.my_config import log
andlog = getLogger(__name__)
– AntonJan 3 at 2:44 -
Maybe I misunderstood this, but at which point the
__main__
logger will be used? I use your extension to provide the logger across my codebase. The logger is created at package base in my__init__.py
– MaKaNuMay 25 at 15:11
I found Django v1.11.15 default config below, hope it helps
DEFAULT_LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse',
},
'require_debug_true': {
'()': 'django.utils.log.RequireDebugTrue',
},
},
'formatters': {
'django.server': {
'()': 'django.utils.log.ServerFormatter',
'format': '[%(server_time)s] %(message)s',
}
},
'handlers': {
'console': {
'level': 'INFO',
'filters': ['require_debug_true'],
'class': 'logging.StreamHandler',
},
'django.server': {
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'django.server',
},
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'django': {
'handlers': ['console', 'mail_admins'],
'level': 'INFO',
},
'django.server': {
'handlers': ['django.server'],
'level': 'INFO',
'propagate': False,
},
}
}
-
4This example is fine, but I think to stand out beyond the accepted answer, some explanation would help. Nov 13, 2018 at 2:17
One more thing in case it's useful to start from the existing logger's config, the current config dictionary is can be obtained via
import logging
logger = logging.getLogger()
current_config = logger.__dict__ # <-- yes, it's just the dict
print(current_config)
It'll be something like:
{'filters': [], 'name': 'root', 'level': 30, 'parent': None, 'propagate': True, 'handlers': [], 'disabled': False, '_cache': {}}
Then, if you just do
new_config=current_config
new_config['version']=1
new_config['name']='fubar'
new_config['level']=20
# ...and whatever other changes you wish
logging.config.dictConfig(new_config)
You will then find:
print(logger.__dict__)
is what you'd hope for
{'filters': [], 'name': 'fubar', 'level': 20, 'parent': None, 'propagate': True, 'handlers': [], 'disabled': False, '_cache': {}, 'version': 1}