Python logging
last modified November 16, 2026 Python 3.15
In this article we show how to do logging in Python with the
logging module.
Logging is the process of recording information about events that
happen while a program runs. Log records help with troubleshooting, auditing,
monitoring, generating statistics, and finding performance bottlenecks. Unlike
print calls, log messages carry a severity level, can be turned on
and off without changing the code, and can be sent to the console, files, or
remote services at the same time.
What to log
Good candidates for logging are application start-ups and shut-downs, configuration changes, input validation failures, authentication and authorization failures, and errors. Log messages should be meaningful, contain enough context to understand the event, and be readable by people as well as parseable by tools.
Some data must never be written to logs: passwords, access tokens, session identifiers, encryption keys, database connection strings, payment card data, and sensitive personal data.
The logging module
The logging module of the standard library implements a flexible
event logging system for applications and libraries. It has four main
components:
- loggers expose the interface that the application code calls,
- handlers send log records to destinations such as the console or a file,
- filters decide which records are output and can add data to them,
- formatters specify the layout of log records in the final output.
Each message has a level that indicates its severity. Loggers and handlers have a threshold level; records below it are ignored.
| Level | Value | Used for |
|---|---|---|
CRITICAL | 50 | a serious error; the program may not be able to continue |
ERROR | 40 | a failure of some operation |
WARNING | 30 | something unexpected that does not stop the program |
INFO | 20 | confirmation that things work as expected |
DEBUG | 10 | detailed information for diagnosing problems |
NOTSET | 0 | no level set; the level is taken from the parent logger |
If the threshold is WARNING, the WARNING,
ERROR, and CRITICAL messages are logged. If it is
ERROR, only ERROR and CRITICAL messages
pass.
Simple example
The module-level functions debug, info,
warning, error, and critical can be used
right away without any configuration. They log through the root
logger.
import logging
logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')
The example calls the five functions. If the root logger has no handlers, the
functions call basicConfig first, which adds a handler that
writes to the standard error output in the
LEVEL:logger:message format.
Only three messages are written. The root logger has the WARNING
level by default, so the debug and info messages are dropped.
$ python simple.py WARNING:root:This is a warning message ERROR:root:This is an error message CRITICAL:root:This is a critical message
Configuring with basicConfig
The basicConfig function does basic configuration of the root
logger. It sets the level and the format and creates a handler. It has an
effect only the first time it is called (unless force=True is
passed), so it should be called once, early in the program.
import logging
logging.basicConfig(level=logging.DEBUG,
format='%(levelname)-8s %(name)s: %(message)s')
logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
We lower the threshold to DEBUG, so all messages are written. The
format string uses the levelname, name, and
message attributes of a log record; -8 pads the level
name to eight characters, which aligns the messages.
$ python basic_config.py DEBUG root: This is a debug message INFO root: This is an info message WARNING root: This is a warning message
The last resort handler
Creating a logger and setting its level is not enough to see all its messages.
A logger needs a handler. When no handler is found in the logger or its
ancestors, Python uses a last resort handler, which prints bare
messages of level WARNING and higher to standard error.
import logging
logger = logging.getLogger('dev')
logger.setLevel(logging.DEBUG)
logger.debug('This is a debug message')
logger.info('This is an info message')
logger.warning('This is a warning message')
logger.error('This is an error message')
print(logger.handlers, logging.getLogger().handlers)
Although the logger level is DEBUG, the debug and info messages
are not shown. Neither the dev logger nor the root logger has a
handler, as the empty lists show, so the warning and the error are printed by
the last resort handler without any formatting. Configuring the root logger
with basicConfig, or adding a handler, fixes this.
$ python last_resort.py This is a warning message This is an error message [] []
Named loggers
The getLogger function returns a logger with the given name.
Calls with the same name return the same logger object, so loggers never have
to be passed around. A common practice is to create one logger per module,
named after the module with __name__.
import logging
logger = logging.getLogger(__name__)
def divide(a, b):
logger.debug('dividing %d by %d', a, b)
return a / b
logging.basicConfig(level=logging.DEBUG,
format='%(levelname)s %(name)s %(funcName)s: %(message)s')
logger.info('starting')
print(divide(10, 4))
print(logging.getLogger(__name__) is logger)
The logger is created at module level and used in a function. In a script run
directly, __name__ is __main__; in an imported module,
it would be the module's dotted name. The funcName attribute
shows the function that issued the message.
The module only creates the logger; the configuration is done by
basicConfig in the main part of the program. Our logger has no
handler and no level of its own, so its messages propagate to the root logger,
which handles them.
$ python named_logger.py INFO __main__ <module>: starting DEBUG __main__ divide: dividing 10 by 4 2.5 True
Handlers
A handler dispatches log records to a destination. A logger can have several
handlers, each with its own level and formatter. The
StreamHandler writes to a stream (standard error by default) and
the FileHandler writes to a file.
import logging
logger = logging.getLogger('app')
logger.setLevel(logging.DEBUG)
console = logging.StreamHandler()
console.setLevel(logging.WARNING)
file_handler = logging.FileHandler('app.log', mode='w', encoding='utf-8')
file_handler.setLevel(logging.DEBUG)
fmt = logging.Formatter('%(levelname)s: %(message)s')
console.setFormatter(fmt)
file_handler.setFormatter(fmt)
logger.addHandler(console)
logger.addHandler(file_handler)
logger.debug('connecting to database')
logger.info('connection established')
logger.warning('disk usage at 91%')
logger.error('query failed')
The logger lets all messages through, as its level is DEBUG. The
handlers then apply their own thresholds: the console shows only warnings and
errors, while the file receives everything.
file_handler = logging.FileHandler('app.log', mode='w', encoding='utf-8')
By default, a FileHandler appends to the file. With
mode='w', the file is truncated each time the program starts. It
is good practice to specify the encoding explicitly.
$ python handlers.py WARNING: disk usage at 91% ERROR: query failed $ cat app.log DEBUG: connecting to database INFO: connection established WARNING: disk usage at 91% ERROR: query failed
Formatters
A formatter defines the layout of the final message. Besides the message text, it can include the time, the logger name, the level, the source file and line, the process and thread, and other LogRecord attributes.
import logging
logger = logging.getLogger('dev')
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
formatter = logging.Formatter(
'%(asctime)s %(name)s %(levelname)s [%(filename)s:%(lineno)d] %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.info('information message')
formatter = logging.Formatter('{asctime} | {levelname:<7} | {message}',
datefmt='%Y-%m-%d %H:%M:%S', style='{')
handler.setFormatter(formatter)
logger.info('information message')
logger.warning('warning message')
The first formatter uses the default percent style. The asctime
attribute is the time of the record, filename and
lineno point to the call in the source code.
formatter = logging.Formatter('{asctime} | {levelname:<7} | {message}',
datefmt='%Y-%m-%d %H:%M:%S', style='{')
With style='{', the format string uses the
str.format syntax, including its format specifications. The
datefmt parameter formats the time with strftime
directives; here we drop the milliseconds.
$ python formatter.py 2026-11-16 23:01:15,568 dev INFO [formatter.py:12] information message 2026-11-16 23:01:15 | INFO | information message 2026-11-16 23:01:15 | WARNING | warning message
Logger hierarchy
Loggers form a hierarchy based on their dotted names. A logger named
app is the parent of app.db, and the root logger is
the ancestor of all loggers.
A logger with no level set (NOTSET) uses the level of the nearest
ancestor that has one; this is its effective level. After a logger
handles a record, it passes the record to the handlers of its ancestors. This
is called propagation. Thanks to it, it is usually enough to
configure handlers on a top-level logger.
import logging
logging.basicConfig(format='%(name)-10s %(levelname)-8s %(message)s')
app = logging.getLogger('app')
app.setLevel(logging.INFO)
db = logging.getLogger('app.db')
api = logging.getLogger('app.api')
api.setLevel(logging.ERROR)
print(db.parent.name, app.parent.name)
print(app.getEffectiveLevel(), db.getEffectiveLevel(), api.getEffectiveLevel())
db.info('connected')
api.info('request received')
api.error('request failed')
db.propagate = False
db.warning('no handler found, last resort used')
print(db.handlers, db.propagate)
We create the app logger with the INFO level and two
children. The app.db logger has no level, so its effective level
is 20, inherited from app. The app.api logger has its
own ERROR level and drops the info message.
None of the three loggers has a handler. Their messages propagate up to the
root logger, whose handler was created by basicConfig.
db.propagate = False
When propagation is turned off, records stop at the app.db
logger. Since it has no handler, the last resort handler prints the warning
without formatting.
$ python hierarchy.py app root 20 20 40 app.db INFO connected app.api ERROR request failed no handler found, last resort used [] False
Logging variable data
Variable data is passed to a logging call as extra arguments, with
% placeholders in the message. The message is formatted only if
the record is actually emitted.
import logging
logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s')
logger = logging.getLogger('auth')
class Expensive:
def __str__(self):
print(' (str() called)')
return 'expensive object'
user = 'jsmith'
attempts = 3
logger.warning('authentication failed for %s after %d attempts', user, attempts)
logger.info('%(user)s logged in from %(ip)s', {'user': user, 'ip': '10.0.0.7'})
logger.debug('lazy: %s', Expensive())
logger.debug(f'eager: {Expensive()}')
The first call fills in a string and an integer. A single dictionary argument can be used with named placeholders.
logger.debug('lazy: %s', Expensive())
logger.debug(f'eager: {Expensive()}')
The Expensive class announces when it is converted to a string.
Both debug messages are below the INFO threshold and are not
logged. With the placeholder style, the conversion never happens. The
f-string is evaluated before debug is called, so the work is done
in vain. Keeping the message template constant also helps tools that group
log messages.
$ python log_variables.py WARNING: authentication failed for jsmith after 3 attempts INFO: jsmith logged in from 10.0.0.7 (str() called)
Logging exceptions
The exception method logs a message with the ERROR
level and adds the traceback of the exception being handled. It should be
called only from an exception handler. Other methods accept
exc_info=True to include the traceback.
import logging
logging.basicConfig(format='%(levelname)s: %(message)s')
logger = logging.getLogger('calc')
vals = [1, 2]
try:
print(vals[4])
except IndexError:
logger.exception('cannot read the value')
try:
result = 1 / 0
except ZeroDivisionError as e:
logger.error('division failed: %s', e)
In the first handler, we log the traceback of the IndexError. In
the second one, we log only the error message, which is enough when the
location of the problem is clear.
$ python exceptions.py
ERROR: cannot read the value
Traceback (most recent call last):
File "/home/user/python/exceptions.py", line 9, in <module>
print(vals[4])
~~~~^^^
IndexError: list index out of range
ERROR: division failed: division by zero
Logging to a file
With the filename parameter, basicConfig creates a
file handler instead of a stream handler.
import logging
logging.basicConfig(filename='test.log', filemode='w', encoding='utf-8',
level=logging.DEBUG,
format='%(asctime)s %(filename)s: %(message)s',
datefmt='%Y-%m-%d %H:%M:%S')
logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('Čerešňa is not a valid product code')
The filemode parameter is the file open mode; the default is
'a'. The encoding parameter is important when the
messages contain non-ASCII characters. The datefmt parameter
formats the time. The program writes nothing to the console.
$ python log_file.py $ cat test.log 2026-11-16 23:01:15 log_file.py: This is a debug message 2026-11-16 23:01:15 log_file.py: This is an info message 2026-11-16 23:01:15 log_file.py: Čerešňa is not a valid product code
Contextual information
The extra parameter adds custom attributes to a log record, which
the formatter can then display. A LoggerAdapter adds the same
attributes to every call automatically.
import logging
logging.basicConfig(level=logging.INFO,
format='%(levelname)s [%(request_id)s] %(message)s')
logger = logging.getLogger('web')
logger.info('request started', extra={'request_id': 'a1f3'})
adapter = logging.LoggerAdapter(logger, {'request_id': 'b72c'})
adapter.info('request started')
adapter.warning('slow query: %d ms', 1840)
The format string refers to the custom request_id attribute. In
the first call, we pass it with extra. The adapter wraps the
logger and supplies the attribute itself, which is convenient when many
messages share the same context. Note that every record reaching this
formatter must have the attribute, otherwise formatting fails.
$ python extra_context.py INFO [a1f3] request started INFO [b72c] request started WARNING [b72c] slow query: 1840 ms
Filters
A filter decides whether a record is logged. It can also modify the record,
for example add an attribute. A filter is either a logging.Filter
subclass or any callable that takes a record and returns a true or false
value.
import logging
class HostFilter(logging.Filter):
def __init__(self, hostname):
super().__init__()
self.hostname = hostname
def filter(self, record):
record.hostname = self.hostname
return True
def no_passwords(record):
return 'password' not in record.getMessage().lower()
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter('%(hostname)s %(levelname)s: %(message)s'))
handler.addFilter(HostFilter('web01'))
handler.addFilter(no_passwords)
logger = logging.getLogger('app')
logger.addHandler(handler)
logger.setLevel(logging.INFO)
logger.info('user jsmith logged in')
logger.info('password for %s changed to %s', 'jsmith', 'abc123')
logger.info('user jsmith logged out')
The HostFilter adds a hostname attribute to every
record and lets it through. The no_passwords function rejects
records whose message mentions a password. The getMessage method
returns the message with the arguments merged in. Both filters are attached
to the handler.
$ python filters.py web01 INFO: user jsmith logged in web01 INFO: user jsmith logged out
Rotating log files
Log files of long-running programs grow without limit. The
RotatingFileHandler from the logging.handlers module
starts a new file when the current one reaches a size limit and keeps a given
number of old files.
import logging
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler('rotating.log', maxBytes=100, backupCount=2)
handler.setFormatter(logging.Formatter('%(levelname)s %(message)s'))
logger = logging.getLogger('rot')
logger.setLevel(logging.INFO)
logger.addHandler(handler)
for i in range(1, 13):
logger.info('message number %02d', i)
To see the rotation, we set an unrealistically small limit of 100 bytes. When
the limit would be exceeded, rotating.log is renamed to
rotating.log.1, the previous .1 file becomes
.2, and so on; files beyond backupCount are deleted.
The oldest messages, 1 to 4, are in the last backup and the newest in the
current file. The TimedRotatingFileHandler rotates by time
instead.
$ python rotating.py $ ls -1 rotating.log* rotating.log rotating.log.1 rotating.log.2 $ cat rotating.log.2 INFO message number 01 INFO message number 02 INFO message number 03 INFO message number 04 $ cat rotating.log INFO message number 09 INFO message number 10 INFO message number 11 INFO message number 12
Configuration with dictConfig
Larger applications configure logging declaratively. The
logging.config.dictConfig function takes a dictionary that
describes formatters, handlers, and loggers. This is the recommended way to
configure logging; the older fileConfig uses an INI file and
supports fewer features.
import logging
import logging.config
config = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'simple': {'format': '%(levelname)s %(name)s: %(message)s'},
'detailed': {
'format': '%(asctime)s %(levelname)s %(name)s %(lineno)d: %(message)s',
'datefmt': '%H:%M:%S',
},
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'level': 'INFO',
'formatter': 'simple',
'stream': 'ext://sys.stdout',
},
'file': {
'class': 'logging.FileHandler',
'level': 'DEBUG',
'formatter': 'detailed',
'filename': 'dict.log',
'mode': 'w',
},
},
'loggers': {
'shop': {'level': 'DEBUG', 'handlers': ['console', 'file'], 'propagate': False},
},
'root': {'level': 'WARNING', 'handlers': ['console']},
}
logging.config.dictConfig(config)
logger = logging.getLogger('shop.cart')
logger.debug('cart created')
logger.info('item added')
logging.getLogger('other').warning('root handles this')
print(sorted(logging.getHandlerNames()))
The configuration defines two formatters and two handlers. The
ext://sys.stdout string refers to an object outside the
configuration. The shop logger sends info messages to the console
and debug messages to the file; 'propagate': False stops its
records from being handled again by the root logger.
The disable_existing_loggers key defaults to True,
which silences loggers created before the configuration, such as module-level
loggers of already imported modules. Setting it to False is
usually what we want. The getHandlerNames function, added in
Python 3.12, returns the names of the configured handlers.
$ python dict_config.py INFO shop.cart: item added WARNING other: root handles this ['console', 'file'] $ cat dict.log 23:01:15 DEBUG shop.cart 38: cart created 23:01:15 INFO shop.cart 39: item added
Configuration in YAML
The dictionary for dictConfig is often stored in a YAML file.
YAML is a human-readable data serialization language commonly used for
configuration. To read it, we need the pyyaml package.
$ pip install pyyaml
Loading the YAML configuration
The YAML file mirrors the structure of the configuration dictionary.
version: 1
disable_existing_loggers: false
formatters:
simple:
format: "%(name)s: %(message)s"
extended:
format: "%(asctime)s %(name)s %(levelname)s: %(message)s"
datefmt: "%Y-%m-%d %H:%M:%S"
handlers:
console:
class: logging.StreamHandler
level: INFO
formatter: simple
stream: ext://sys.stdout
file_handler:
class: logging.FileHandler
level: DEBUG
filename: test.log
mode: w
formatter: extended
loggers:
dev:
level: DEBUG
handlers: [console, file_handler]
propagate: false
root:
level: WARNING
handlers: [file_handler]
The dev logger writes simple messages to the console and
detailed ones to test.log. The propagate option
belongs to a logger, not a handler; set to false, it prevents the
dev records from reaching the root logger's file handler and
being written twice.
import logging
import logging.config
import yaml
with open('config.yaml', encoding='utf-8') as f:
log_cfg = yaml.safe_load(f)
logging.config.dictConfig(log_cfg)
logger = logging.getLogger('dev')
logger.debug('This is a debug message')
logger.info('This is an info message')
logger.error('This is an error message')
logging.getLogger('legacy').warning('This goes to the root logger')
We load the file with yaml.safe_load and pass the resulting
dictionary to dictConfig. The legacy logger is not
configured, so its warning propagates to the root logger and ends up in the
file.
$ python log_yaml.py dev: This is an info message dev: This is an error message $ cat test.log 2026-11-16 23:01:15 dev DEBUG: This is a debug message 2026-11-16 23:01:15 dev INFO: This is an info message 2026-11-16 23:01:15 dev ERROR: This is an error message 2026-11-16 23:01:15 legacy WARNING: This goes to the root logger
JSON log output
Log aggregation systems prefer structured data. A custom formatter can output each record as a JSON object on one line.
import json
import logging
from datetime import datetime, timezone
class JsonFormatter(logging.Formatter):
def format(self, record):
entry = {
'time': datetime.fromtimestamp(record.created, timezone.utc)
.isoformat(timespec='seconds'),
'level': record.levelname,
'logger': record.name,
'message': record.getMessage(),
}
if record.exc_info:
entry['exception'] = self.formatException(record.exc_info)
return json.dumps(entry)
handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logger = logging.getLogger('api')
logger.addHandler(handler)
logger.setLevel(logging.INFO)
logger.info('order %d shipped', 1042)
try:
int('abc')
except ValueError:
logger.exception('bad input')
The JsonFormatter overrides the format method. The
created attribute is the record time as a timestamp; we convert it
to an ISO 8601 string in UTC. If the record carries exception information,
formatException renders the traceback, and json.dumps
escapes its newlines, so each record stays on a single line.
$ python json_logs.py
{"time": "2026-11-16T22:01:15+00:00", "level": "INFO", "logger": "api", "message": "order 1042 shipped"}
{"time": "2026-11-16T22:01:15+00:00", "level": "ERROR", "logger": "api", "message": "bad input", "exception": "Traceback (most recent call last):\n File \"/home/user/python/json_logs.py\", line 30, in <module>\n int('abc')\n ~~~^^^^^^^\nValueError: invalid literal for int() with base 10: 'abc'"}
Level names
The getLevelNamesMapping function, added in Python 3.11, returns
the mapping of level names to their numeric values.
import logging
for name, value in logging.getLevelNamesMapping().items():
print(f'{name:<8} {value:>2}')
print(logging.getLevelName(30), logging.getLevelName('ERROR'))
The mapping also contains the FATAL and WARN
aliases. The getLevelName function converts in both directions,
from a number to a name and from a name to a number. Level names can also be
passed directly to setLevel, for example
logger.setLevel('DEBUG').
$ python levels.py CRITICAL 50 FATAL 50 ERROR 40 WARN 30 WARNING 30 INFO 20 DEBUG 10 NOTSET 0 WARNING 40
Logging in libraries
A library should create loggers and log messages but leave the configuration
to the application. To avoid output from the last resort handler when the
application does not configure logging, a library adds a
NullHandler to its top-level logger.
import logging
logger = logging.getLogger('mylib')
logger.addHandler(logging.NullHandler())
def fetch():
logger.warning('using cached data')
return [1, 2, 3]
print(fetch())
logging.basicConfig(format='%(levelname)s %(name)s: %(message)s')
print(fetch())
The first call of fetch logs a warning, but the
NullHandler counts as a handler and discards it, so nothing is
printed. After the application configures logging with
basicConfig, the library's messages propagate to the root logger
and appear in the output.
$ python library.py [1, 2, 3] WARNING mylib: using cached data [1, 2, 3]
Source
logging – Python documentation
Logging HOWTO – Python documentation
Logging cookbook – Python documentation
In this article we have worked with the Python logging module. See also Python YAML and Python exceptions.
Author
List all Python tutorials.