ZetCode

Python logging

last modified November 16, 2026 Python 3.15

In this article we show how to do logging in Python with the logging module.

Logging is the process of recording information about events that happen while a program runs. Log records help with troubleshooting, auditing, monitoring, generating statistics, and finding performance bottlenecks. Unlike print calls, log messages carry a severity level, can be turned on and off without changing the code, and can be sent to the console, files, or remote services at the same time.

What to log

Good candidates for logging are application start-ups and shut-downs, configuration changes, input validation failures, authentication and authorization failures, and errors. Log messages should be meaningful, contain enough context to understand the event, and be readable by people as well as parseable by tools.

Some data must never be written to logs: passwords, access tokens, session identifiers, encryption keys, database connection strings, payment card data, and sensitive personal data.

The logging module

The logging module of the standard library implements a flexible event logging system for applications and libraries. It has four main components:

Each message has a level that indicates its severity. Loggers and handlers have a threshold level; records below it are ignored.

LevelValueUsed for
CRITICAL50a serious error; the program may not be able to continue
ERROR40a failure of some operation
WARNING30something unexpected that does not stop the program
INFO20confirmation that things work as expected
DEBUG10detailed information for diagnosing problems
NOTSET0no level set; the level is taken from the parent logger
Table: Logging levels

If the threshold is WARNING, the WARNING, ERROR, and CRITICAL messages are logged. If it is ERROR, only ERROR and CRITICAL messages pass.

Simple example

The module-level functions debug, info, warning, error, and critical can be used right away without any configuration. They log through the root logger.

simple.py
import logging

logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

The example calls the five functions. If the root logger has no handlers, the functions call basicConfig first, which adds a handler that writes to the standard error output in the LEVEL:logger:message format.

Only three messages are written. The root logger has the WARNING level by default, so the debug and info messages are dropped.

$ python simple.py
WARNING:root:This is a warning message
ERROR:root:This is an error message
CRITICAL:root:This is a critical message

Configuring with basicConfig

The basicConfig function does basic configuration of the root logger. It sets the level and the format and creates a handler. It has an effect only the first time it is called (unless force=True is passed), so it should be called once, early in the program.

basic_config.py
import logging

logging.basicConfig(level=logging.DEBUG,
                    format='%(levelname)-8s %(name)s: %(message)s')

logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')

We lower the threshold to DEBUG, so all messages are written. The format string uses the levelname, name, and message attributes of a log record; -8 pads the level name to eight characters, which aligns the messages.

$ python basic_config.py
DEBUG    root: This is a debug message
INFO     root: This is an info message
WARNING  root: This is a warning message

The last resort handler

Creating a logger and setting its level is not enough to see all its messages. A logger needs a handler. When no handler is found in the logger or its ancestors, Python uses a last resort handler, which prints bare messages of level WARNING and higher to standard error.

last_resort.py
import logging

logger = logging.getLogger('dev')
logger.setLevel(logging.DEBUG)

logger.debug('This is a debug message')
logger.info('This is an info message')
logger.warning('This is a warning message')
logger.error('This is an error message')

print(logger.handlers, logging.getLogger().handlers)

Although the logger level is DEBUG, the debug and info messages are not shown. Neither the dev logger nor the root logger has a handler, as the empty lists show, so the warning and the error are printed by the last resort handler without any formatting. Configuring the root logger with basicConfig, or adding a handler, fixes this.

$ python last_resort.py
This is a warning message
This is an error message
[] []

Named loggers

The getLogger function returns a logger with the given name. Calls with the same name return the same logger object, so loggers never have to be passed around. A common practice is to create one logger per module, named after the module with __name__.

named_logger.py
import logging

logger = logging.getLogger(__name__)


def divide(a, b):
    logger.debug('dividing %d by %d', a, b)
    return a / b


logging.basicConfig(level=logging.DEBUG,
                    format='%(levelname)s %(name)s %(funcName)s: %(message)s')

logger.info('starting')
print(divide(10, 4))
print(logging.getLogger(__name__) is logger)

The logger is created at module level and used in a function. In a script run directly, __name__ is __main__; in an imported module, it would be the module's dotted name. The funcName attribute shows the function that issued the message.

The module only creates the logger; the configuration is done by basicConfig in the main part of the program. Our logger has no handler and no level of its own, so its messages propagate to the root logger, which handles them.

$ python named_logger.py
INFO __main__ <module>: starting
DEBUG __main__ divide: dividing 10 by 4
2.5
True

Handlers

A handler dispatches log records to a destination. A logger can have several handlers, each with its own level and formatter. The StreamHandler writes to a stream (standard error by default) and the FileHandler writes to a file.

handlers.py
import logging

logger = logging.getLogger('app')
logger.setLevel(logging.DEBUG)

console = logging.StreamHandler()
console.setLevel(logging.WARNING)

file_handler = logging.FileHandler('app.log', mode='w', encoding='utf-8')
file_handler.setLevel(logging.DEBUG)

fmt = logging.Formatter('%(levelname)s: %(message)s')
console.setFormatter(fmt)
file_handler.setFormatter(fmt)

logger.addHandler(console)
logger.addHandler(file_handler)

logger.debug('connecting to database')
logger.info('connection established')
logger.warning('disk usage at 91%')
logger.error('query failed')

The logger lets all messages through, as its level is DEBUG. The handlers then apply their own thresholds: the console shows only warnings and errors, while the file receives everything.

file_handler = logging.FileHandler('app.log', mode='w', encoding='utf-8')

By default, a FileHandler appends to the file. With mode='w', the file is truncated each time the program starts. It is good practice to specify the encoding explicitly.

$ python handlers.py
WARNING: disk usage at 91%
ERROR: query failed
$ cat app.log
DEBUG: connecting to database
INFO: connection established
WARNING: disk usage at 91%
ERROR: query failed

Formatters

A formatter defines the layout of the final message. Besides the message text, it can include the time, the logger name, the level, the source file and line, the process and thread, and other LogRecord attributes.

formatter.py
import logging

logger = logging.getLogger('dev')
logger.setLevel(logging.INFO)

handler = logging.StreamHandler()
formatter = logging.Formatter(
    '%(asctime)s %(name)s %(levelname)s [%(filename)s:%(lineno)d] %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)

logger.info('information message')

formatter = logging.Formatter('{asctime} | {levelname:<7} | {message}',
                              datefmt='%Y-%m-%d %H:%M:%S', style='{')
handler.setFormatter(formatter)

logger.info('information message')
logger.warning('warning message')

The first formatter uses the default percent style. The asctime attribute is the time of the record, filename and lineno point to the call in the source code.

formatter = logging.Formatter('{asctime} | {levelname:<7} | {message}',
                              datefmt='%Y-%m-%d %H:%M:%S', style='{')

With style='{', the format string uses the str.format syntax, including its format specifications. The datefmt parameter formats the time with strftime directives; here we drop the milliseconds.

$ python formatter.py
2026-11-16 23:01:15,568 dev INFO [formatter.py:12] information message
2026-11-16 23:01:15 | INFO    | information message
2026-11-16 23:01:15 | WARNING | warning message

Logger hierarchy

Loggers form a hierarchy based on their dotted names. A logger named app is the parent of app.db, and the root logger is the ancestor of all loggers.

A logger with no level set (NOTSET) uses the level of the nearest ancestor that has one; this is its effective level. After a logger handles a record, it passes the record to the handlers of its ancestors. This is called propagation. Thanks to it, it is usually enough to configure handlers on a top-level logger.

hierarchy.py
import logging

logging.basicConfig(format='%(name)-10s %(levelname)-8s %(message)s')

app = logging.getLogger('app')
app.setLevel(logging.INFO)

db = logging.getLogger('app.db')
api = logging.getLogger('app.api')
api.setLevel(logging.ERROR)

print(db.parent.name, app.parent.name)
print(app.getEffectiveLevel(), db.getEffectiveLevel(), api.getEffectiveLevel())

db.info('connected')
api.info('request received')
api.error('request failed')

db.propagate = False
db.warning('no handler found, last resort used')
print(db.handlers, db.propagate)

We create the app logger with the INFO level and two children. The app.db logger has no level, so its effective level is 20, inherited from app. The app.api logger has its own ERROR level and drops the info message.

None of the three loggers has a handler. Their messages propagate up to the root logger, whose handler was created by basicConfig.

db.propagate = False

When propagation is turned off, records stop at the app.db logger. Since it has no handler, the last resort handler prints the warning without formatting.

$ python hierarchy.py
app root
20 20 40
app.db     INFO     connected
app.api    ERROR    request failed
no handler found, last resort used
[] False

Logging variable data

Variable data is passed to a logging call as extra arguments, with % placeholders in the message. The message is formatted only if the record is actually emitted.

log_variables.py
import logging

logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s')
logger = logging.getLogger('auth')


class Expensive:
    def __str__(self):
        print('  (str() called)')
        return 'expensive object'


user = 'jsmith'
attempts = 3

logger.warning('authentication failed for %s after %d attempts', user, attempts)
logger.info('%(user)s logged in from %(ip)s', {'user': user, 'ip': '10.0.0.7'})

logger.debug('lazy: %s', Expensive())
logger.debug(f'eager: {Expensive()}')

The first call fills in a string and an integer. A single dictionary argument can be used with named placeholders.

logger.debug('lazy: %s', Expensive())
logger.debug(f'eager: {Expensive()}')

The Expensive class announces when it is converted to a string. Both debug messages are below the INFO threshold and are not logged. With the placeholder style, the conversion never happens. The f-string is evaluated before debug is called, so the work is done in vain. Keeping the message template constant also helps tools that group log messages.

$ python log_variables.py
WARNING: authentication failed for jsmith after 3 attempts
INFO: jsmith logged in from 10.0.0.7
  (str() called)

Logging exceptions

The exception method logs a message with the ERROR level and adds the traceback of the exception being handled. It should be called only from an exception handler. Other methods accept exc_info=True to include the traceback.

exceptions.py
import logging

logging.basicConfig(format='%(levelname)s: %(message)s')
logger = logging.getLogger('calc')

vals = [1, 2]

try:
    print(vals[4])
except IndexError:
    logger.exception('cannot read the value')

try:
    result = 1 / 0
except ZeroDivisionError as e:
    logger.error('division failed: %s', e)

In the first handler, we log the traceback of the IndexError. In the second one, we log only the error message, which is enough when the location of the problem is clear.

$ python exceptions.py
ERROR: cannot read the value
Traceback (most recent call last):
  File "/home/user/python/exceptions.py", line 9, in <module>
    print(vals[4])
          ~~~~^^^
IndexError: list index out of range
ERROR: division failed: division by zero

Logging to a file

With the filename parameter, basicConfig creates a file handler instead of a stream handler.

log_file.py
import logging

logging.basicConfig(filename='test.log', filemode='w', encoding='utf-8',
                    level=logging.DEBUG,
                    format='%(asctime)s %(filename)s: %(message)s',
                    datefmt='%Y-%m-%d %H:%M:%S')

logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('Čerešňa is not a valid product code')

The filemode parameter is the file open mode; the default is 'a'. The encoding parameter is important when the messages contain non-ASCII characters. The datefmt parameter formats the time. The program writes nothing to the console.

$ python log_file.py
$ cat test.log
2026-11-16 23:01:15 log_file.py: This is a debug message
2026-11-16 23:01:15 log_file.py: This is an info message
2026-11-16 23:01:15 log_file.py: Čerešňa is not a valid product code

Contextual information

The extra parameter adds custom attributes to a log record, which the formatter can then display. A LoggerAdapter adds the same attributes to every call automatically.

extra_context.py
import logging

logging.basicConfig(level=logging.INFO,
                    format='%(levelname)s [%(request_id)s] %(message)s')
logger = logging.getLogger('web')

logger.info('request started', extra={'request_id': 'a1f3'})

adapter = logging.LoggerAdapter(logger, {'request_id': 'b72c'})
adapter.info('request started')
adapter.warning('slow query: %d ms', 1840)

The format string refers to the custom request_id attribute. In the first call, we pass it with extra. The adapter wraps the logger and supplies the attribute itself, which is convenient when many messages share the same context. Note that every record reaching this formatter must have the attribute, otherwise formatting fails.

$ python extra_context.py
INFO [a1f3] request started
INFO [b72c] request started
WARNING [b72c] slow query: 1840 ms

Filters

A filter decides whether a record is logged. It can also modify the record, for example add an attribute. A filter is either a logging.Filter subclass or any callable that takes a record and returns a true or false value.

filters.py
import logging


class HostFilter(logging.Filter):
    def __init__(self, hostname):
        super().__init__()
        self.hostname = hostname

    def filter(self, record):
        record.hostname = self.hostname
        return True


def no_passwords(record):
    return 'password' not in record.getMessage().lower()


handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter('%(hostname)s %(levelname)s: %(message)s'))
handler.addFilter(HostFilter('web01'))
handler.addFilter(no_passwords)

logger = logging.getLogger('app')
logger.addHandler(handler)
logger.setLevel(logging.INFO)

logger.info('user jsmith logged in')
logger.info('password for %s changed to %s', 'jsmith', 'abc123')
logger.info('user jsmith logged out')

The HostFilter adds a hostname attribute to every record and lets it through. The no_passwords function rejects records whose message mentions a password. The getMessage method returns the message with the arguments merged in. Both filters are attached to the handler.

$ python filters.py
web01 INFO: user jsmith logged in
web01 INFO: user jsmith logged out

Rotating log files

Log files of long-running programs grow without limit. The RotatingFileHandler from the logging.handlers module starts a new file when the current one reaches a size limit and keeps a given number of old files.

rotating.py
import logging
from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler('rotating.log', maxBytes=100, backupCount=2)
handler.setFormatter(logging.Formatter('%(levelname)s %(message)s'))

logger = logging.getLogger('rot')
logger.setLevel(logging.INFO)
logger.addHandler(handler)

for i in range(1, 13):
    logger.info('message number %02d', i)

To see the rotation, we set an unrealistically small limit of 100 bytes. When the limit would be exceeded, rotating.log is renamed to rotating.log.1, the previous .1 file becomes .2, and so on; files beyond backupCount are deleted. The oldest messages, 1 to 4, are in the last backup and the newest in the current file. The TimedRotatingFileHandler rotates by time instead.

$ python rotating.py
$ ls -1 rotating.log*
rotating.log
rotating.log.1
rotating.log.2
$ cat rotating.log.2
INFO message number 01
INFO message number 02
INFO message number 03
INFO message number 04
$ cat rotating.log
INFO message number 09
INFO message number 10
INFO message number 11
INFO message number 12

Configuration with dictConfig

Larger applications configure logging declaratively. The logging.config.dictConfig function takes a dictionary that describes formatters, handlers, and loggers. This is the recommended way to configure logging; the older fileConfig uses an INI file and supports fewer features.

dict_config.py
import logging
import logging.config

config = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'simple': {'format': '%(levelname)s %(name)s: %(message)s'},
        'detailed': {
            'format': '%(asctime)s %(levelname)s %(name)s %(lineno)d: %(message)s',
            'datefmt': '%H:%M:%S',
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'level': 'INFO',
            'formatter': 'simple',
            'stream': 'ext://sys.stdout',
        },
        'file': {
            'class': 'logging.FileHandler',
            'level': 'DEBUG',
            'formatter': 'detailed',
            'filename': 'dict.log',
            'mode': 'w',
        },
    },
    'loggers': {
        'shop': {'level': 'DEBUG', 'handlers': ['console', 'file'], 'propagate': False},
    },
    'root': {'level': 'WARNING', 'handlers': ['console']},
}

logging.config.dictConfig(config)

logger = logging.getLogger('shop.cart')
logger.debug('cart created')
logger.info('item added')

logging.getLogger('other').warning('root handles this')
print(sorted(logging.getHandlerNames()))

The configuration defines two formatters and two handlers. The ext://sys.stdout string refers to an object outside the configuration. The shop logger sends info messages to the console and debug messages to the file; 'propagate': False stops its records from being handled again by the root logger.

The disable_existing_loggers key defaults to True, which silences loggers created before the configuration, such as module-level loggers of already imported modules. Setting it to False is usually what we want. The getHandlerNames function, added in Python 3.12, returns the names of the configured handlers.

$ python dict_config.py
INFO shop.cart: item added
WARNING other: root handles this
['console', 'file']
$ cat dict.log
23:01:15 DEBUG shop.cart 38: cart created
23:01:15 INFO shop.cart 39: item added

Configuration in YAML

The dictionary for dictConfig is often stored in a YAML file. YAML is a human-readable data serialization language commonly used for configuration. To read it, we need the pyyaml package.

$ pip install pyyaml

Loading the YAML configuration

The YAML file mirrors the structure of the configuration dictionary.

config.yaml
version: 1
disable_existing_loggers: false

formatters:
  simple:
    format: "%(name)s: %(message)s"
  extended:
    format: "%(asctime)s %(name)s %(levelname)s: %(message)s"
    datefmt: "%Y-%m-%d %H:%M:%S"

handlers:
  console:
    class: logging.StreamHandler
    level: INFO
    formatter: simple
    stream: ext://sys.stdout

  file_handler:
    class: logging.FileHandler
    level: DEBUG
    filename: test.log
    mode: w
    formatter: extended

loggers:
  dev:
    level: DEBUG
    handlers: [console, file_handler]
    propagate: false

root:
  level: WARNING
  handlers: [file_handler]

The dev logger writes simple messages to the console and detailed ones to test.log. The propagate option belongs to a logger, not a handler; set to false, it prevents the dev records from reaching the root logger's file handler and being written twice.

log_yaml.py
import logging
import logging.config

import yaml

with open('config.yaml', encoding='utf-8') as f:
    log_cfg = yaml.safe_load(f)

logging.config.dictConfig(log_cfg)

logger = logging.getLogger('dev')

logger.debug('This is a debug message')
logger.info('This is an info message')
logger.error('This is an error message')

logging.getLogger('legacy').warning('This goes to the root logger')

We load the file with yaml.safe_load and pass the resulting dictionary to dictConfig. The legacy logger is not configured, so its warning propagates to the root logger and ends up in the file.

$ python log_yaml.py
dev: This is an info message
dev: This is an error message
$ cat test.log
2026-11-16 23:01:15 dev DEBUG: This is a debug message
2026-11-16 23:01:15 dev INFO: This is an info message
2026-11-16 23:01:15 dev ERROR: This is an error message
2026-11-16 23:01:15 legacy WARNING: This goes to the root logger

JSON log output

Log aggregation systems prefer structured data. A custom formatter can output each record as a JSON object on one line.

json_logs.py
import json
import logging
from datetime import datetime, timezone


class JsonFormatter(logging.Formatter):
    def format(self, record):
        entry = {
            'time': datetime.fromtimestamp(record.created, timezone.utc)
                            .isoformat(timespec='seconds'),
            'level': record.levelname,
            'logger': record.name,
            'message': record.getMessage(),
        }
        if record.exc_info:
            entry['exception'] = self.formatException(record.exc_info)
        return json.dumps(entry)


handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())

logger = logging.getLogger('api')
logger.addHandler(handler)
logger.setLevel(logging.INFO)

logger.info('order %d shipped', 1042)

try:
    int('abc')
except ValueError:
    logger.exception('bad input')

The JsonFormatter overrides the format method. The created attribute is the record time as a timestamp; we convert it to an ISO 8601 string in UTC. If the record carries exception information, formatException renders the traceback, and json.dumps escapes its newlines, so each record stays on a single line.

$ python json_logs.py
{"time": "2026-11-16T22:01:15+00:00", "level": "INFO", "logger": "api", "message": "order 1042 shipped"}
{"time": "2026-11-16T22:01:15+00:00", "level": "ERROR", "logger": "api", "message": "bad input", "exception": "Traceback (most recent call last):\n  File \"/home/user/python/json_logs.py\", line 30, in <module>\n    int('abc')\n    ~~~^^^^^^^\nValueError: invalid literal for int() with base 10: 'abc'"}

Level names

The getLevelNamesMapping function, added in Python 3.11, returns the mapping of level names to their numeric values.

levels.py
import logging

for name, value in logging.getLevelNamesMapping().items():
    print(f'{name:<8} {value:>2}')

print(logging.getLevelName(30), logging.getLevelName('ERROR'))

The mapping also contains the FATAL and WARN aliases. The getLevelName function converts in both directions, from a number to a name and from a name to a number. Level names can also be passed directly to setLevel, for example logger.setLevel('DEBUG').

$ python levels.py
CRITICAL 50
FATAL    50
ERROR    40
WARN     30
WARNING  30
INFO     20
DEBUG    10
NOTSET    0
WARNING 40

Logging in libraries

A library should create loggers and log messages but leave the configuration to the application. To avoid output from the last resort handler when the application does not configure logging, a library adds a NullHandler to its top-level logger.

library.py
import logging

logger = logging.getLogger('mylib')
logger.addHandler(logging.NullHandler())


def fetch():
    logger.warning('using cached data')
    return [1, 2, 3]


print(fetch())

logging.basicConfig(format='%(levelname)s %(name)s: %(message)s')
print(fetch())

The first call of fetch logs a warning, but the NullHandler counts as a handler and discards it, so nothing is printed. After the application configures logging with basicConfig, the library's messages propagate to the root logger and appear in the output.

$ python library.py
[1, 2, 3]
WARNING mylib: using cached data
[1, 2, 3]

Source

logging – Python documentation

Logging HOWTO – Python documentation

Logging cookbook – Python documentation

In this article we have worked with the Python logging module. See also Python YAML and Python exceptions.

Author

I am Jan Bodnar, a programmer and technical writer. I have been writing programming tutorials at ZetCode since 2007 and have published eight e-books. See about the author.

List all Python tutorials.