Back to Python

Python Logging

Logging Basics

Python's built-in logging module provides a flexible framework for emitting log messages from applications. It's more powerful than simple print statements and is the standard way to handle logging in Python applications.

# Basic logging example
import logging

# Configure basic logging
logging.basicConfig(level=logging.INFO)

# Create a logger
logger = logging.getLogger(__name__)

# Log messages at different levels
logger.debug('Debug message') # Won't be logged
logger.info('Info message')
logger.warning('Warning message')
logger.error('Error message')
logger.critical('Critical message')

Key advantages over print statements:

  • Multiple log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)
  • Output to multiple destinations (files, console, network, etc.)
  • Flexible formatting of log messages
  • Runtime configuration without code changes
  • Better performance for production applications

Log Levels

Python's logging module provides several levels of severity for messages. Each level has a specific purpose and numeric value associated with it.

Level Numeric Value When to Use
DEBUG 10 Detailed information for diagnosing problems
INFO 20 Confirmation that things are working as expected
WARNING 30 Indication something unexpected happened
ERROR 40 Serious problem, software unable to perform function
CRITICAL 50 Severe error indicating program may be unable to continue
# Setting log levels
logging.basicConfig(level=logging.DEBUG) # Show all messages
logging.basicConfig(level=logging.WARNING) # Show warnings and above

# Temporarily change level for a logger
logger.setLevel(logging.DEBUG) # For development
logger.setLevel(logging.ERROR) # For production

Logging Configuration

Python's logging can be configured in several ways - programmatically, via configuration files, or via dictionaries. Each method has its advantages.

# Programmatic configuration
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    filename='app.log',
    filemode='a'
)

# Dictionary configuration (Python 3.2+)
config = {
    'version': 1,
    'formatters': {
        'default': {
            'format': '%(asctime)s - %(levelname)s - %(message)s'
        }
    },
    'handlers': {
        'file': {
            'class': 'logging.FileHandler',
            'filename': 'app.log',
            'formatter': 'default'
        }
    },
    'root': {
        'handlers': ['file'],
        'level': logging.INFO
    }
}
logging.config.dictConfig(config)

Common configuration options:

  • level: Minimum logging level to capture
  • format: Format string for log messages
  • filename: File to write logs to
  • filemode: File opening mode ('w' for overwrite, 'a' for append)
  • handlers: List of handlers to attach to logger

Advanced Logging Features

Python's logging module offers many advanced features for complex applications.

# Multiple handlers with different levels
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG) # Capture all messages

# Console handler for WARNING and above
c_handler = logging.StreamHandler()
c_handler.setLevel(logging.WARNING)
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
logger.addHandler(c_handler)

# File handler for all DEBUG messages
f_handler = logging.FileHandler('app.log')
f_handler.setLevel(logging.DEBUG)
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
f_handler.setFormatter(f_format)
logger.addHandler(f_handler)

# Example usage
logger.debug('Debug message') # Only in file
logger.warning('Warning message') # In both console and file

Other Advanced Features

  • Logging exceptions - logger.exception() automatically includes stack trace
  • Rotating file handlers - logging.handlers.RotatingFileHandler for log rotation
  • Timed rotation - logging.handlers.TimedRotatingFileHandler
  • Network logging - Handlers for sending logs over network (SysLogHandler, HTTPHandler)
  • Filters - Custom logic to filter log records
  • Custom log levels - Define your own levels if needed

Logging Best Practices

Follow these guidelines for effective logging in your applications:

  • Use appropriate log levels - DEBUG for development, WARNING/ERROR for production
  • Create module-level loggers - logger = logging.getLogger(__name__)
  • Include useful context - Add relevant variables and state to log messages
  • Avoid sensitive data - Never log passwords, API keys, or personal information
  • Use structured logging - For easier parsing and analysis (JSON format)
  • Configure logging early - Set up logging at application startup
  • Handle exceptions properly - Use logger.exception() in except blocks
  • Consider performance - Avoid expensive operations in debug messages
# Good logging example
try:
    result = some_operation()
    logger.info('Operation completed successfully. Result: %s', result)
except Exception as e:
    logger.error('Operation failed with error: %s', str(e))
    logger.exception('Stack trace:') # Logs full traceback

Python Logging Videos

Master Python logging with these handpicked YouTube tutorials:

Logging Basics

Learn the fundamentals:

Logging Tutorial (Corey Schafer) Python Logging (Real Python)

Advanced Techniques

Beyond basic logging:

Advanced Logging (PyCon) Structured Logging (mCoding)

Web Development

Logging in web apps:

Django Logging (Very Academy) Flask Logging (Pretty Printed)

Performance & Scaling

Logging at scale:

Logging in Production (PyCon) Centralized Logging (mCoding)

Python Logging Quiz