Logging Guide - Physical AI Handbook Backend

Overview

This backend implements a centralized, environment-aware logging system that provides:

• Consistent logging across all modules
• Environment-specific log formats and levels
• Production-safe logging (no secrets)
• Zero print() statements
• Structured logging for log aggregation

---

Why Centralized Logging?

Problems with print()

NEVER use print() in this codebase. Here's why:

Issue	print()	Logging
Control	No control over destination	Configurable handlers
Filtering	Can't filter by severity	Log levels (DEBUG, INFO, etc.)
Context	No timestamps or metadata	Automatic timestamps, module names
Production	Pollutes stdout, can't disable	Configurable per environment
Aggregation	Not captured by tools	Works with ELK, Datadog, etc.
Thread Safety	Not thread-safe	Thread-safe

Benefits of Centralized Logging

1. Single Source of Truth: All logging configuration in one module
2. Consistency: All modules use same format and levels
3. Environment-Aware: Different behavior for dev/test/prod
4. Easy to Change: Update logging globally by changing one file
5. Production-Ready: Structured format for log aggregation tools

---

Environment-Specific Behavior

Development Environment

Purpose: Local development with maximum visibility

Configuration:

# .env.development
ENV=development
LOG_LEVEL=DEBUG

Format: Human-readable with colors

[2024-01-15 10:30:45] INFO main.py:40 - Application started
[2024-01-15 10:30:46] DEBUG api.py:22 - Processing request: GET /health
[2024-01-15 10:30:47] WARNING service.py:55 - Cache miss for key: user_123

Features:

• ✅ ANSI colors for different log levels
• ✅ Millisecond timestamps
• ✅ Full module paths and line numbers
• ✅ Stack traces for errors
• ✅ Verbose output (DEBUG level)

Use Case: Debugging, local development, troubleshooting

---

Testing Environment

Purpose: Test execution with minimal noise

Configuration:

# .env.testing
ENV=testing
LOG_LEVEL=ERROR  # Only show errors

Format: Minimal

ERROR - Database connection failed
ERROR - Test assertion failed

Features:

• ✅ Minimal format (no timestamps in tests)
• ✅ Only ERROR and above (reduces noise)
• ✅ No colors (cleaner test output)
• ✅ Fast and simple

Use Case: pytest execution, CI/CD pipelines

---

Production Environment

Purpose: Live deployment with structured logging

Configuration:

# .env.production
ENV=production
LOG_LEVEL=WARNING  # Only warnings and errors

Format: Structured (JSON-compatible)

timestamp=2024-01-15T10:30:45.123Z level=WARNING logger=main message=High memory usage detected
timestamp=2024-01-15T10:30:46.456Z level=ERROR logger=database message=Connection pool exhausted

Features:

• ✅ Structured key-value format
• ✅ ISO timestamps with timezone
• ✅ Ready for log aggregation (ELK, Datadog)
• ✅ No colors (log files)
• ✅ Error-focused (WARNING+)

Use Case: Production deployment, log monitoring, alerting

---

Usage

Basic Usage

# In any module
from lib.logging_config import get_logger

# Get logger for this module
logger = get_logger(__name__)

# Log at different levels
logger.debug("Detailed debugging information")
logger.info("General informational message")
logger.warning("Warning: something unexpected")
logger.error("Error occurred", exc_info=True)  # Includes stack trace
logger.critical("Critical failure!")

Application Startup

# In main.py (already configured)
from lib.logging_config import setup_logging, get_logger
from lib.config import settings

# Initialize logging before app creation
setup_logging(settings)

# Get logger
logger = get_logger(__name__)

logger.info("Application initialized")

Logging with Context

logger = get_logger(__name__)

# Log with additional context
user_id = 123
logger.info(f"User {user_id} logged in successfully")

# Log errors with stack traces
try:
    risky_operation()
except Exception as e:
    logger.error(f"Failed to process user {user_id}", exc_info=True)

---

Log Levels

When to Use Each Level

Level	Use Case	Example
DEBUG	Detailed diagnostic info	logger.debug(f"Cache hit: {cache_key}")
INFO	General informational events	logger.info("User logged in successfully")
WARNING	Unexpected but handled situations	logger.warning("API rate limit approaching")
ERROR	Error that needs attention	logger.error("Database query failed", exc_info=True)
CRITICAL	Severe error, app may crash	logger.critical("Out of memory!")

Log Level Hierarchy

DEBUG < INFO < WARNING < ERROR < CRITICAL

If LOG_LEVEL=WARNING, only WARNING, ERROR, and CRITICAL are logged.

---

Production Safety

What NOT to Log

❌ NEVER log these:

# ❌ WRONG - Logs secrets
logger.info(f"API key: {settings.ANTHROPIC_API_KEY}")

# ❌ WRONG - Logs passwords
logger.info(f"User password: {password}")

# ❌ WRONG - Logs connection strings with credentials
logger.info(f"Database URL: {settings.DATABASE_URL}")

# ❌ WRONG - Logs sensitive user data
logger.info(f"Credit card: {user.credit_card}")

What TO Log

✅ DO log these:

# ✅ CORRECT - Logs events, not secrets
logger.info("API call to Anthropic successful")

# ✅ CORRECT - Logs sanitized info
logger.info(f"User authenticated: user_id={user.id}")

# ✅ CORRECT - Logs connection status, not credentials
logger.info("Database connection established")

# ✅ CORRECT - Logs metadata
logger.info(f"Processing {len(documents)} documents")

---

Best Practices

1. Use Module-Level Loggers

✅ CORRECT:

# At top of module
from lib.logging_config import get_logger
logger = get_logger(__name__)

def my_function():
    logger.info("Function called")

❌ WRONG:

def my_function():
    # Don't create logger inside functions
    logger = get_logger(__name__)
    logger.info("Function called")

2. Use Appropriate Log Levels

✅ CORRECT:

logger.debug(f"Processing item {i} of {total}")  # Detailed diagnostics
logger.info("Request completed successfully")    # General events
logger.warning("Cache size exceeded threshold")  # Warnings
logger.error("Failed to save data", exc_info=True)  # Errors

❌ WRONG:

logger.info(f"Processing item {i} of {total}")  # Too verbose for INFO
logger.error("User clicked button")  # Not an error

3. Include Context in Logs

✅ CORRECT:

logger.error(f"Failed to process user_id={user_id}", exc_info=True)
logger.warning(f"High latency: {latency}ms for endpoint={endpoint}")

❌ WRONG:

logger.error("Failed")  # No context
logger.warning("Slow")   # No details

4. Use exc_info for Exceptions

✅ CORRECT:

try:
    process_data()
except Exception as e:
    logger.error("Failed to process data", exc_info=True)
    raise

❌ WRONG:

try:
    process_data()
except Exception as e:
    logger.error(f"Error: {e}")  # No stack trace

5. Never Log Secrets

✅ CORRECT:

logger.info("API call to Anthropic successful")
logger.info(f"Authenticated user: {user.id}")

❌ WRONG:

logger.info(f"API key: {api_key}")
logger.info(f"Password: {password}")

---

Summary

Do's

• ✅ Use get_logger(__name__) at module level
• ✅ Use appropriate log levels (DEBUG, INFO, WARNING, ERROR)
• ✅ Include context in log messages
• ✅ Use exc_info=True for exceptions
• ✅ Configure via LOG_LEVEL environment variable

Don'ts

• ❌ Never use print()
• ❌ Never log secrets or credentials
• ❌ Don't create loggers inside functions
• ❌ Don't use INFO for debugging (use DEBUG)
• ❌ Don't hardcode log levels

Quick Reference

# Setup (in main.py - already done)
from lib.logging_config import setup_logging
setup_logging()

# Usage (in any module)
from lib.logging_config import get_logger
logger = get_logger(__name__)

logger.debug("Debug info")
logger.info("Info message")
logger.warning("Warning")
logger.error("Error", exc_info=True)
logger.critical("Critical failure")