Deploying Python Applications with Docker: Beyond the Basics¶
I've deployed dozens of Python applications with Docker. The first Dockerfile I wrote took 10 minutes and worked perfectly on my laptop. The first one that worked reliably in production took three weeks of debugging production issues.
The gap between "it works on my machine" and "it runs reliably in production" is filled with details nobody tells you about in the tutorials. Here's what I've learned from building Python applications that run in production Docker containers processing millions of dollars in transactions every day.
Multi-Stage Builds: The Right Way¶
Most Python Dockerfiles start with a single stage that installs everything and runs the app. This creates bloated images with build tools, test dependencies, and source files you don't need in production.
The naive approach:
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "app.py"]
Problems: - Image contains pip, setuptools, and all build dependencies - Source code includes tests, documentation, and development files - Final image is 800MB+ for what should be a 200MB app - No separation between build-time and runtime dependencies
The production approach:
# Build stage
FROM python:3.11-slim AS builder
WORKDIR /build
# Install build dependencies in a virtual environment
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Runtime stage
FROM python:3.11-slim
# Create non-root user
RUN useradd -m -u 1000 appuser
WORKDIR /app
# Copy only the virtual environment from builder
COPY --from=builder /opt/venv /opt/venv
# Copy only application code (exclude tests, docs, etc.)
COPY --chown=appuser:appuser src/ ./src/
COPY --chown=appuser:appuser app.py .
# Use the virtual environment
ENV PATH="/opt/venv/bin:$PATH"
# Run as non-root user
USER appuser
CMD ["python", "app.py"]
What changed:
- Two-stage build — build stage installs dependencies, runtime stage gets clean copy
- Virtual environment — isolates dependencies cleanly
- Non-root user — security best practice (containers shouldn't run as root)
- Selective copying — only production code makes it to final image
- No cache — --no-cache-dir reduces image size by not storing pip cache
Result: Image size drops from 800MB to 200MB. Attack surface reduced. Faster pulls and deployments.
Secrets Management: Don't Commit Credentials¶
The number one mistake I see in Python Docker deployments: hardcoded credentials or secrets baked into images.
Anti-patterns to avoid:
# DON'T: Hardcode secrets in Dockerfile
ENV DATABASE_PASSWORD="super_secret_password"
# DON'T: Copy .env files into the image
COPY .env /app/.env
# DON'T: Build secrets into the image
RUN echo "API_KEY=secret123" > /app/config.ini
Why this fails:
- Secrets are visible in docker history
- Anyone with access to the image has your credentials
- You can't rotate secrets without rebuilding the image
- Secrets leak into logs and error messages
The right approach: Environment variables at runtime
# Dockerfile - no secrets here
FROM python:3.11-slim
WORKDIR /app
COPY src/ ./src/
COPY app.py .
CMD ["python", "app.py"]
# app.py - read secrets from environment
import os
DATABASE_URL = os.environ['DATABASE_URL']
API_KEY = os.environ['API_KEY']
SECRET_KEY = os.environ.get('SECRET_KEY') # Optional with default
if not DATABASE_URL:
raise ValueError("DATABASE_URL environment variable required")
Run with secrets:
# Development (local .env file)
docker run --env-file .env myapp
# Production (orchestrator provides secrets)
docker run \
-e DATABASE_URL="$DATABASE_URL" \
-e API_KEY="$API_KEY" \
myapp
Better: Docker secrets for sensitive data
If you're using Docker Swarm or Kubernetes, use their secrets management:
# Read secret from mounted file (Docker Swarm pattern)
def read_secret(secret_name):
secret_path = f"/run/secrets/{secret_name}"
if os.path.exists(secret_path):
with open(secret_path) as f:
return f.read().strip()
return os.environ.get(secret_name)
DATABASE_PASSWORD = read_secret('database_password')
The rule: Secrets enter the container at runtime, never at build time.
Health Checks: Know When Your App Is Actually Ready¶
Your container can be running without your application being ready to serve traffic. Health checks tell your orchestrator (Docker Swarm, Kubernetes, ECS) when your app is actually ready and healthy.
Without health checks: - Container starts → orchestrator sends traffic immediately - App is still initializing database connections - First 20 requests fail with 500 errors - Users see errors, monitoring alerts fire
With health checks: - Container starts → orchestrator waits for health check to pass - App initializes, health check returns success - Only then does traffic flow to the container - Users never see startup errors
Add a health check endpoint:
from flask import Flask, jsonify
app = Flask(__name__)
@app.route('/health')
def health_check():
"""Health check endpoint for Docker"""
# Check critical dependencies
checks = {
'database': check_database_connection(),
'cache': check_redis_connection(),
'status': 'healthy'
}
if all(checks.values()):
return jsonify(checks), 200
else:
return jsonify(checks), 503
def check_database_connection():
try:
# Attempt simple query
db.execute('SELECT 1')
return True
except Exception:
return False
def check_redis_connection():
try:
redis_client.ping()
return True
except Exception:
return False
Add health check to Dockerfile:
FROM python:3.11-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
# Health check configuration
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
CMD python -c "import requests; requests.get('http://localhost:5000/health').raise_for_status()"
CMD ["python", "app.py"]
What this does: - Interval: Check every 30 seconds - Timeout: Fail if check takes longer than 10 seconds - Start period: Give app 40 seconds to start before failing checks - Retries: Mark unhealthy after 3 consecutive failures
Result: Orchestrator automatically removes unhealthy containers and replaces them. Zero-downtime deployments become reliable.
Logging: Make Your App Observable¶
Containerized applications write logs to stdout/stderr, not files. Docker captures these and forwards them to your logging system (CloudWatch, Splunk, ELK stack).
Anti-pattern:
# DON'T: Write logs to files in containers
logging.basicConfig(
filename='/var/log/app.log',
level=logging.INFO
)
Problems:
- Logs stay inside the container (lost when container stops)
- No centralized logging
- Debugging requires docker exec into running containers
- No log rotation (fills disk)
The right pattern:
import logging
import sys
# Configure logging to stdout
logging.basicConfig(
stream=sys.stdout,
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
# Use structured logging for better searchability
logger.info("Processing transaction", extra={
'transaction_id': txn_id,
'customer_id': customer_id,
'amount': amount
})
Better: JSON structured logs
import logging
import json
import sys
class JSONFormatter(logging.Formatter):
def format(self, record):
log_obj = {
'timestamp': self.formatTime(record),
'level': record.levelname,
'message': record.getMessage(),
'logger': record.name
}
# Include extra fields if present
if hasattr(record, 'transaction_id'):
log_obj['transaction_id'] = record.transaction_id
if hasattr(record, 'customer_id'):
log_obj['customer_id'] = record.customer_id
return json.dumps(log_obj)
# Configure JSON logging
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JSONFormatter())
logger = logging.getLogger(__name__)
logger.addHandler(handler)
logger.setLevel(logging.INFO)
# Usage
logger.info("Processing transaction", extra={
'transaction_id': 'txn_123',
'customer_id': 'cust_456',
'amount': 99.99
})
Output:
{"timestamp": "2025-12-01 10:15:23", "level": "INFO", "message": "Processing transaction", "transaction_id": "txn_123", "customer_id": "cust_456"}
Why JSON logs: - Easily parsed by logging systems - Searchable by specific fields (transaction_id, customer_id) - Supports complex data structures - Standard format for cloud logging platforms
CI/CD Integration: Automate the Build¶
Manual Docker builds don't scale. Automate building, testing, and pushing images in your CI/CD pipeline.
GitHub Actions example:
name: Build and Push Docker Image
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and test
run: |
docker build -t myapp:test .
docker run myapp:test pytest tests/
- name: Build and push production image
if: github.ref == 'refs/heads/main'
uses: docker/build-push-action@v4
with:
context: .
push: true
tags: |
myorg/myapp:latest
myorg/myapp:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
What this does: - Builds image on every PR and push to main - Runs tests inside container (ensures tests pass in production environment) - Tags images with commit SHA (traceable deployments) - Uses GitHub Actions cache for faster builds - Only pushes to registry on successful main branch builds
Common Production Mistakes¶
Mistake 1: Using latest tag in production¶
Don't do this:
docker pull myapp:latest
docker run myapp:latest
Problem: latest is a moving target. You can't roll back. You don't know what version is running.
Do this instead:
docker pull myapp:v1.2.3
docker run myapp:v1.2.3
Use semantic versioning or commit SHAs. Always deploy pinned versions.
Mistake 2: Running as root¶
# DON'T
FROM python:3.11-slim
COPY . /app
CMD ["python", "app.py"]
If your application is compromised, the attacker has root access to the container and potentially the host.
# DO
FROM python:3.11-slim
RUN useradd -m appuser
USER appuser
COPY . /app
CMD ["python", "app.py"]
Mistake 3: No resource limits¶
Without resource limits, one container can consume all CPU/memory and starve other containers.
Set resource limits in docker-compose.yml:
services:
app:
image: myapp:v1.2.3
deploy:
resources:
limits:
cpus: '2'
memory: 1G
reservations:
cpus: '0.5'
memory: 256M
Mistake 4: Not handling signals properly¶
Your Python app needs to handle SIGTERM for graceful shutdowns (finish processing current requests before stopping).
import signal
import sys
def signal_handler(sig, frame):
logger.info("Received shutdown signal, finishing current requests...")
# Finish in-flight requests
app.cleanup()
sys.exit(0)
signal.signal(signal.SIGTERM, signal_handler)
signal.signal(signal.SIGINT, signal_handler)
The Production Dockerfile Template¶
Here's the pattern I use for all production Python applications:
# Multi-stage build for minimal image size
FROM python:3.11-slim AS builder
WORKDIR /build
# Install dependencies in virtual environment
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Production stage
FROM python:3.11-slim
# Create non-root user
RUN useradd -m -u 1000 appuser
WORKDIR /app
# Copy virtual environment from builder
COPY --from=builder /opt/venv /opt/venv
# Copy application code
COPY --chown=appuser:appuser src/ ./src/
COPY --chown=appuser:appuser app.py .
# Use virtual environment
ENV PATH="/opt/venv/bin:$PATH"
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
CMD python -c "import requests; requests.get('http://localhost:8000/health').raise_for_status()"
# Run as non-root user
USER appuser
# Use exec form for proper signal handling
CMD ["python", "app.py"]
This template: - Uses multi-stage build for minimal image size - Runs as non-root user for security - Includes health check for orchestrator integration - Uses virtual environment for clean dependency isolation - Properly handles signals with exec form CMD
The Bottom Line¶
Getting Docker working locally is easy. Getting it working reliably in production requires attention to details most tutorials skip: multi-stage builds for image size, secrets management for security, health checks for reliability, structured logging for observability, and CI/CD automation for consistency.
Start with these patterns. They're not the fanciest or most cutting-edge approaches, but they're the ones that work reliably when you're deploying revenue-critical Python applications that need to run 24/7 without manual intervention.
Your production Docker deployment should be boring and reliable. Save the experimentation for your side projects.