Docker for AI Stacks

Docker isolates services (Ollama, n8n, PostgreSQL) in containers. Essential for reproducible AI deployments.

What is Docker?

Container = lightweight VM with your app + dependencies.

Traditional Setup          Docker Setup
├─ Python 3.9            ├─ ollama:latest container
├─ Ollama                │  └─ ollama + dependencies
├─ PostgreSQL            ├─ n8n container
├─ n8n                   │  └─ node.js + n8n
└─ Manual config         ├─ postgres container
                         │  └─ PostgreSQL + data
   Breaks on              └─ Reproducible everywhere
   different machines

Installation

Windows

Download: https://docker.com/products/docker-desktop
Install Docker Desktop
Enable WSL2 (Windows Subsystem for Linux 2)
Restart

Verify:

docker --version
docker run hello-world

macOS

# Homebrew
brew install --cask docker

# Or download: https://docker.com/products/docker-desktop
# Then launch Docker.app

Verify:

docker --version

Linux (Ubuntu)

# Install Docker
sudo apt-get update
sudo apt-get install docker.io docker-compose

# Add your user to docker group (avoid sudo)
sudo usermod -aG docker $USER
newgrp docker

# Verify
docker --version

Docker Compose

Instead of running containers individually, define them in YAML.

Basic docker-compose.yml

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0:11434

  postgres:
    image: postgres:15-alpine
    container_name: postgres
    ports:
      - "5432:5432"
    environment:
      POSTGRES_DB: n8n
      POSTGRES_PASSWORD: secret_password
      POSTGRES_USER: n8n
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  ollama_data:
  postgres_data:

Commands

# Start all services in background
docker-compose up -d

# See running containers
docker-compose ps

# View logs
docker-compose logs -f ollama

# Stop all
docker-compose down

# Remove volumes (WARNING: deletes data)
docker-compose down -v

# Rebuild from scratch
docker-compose up --build -d

GPU Support (NVIDIA)

Run models on GPU instead of CPU for 10x speed boost.

Install nvidia-docker

Windows/macOS: Docker Desktop handles GPU automatically (if Docker version >= 4.25)

Linux:

# Install NVIDIA Container Runtime
sudo apt-get install -y nvidia-docker2

# Verify
docker run --rm --gpus all nvidia/cuda:12.0-runtime nvidia-smi

Output should show GPU(s).

Enable GPU in docker-compose.yml

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    environment:
      - OLLAMA_NUM_GPU=1  # Use 1 GPU
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all  # Use all GPUs
              capabilities: [gpu]

Verify GPU is used

# Log into container
docker exec -it ollama bash

# Check if GPU visible
nvidia-smi

# Run Ollama model
ollama run llama2
# Should show GPU usage in nvidia-smi

Networking

Port Mapping

# localhost:3000 → container:8080
services:
  app:
    ports:
      - "3000:8080"  # host:container

Access from host:

curl http://localhost:3000

Service-to-Service Communication

Services on the same network can reach each other by name.

services:
  n8n:
    environment:
      - DB_POSTGRESDB_HOST=postgres  # Name of postgres service
      - DB_POSTGRESDB_PORT=5432
  postgres:
    # No special config needed

Inside n8n, PostgreSQL is reachable at postgres:5432.

Custom Network

version: '3.8'

services:
  ollama:
    networks:
      - ai-network

  n8n:
    networks:
      - ai-network

networks:
  ai-network:
    driver: bridge

Expose to External Network

services:
  n8n:
    ports:
      - "0.0.0.0:5678:5678"  # All interfaces
    # OR
    ports:
      - "192.168.1.10:5678:5678"  # Specific IP

Volumes & Persistent Data

Containers are stateless. Volumes persist data across restarts.

Volume Types

Named volume (recommended):

services:
  postgres:
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:  # Managed by Docker

Data lives in: /var/lib/docker/volumes/postgres_data/_data/

Bind mount (direct directory):

services:
  n8n:
    volumes:
      - ./n8n_data:/home/node/.n8n  # ./n8n_data on host

Temporary (lost on restart):

services:
  app:
    # No volumes = ephemeral

Backup Volume

# Backup postgres_data volume
docker run --rm \
  -v postgres_data:/data \
  -v /tmp:/backup \
  ubuntu tar czf /backup/postgres.tar.gz /data

# Restore
docker run --rm \
  -v postgres_data:/data \
  -v /tmp:/backup \
  ubuntu tar xzf /backup/postgres.tar.gz -C /data

Environment Variables

Pass configuration without editing files.

services:
  app:
    environment:
      - DATABASE_URL=postgres://user:pass@postgres:5432/db
      - API_KEY=${API_KEY}  # From .env file

# .env file
API_KEY=sk-1234567890

Or inline:

docker-compose run -e API_KEY=sk-xxx app

Troubleshooting

Container won't start

# Check logs
docker-compose logs ollama

# Common issues:
# - Port already in use: change ports in docker-compose.yml
# - Image not found: docker-compose pull
# - Out of memory: docker-compose up --no-cache

Out of Memory (OOM)

# Container uses too much RAM
# Option 1: Limit memory in compose
services:
  ollama:
    deploy:
      resources:
        limits:
          memory: 16G

# Option 2: Reduce model size
docker exec ollama ollama pull mistral  # Smaller than llama2

Slow performance

# Check if using disk instead of SSD
docker inspect container_name | grep "MergedDir"

# Move to faster storage or use tmpfs for cache
services:
  ollama:
    tmpfs:
      - /tmp  # In-memory cache

GPU not detected

# Verify docker has GPU support
docker run --rm --gpus all nvidia/cuda:12.0-runtime nvidia-smi

# If not:
# 1. Restart docker: sudo systemctl restart docker
# 2. Check nvidia-docker: nvidia-docker --version
# 3. Restart host (last resort)

Can't connect between containers

# Check network
docker network ls
docker network inspect <network_name>

# Verify service names in docker-compose.yml
# Service-to-service requires internal network

# Test connection from container
docker exec n8n curl http://postgres:5432

Building Custom Images

# Dockerfile
FROM python:3.10-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .
CMD ["python", "app.py"]

Build and run:

docker build -t my-app:latest .
docker run -p 8000:8000 my-app:latest

In docker-compose.yml:

services:
  my-app:
    build: .  # Builds from Dockerfile in current dir
    ports:
      - "8000:8000"

Resource Management

Limit CPU & Memory

services:
  ollama:
    deploy:
      resources:
        limits:
          cpus: '4'      # Max 4 CPU cores
          memory: 16G    # Max 16GB RAM
        reservations:
          cpus: '2'      # Reserved 2 cores
          memory: 8G     # Reserved 8GB

Restart Policy

services:
  ollama:
    restart: always  # Restart if crashes
    # Options: no, always, unless-stopped, on-failure

Monitoring

View resource usage

docker stats

# Output:
# CONTAINER   CPU %   MEM USAGE   MEM %
# ollama      25%     4.5G        28%
# postgres    5%      2.1G        13%

Container logs

# Follow logs
docker-compose logs -f

# Last 100 lines
docker-compose logs --tail 100 ollama

# Since timestamp
docker-compose logs --since 2026-03-20T10:00:00Z

Common Patterns

Run migrations on startup

services:
  db-migrate:
    image: migrate/migrate:latest
    command: -path /migrations -database postgres://... up
    volumes:
      - ./migrations:/migrations
    depends_on:
      - postgres

  postgres:
    image: postgres:15
    # Wait for migrate to complete before starting app

Sidecar pattern (logging, monitoring)

services:
  app:
    image: my-app:latest

  app-logs:  # Sidecar
    image: prom/node-exporter:latest
    volumes:
      - /var/log/app:/var/log
    depends_on:
      - app

Multi-stage builds (smaller images)

# Stage 1: Build
FROM python:3.10 as builder
WORKDIR /app
COPY . .
RUN pip install --user -r requirements.txt

# Stage 2: Runtime (small)
FROM python:3.10-slim
COPY --from=builder /root/.local /root/.local
COPY . .
CMD ["python", "app.py"]

Result: 500MB image (vs 1.5GB with single stage)

Security Best Practices

Don't run as root
```
RUN useradd -m app
USER app
```

Use secrets, not env vars

docker secret create db_password -
# In compose: secrets: [db_password]

Scan for vulnerabilities
```
docker scan my-app:latest
```

Use official images only

# Good
image: ollama/ollama:latest
# Bad
image: random-user/ollama:latest