Docker for AI: Why Containers Make Your Stack Production-Ready

┌─────────────────────────────────────────────────────────────┐
│                    Docker Image Layers                       │
├─────────────────────────────────────────────────────────────┤
│  Layer 5  │  YOUR CONFIG        .env, docker-compose.yml    │
│  Layer 4  │  APP CODE           ollama, open-webui, n8n     │
│  Layer 3  │  DEPENDENCIES       Python, CUDA libs, Node.js  │
│  Layer 2  │  RUNTIME            Ubuntu 22.04 / Alpine       │
│  Layer 1  │  BASE IMAGE         FROM ubuntu:22.04            │
├─────────────────────────────────────────────────────────────┤
│  HOST OS  │  Shared Kernel  ──  No full VM overhead         │
└─────────────────────────────────────────────────────────────┘

  Image  ──build──>  Container  ──run──>  Service
  (blueprint)        (instance)           (accessible)

  Volume ──mount──>  Persistent Data (survives container restart)

Without Docker, your AI stack does not work on another machine. With Docker: docker compose up.

That is not an exaggeration. If you run Ollama, Open WebUI, and related services without containers, they depend on the specific drivers, paths, and dependencies of your local installation. On another machine, on a server, after an OS update — broken. Docker solves this structurally.

What Is Docker?

Docker packages an application together with everything it needs: runtime environment, dependencies, configuration. The result is a container image that runs the same everywhere — on your laptop, on a server, on a VM. The underlying operating system no longer matters.

Containers are not virtual machines. They share the host system's kernel, start in seconds, and require significantly less RAM than a VM. One process, one task, one container — that is the principle.

Docker has over 20 million active users worldwide (source). The official documentation is at docs.docker.com.

The 5 Commands You Need

# Which containers are currently running?
docker ps

# Start a container (example: Ollama)
docker run -d --name ollama -p 11434:11434 ollama/ollama

# Start a stack from docker-compose.yml
docker compose up -d

# Show logs of a container
docker logs ollama

# Stop a container
docker stop ollama

This covers 90 % of daily Docker usage. The rest comes with time.

docker-compose.yml: Your Stack as Code

Instead of starting each container individually with docker run, you define your entire stack in a docker-compose.yml. This is the file you commit to Git, share with your team, and deploy to the server.

Minimal example for Ollama and Open WebUI:

services:
  ollama:
    image: ollama/ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama:/root/.ollama
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    ports:
      - "3000:8080"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - open-webui:/app/backend/data
    restart: unless-stopped
    depends_on:
      - ollama

volumes:
  ollama:
  open-webui:

docker compose up -d starts both services. docker compose down stops them. Data persists in the volumes.

The Compose specification is documented at compose-spec.io (GitHub).

Why Docker Matters Especially for AI

AI services come with complicated dependencies: CUDA drivers for GPU acceleration, Python versions, specific library versions that can conflict. Without containers, "works on my machine" means nothing.

Concrete advantages:

GPU drivers: NVIDIA provides an official Docker image with CUDA support. docker run --gpus all ollama/ollama — done. No manual CUDA library installation.

Reproducibility: Every team member, every server, every deployment environment runs exactly the same version with exactly the same dependencies.

Isolation: Different AI projects with incompatible Python versions? Not a problem. Each container has its own environment.

Updates: New version of Open WebUI? docker pull ghcr.io/open-webui/open-webui:main && docker compose up -d. Rollback: set the old image tag back.

Our stack currently runs 31 Docker services across 5 nodes — Ollama, Whisper STT, Team-Chat, n8n, ERPNext, Grafana, and more. Without Docker, this would not be manageable.

The Next Step

Docker gives you reproducibility and portability. What is still missing for real production? Orchestration across multiple machines, health checks, automatic restarts, monitoring.

That is where Docker Swarm or Kubernetes come in. How 11 AI agents work together on this foundation — and what that kind of architecture looks like in practice — is covered in detail here: Agent Team Architecture: How We Orchestrate 11 AI Agents →

All the details for building the complete stack — from Ollama to Docker Swarm, from monitoring to GDPR compliance — are in 70 pages: The Local AI Stack Playbook for EUR 49

Sources:

• docs.docker.com — official Docker documentation
• compose-spec.io — Compose specification
• github.com/docker/compose — Docker Compose source code