Docker Compose: Orchestrating Multi-Container Applications

Docker Compose is a powerful tool for defining and running multi-container Docker applications. With Compose, you can use a YAML file to configure your application’s services, networks, and volumes, then create and start all services with a single command.

Open Table of Contents

What is Docker Compose?
- Key Benefits
Installing Docker Compose
Basic Docker Compose File Structure
Essential Docker Compose Commands
- Basic Commands
- Advanced Commands
Practical Examples
Advanced Configuration
Best Practices
Troubleshooting Common Issues
Production Considerations
- Using Docker Swarm Mode
- Alternative: Kubernetes
Conclusion

What is Docker Compose?

Docker Compose is a tool that allows you to define and manage multi-container Docker applications using a simple YAML configuration file called docker-compose.yml. Instead of running multiple docker run commands, you can define your entire application stack in one file and manage it as a single unit.

Key Benefits

Simplified Multi-Container Management: Define complex applications with multiple interconnected services
Environment Consistency: Ensure the same configuration across development, testing, and production
Easy Service Discovery: Containers can communicate using service names as hostnames
Volume and Network Management: Automatically create and manage shared resources
Scalability: Easily scale services up or down with simple commands

Installing Docker Compose

Docker Compose comes pre-installed with Docker Desktop on Windows and macOS. For Linux installations:

Using Docker Desktop

If you have Docker Desktop installed, Docker Compose is already available.

Standalone Installation (Linux)

# Download the latest version
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

# Make it executable
sudo chmod +x /usr/local/bin/docker-compose

# Verify installation
docker-compose --version

Using pip

pip install docker-compose

Basic Docker Compose File Structure

A docker-compose.yml file defines services, networks, and volumes for your application:

version: '3.8'  # Compose file format version

services:
  service1:
    # Service configuration
  service2:
    # Service configuration

networks:
  # Network definitions (optional)

volumes:
  # Volume definitions (optional)

Essential Docker Compose Commands

Basic Commands

# Start all services in detached mode
docker-compose up -d

# Start all services in foreground
docker-compose up

# Stop all services
docker-compose down

# View running services
docker-compose ps

# View logs from all services
docker-compose logs

# View logs from a specific service
docker-compose logs service-name

# Restart a specific service
docker-compose restart service-name

# Build or rebuild services
docker-compose build

# Pull latest images for all services
docker-compose pull

Advanced Commands

# Scale a service to multiple instances
docker-compose up --scale web=3

# Execute a command in a running container
docker-compose exec service-name bash

# Run a one-time command
docker-compose run service-name command

# Remove stopped containers
docker-compose rm

# Stop and remove containers, networks, and volumes
docker-compose down --volumes --remove-orphans

Practical Examples

Example 1: Simple Web Application with Database

Create a docker-compose.yml file for a web application with a PostgreSQL database:

version: '3.8'

services:
  web:
    build: .
    ports:
      - "8000:8000"
    depends_on:
      - db
    environment:
      - DATABASE_URL=postgresql://user:password@db:5432/myapp
    volumes:
      - .:/app
    command: python manage.py runserver 0.0.0.0:8000

  db:
    image: postgres:13
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"

volumes:
  postgres_data:

Example 2: Full-Stack Application (React + Node.js + MongoDB)

version: '3.8'

services:
  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    ports:
      - "3000:3000"
    depends_on:
      - backend
    environment:
      - REACT_APP_API_URL=http://localhost:5000

  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    ports:
      - "5000:5000"
    depends_on:
      - mongo
    environment:
      - MONGODB_URI=mongodb://mongo:27017/myapp
      - JWT_SECRET=your-secret-key
    volumes:
      - ./backend:/app
      - /app/node_modules

  mongo:
    image: mongo:5.0
    restart: always
    ports:
      - "27017:27017"
    volumes:
      - mongo_data:/data/db
    environment:
      MONGO_INITDB_ROOT_USERNAME: admin
      MONGO_INITDB_ROOT_PASSWORD: password

volumes:
  mongo_data:

Example 3: Development Environment with Multiple Services

version: '3.8'

services:
  app:
    build:
      context: .
      target: development
    ports:
      - "3000:3000"
    volumes:
      - .:/app
      - /app/node_modules
    environment:
      - NODE_ENV=development
      - DATABASE_URL=postgres://user:pass@postgres:5432/devdb
      - REDIS_URL=redis://redis:6379
    depends_on:
      - postgres
      - redis

  postgres:
    image: postgres:13
    environment:
      POSTGRES_DB: devdb
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"

  redis:
    image: redis:6-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    depends_on:
      - app

volumes:
  postgres_data:
  redis_data:

Advanced Configuration

Environment Variables

Use environment files to manage configurations:

# docker-compose.yml
version: '3.8'

services:
  web:
    build: .
    env_file:
      - .env
      - .env.local
    environment:
      - NODE_ENV=production

Create a .env file:

DATABASE_URL=postgresql://user:password@localhost:5432/myapp
API_KEY=your-api-key
DEBUG=false

Custom Networks

Define custom networks for service isolation:

version: '3.8'

services:
  frontend:
    image: nginx
    networks:
      - frontend-network

  backend:
    image: node:14
    networks:
      - frontend-network
      - backend-network

  database:
    image: postgres
    networks:
      - backend-network

networks:
  frontend-network:
    driver: bridge
  backend-network:
    driver: bridge
    internal: true  # No external access

Health Checks

Add health checks to ensure services are ready:

version: '3.8'

services:
  web:
    build: .
    ports:
      - "8000:8000"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    depends_on:
      db:
        condition: service_healthy

  db:
    image: postgres:13
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5

Resource Constraints

Control resource usage:

version: '3.8'

services:
  web:
    image: nginx
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
        reservations:
          cpus: '0.25'
          memory: 256M
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3

Best Practices

1. Use Specific Image Tags

# Good
services:
  db:
    image: postgres:13.4

# Avoid
services:
  db:
    image: postgres:latest

2. Use Multi-Stage Builds

# Dockerfile
FROM node:14 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

FROM node:14-alpine AS production
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY . .
CMD ["npm", "start"]

3. Separate Configuration Files

Use different compose files for different environments:

# Development
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

# Production
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up

4. Use Named Volumes for Data Persistence

volumes:
  postgres_data:
    driver: local
  redis_data:
    external: true  # Use existing volume

5. Security Considerations

services:
  db:
    image: postgres:13
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password

secrets:
  db_password:
    file: ./db_password.txt

Troubleshooting Common Issues

1. Port Conflicts

# Check which process is using a port
lsof -i :8080

# Use different host ports
ports:
  - "8081:8080"  # Host:Container

2. Volume Mount Issues

# Check volume permissions
docker-compose exec service-name ls -la /mounted/path

# Fix permission issues in Dockerfile
RUN chown -R user:user /app

3. Service Dependencies

services:
  web:
    depends_on:
      - db
    # Use wait scripts or health checks for true dependency
    command: ["./wait-for-it.sh", "db:5432", "--", "npm", "start"]

4. Debugging Services

# View detailed logs
docker-compose logs --follow service-name

# Execute commands in containers
docker-compose exec service-name bash

# Check service status
docker-compose ps

Production Considerations

Using Docker Swarm Mode

For production orchestration, consider Docker Swarm:

# Initialize swarm
docker swarm init

# Deploy stack
docker stack deploy -c docker-compose.yml myapp

# Scale services
docker service scale myapp_web=3

Alternative: Kubernetes

For larger deployments, consider migrating to Kubernetes using tools like Kompose:

# Convert docker-compose.yml to Kubernetes manifests
kompose convert -f docker-compose.yml

Conclusion

Docker Compose is an essential tool for modern application development and deployment. It simplifies the management of multi-container applications, provides consistency across environments, and enables rapid development workflows. By mastering Docker Compose, you can efficiently orchestrate complex applications, from simple web services to full-stack applications with multiple databases and services.

Whether you’re developing locally, setting up CI/CD pipelines, or deploying to production, Docker Compose provides the foundation for containerized application management. Combined with proper practices around security, monitoring, and resource management, Docker Compose becomes a powerful tool in your DevOps toolkit.