Docker Compose for Beginners: A Step-by-Step Guide

Docker Compose for Beginners: A Step-by-Step Guide

by

Docker Compose for Beginners: A Step-by-Step Guide

What is Docker Compose and Why Do You Need It?

Docker Compose is a tool for defining and running multi-container Docker applications. With a single command, you can spin up an entire stack of services—like a web server, database, and cache—that are configured to work together. Instead of manually typing long docker run commands for each container, you write a declarative YAML file (typically docker-compose.yml) that specifies every service, network, and volume.

Key benefits for beginners:

  • Single command orchestration: docker compose up handles everything.
  • Reproducible environments: Your entire app stack is defined in one file.
  • Simplified networking: Services automatically discover each other.
  • Centralized configuration: All environment variables, ports, and volumes are in one place.
  • Version control friendly: The YAML file can be committed to Git, making team collaboration seamless.

Prerequisites: What You Need to Have Installed

Before you start, ensure you have the following:

  1. Docker Engine: Both Docker Desktop (Windows/macOS) or Docker Engine (Linux) work. Verify with docker --version.
  2. Docker Compose (v2 or v3): Modern Docker installations include Compose. Check with docker compose version (v2) or docker-compose --version (v1). If missing, install it via your package manager or Docker’s official site.
  3. A code editor (e.g., VS Code, Sublime) and basic familiarity with the terminal.
  4. To check installation: Run docker run hello-world to confirm Docker works.

Step 1: Project Structure and Your First YAML File

Create a new directory for your project:

mkdir my-first-compose-app
cd my-first-compose-app

Inside this directory, create a file named docker-compose.yml. This file will define your application’s services. Here’s the structure of a basic Compose file:

version: '3.8'

services:
  web:
    image: nginx:latest
    ports:
      - "8080:80"

Explanation:

  • version: Specifies the Compose file format (3.8 is widely supported). Newer versions have additional features but 3.8 is stable for most beginners.
  • services: The top-level section where you define each container.
  • web: A service name—you choose this. Docker Compose will create a container named my-first-compose-app_web_1 by default.
  • image: The Docker image to use (pulled from Docker Hub if not local).
  • ports: Maps the host port (8080) to the container port (80). Now you can access Nginx at http://localhost:8080.

Save the file and run:

docker compose up

You will see logs from Nginx. Open your browser to http://localhost:8080—you should see the Nginx welcome page. Press Ctrl+C to stop.

Step 2: Running in Detached Mode and Understanding Logs

To run containers in the background (detached), use:

docker compose up -d

Now you can continue using the terminal. View running containers with docker compose ps. See logs with:

docker compose logs web

To follow logs in real-time: docker compose logs -f. Stop and remove everything with:

docker compose down

Important: down stops containers and removes the default network, but preserves volumes by default.

Step 3: Adding a Second Service (Database)

A single service isn’t compelling. Let’s add a PostgreSQL database. Update your docker-compose.yml:

version: '3.8'

services:
  web:
    image: nginx:latest
    ports:
      - "8080:80"
    depends_on:
      - db

  db:
    image: postgres:15
    environment:
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
      POSTGRES_DB: myapp
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

New elements explained:

  • depends_on: Tells Compose to start the db service before web. Note: this does not wait for the database to be ready, only for the container to start.
  • environment: Sets environment variables inside the container. The PostgreSQL image uses these to create a database and user automatically.
  • volumes: At the service level, this mounts a named volume (postgres_data) to persist database files. The volumes section at the bottom declares the volume so Docker manages its lifecycle.
  • Named volumes are stored by Docker and survive container restarts or removal.

Run docker compose up -d. Verify both containers are running with docker compose ps. You can inspect the database container:

docker compose exec db psql -U myuser -d myapp

Type l to list databases, then q to quit.

Step 4: Using a Custom Application (Python Flask Example)

Replace the generic Nginx service with a simple Python Flask app. Create a file named app.py in the same directory:

from flask import Flask
import psycopg2
import os

app = Flask(__name__)

def get_db_connection():
    conn = psycopg2.connect(
        host="db",
        database=os.environ['POSTGRES_DB'],
        user=os.environ['POSTGRES_USER'],
        password=os.environ['POSTGRES_PASSWORD']
    )
    return conn

@app.route('/')
def index():
    conn = get_db_connection()
    cur = conn.cursor()
    cur.execute('SELECT version();')
    db_version = cur.fetchone()
    cur.close()
    conn.close()
    return f'Database connected. PostgreSQL version: {db_version[0]}'

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Create a requirements.txt file:

Flask==3.0.0
psycopg2-binary==2.9.9

Create a Dockerfile:

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "app.py"]

Now update docker-compose.yml to use your custom image instead of Nginx:

version: '3.8'

services:
  web:
    build: .
    ports:
      - "5000:5000"
    depends_on:
      - db
    environment:
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
      POSTGRES_DB: myapp

  db:
    image: postgres:15
    environment:
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
      POSTGRES_DB: myapp
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

Key change: build: . tells Compose to build an image from the Dockerfile in the current directory instead of pulling an existing image.

Run docker compose up -d --build. The --build flag forces a rebuild even if the image exists. Visit http://localhost:5000—you should see the PostgreSQL version string. This confirms your Flask app is communicating with the database container using the service name db as the host.

Step 5: Environment Variables and .env Files

Hardcoding credentials in Compose files is insecure. Use a .env file. Create .env in the same directory:

POSTGRES_USER=myuser
POSTGRES_PASSWORD=supersecret
POSTGRES_DB=myapp

Reference these variables in your docker-compose.yml:

version: '3.8'

services:
  web:
    build: .
    ports:
      - "5000:5000"
    depends_on:
      - db
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}

  db:
    image: postgres:15
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

Add .env to .gitignore to prevent secrets from being committed. Compose automatically reads the .env file from the project root. You can also pass variables inline: POSTGRES_USER=admin docker compose up.

Step 6: Networking, Volumes, and Bind Mounts

Networking: By default, Compose creates a single network for your project. All services can reach each other using the service name as the hostname. You can define custom networks:

services:
  web:
    networks:
      - frontend
  db:
    networks:
      - backend

networks:
  frontend:
  backend:

This isolates database traffic from the frontend. Services must be on the same network to communicate.

Volumes vs. Bind Mounts:

  • Volumes are managed by Docker (e.g., postgres_data above). They persist data and are portable across hosts.
  • Bind mounts map a host directory directly into the container. Useful for development:
services:
  web:
    volumes:
      - ./app:/app   # Host directory ./app mounted to /app in container

This allows live code reloading—any change on your host machine is immediately reflected inside the container.

Step 7: Managing Dependencies and Startup Order

depends_on only controls container startup order, not readiness. For services like databases that take time to become ready, use a tool like wait-for-it.sh or the healthcheck directive.

Add a healthcheck to the database service:

services:
  db:
    image: postgres:15
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U myuser"]
      interval: 5s
      timeout: 5s
      retries: 5

Then modify the web service to wait:

web:
  depends_on:
    db:
      condition: service_healthy

Now Compose will hold the web service until the healthcheck passes.

Step 8: Scaling Services and Resource Limits

Scale a service horizontally using --scale:

docker compose up -d --scale web=3

This runs three instances of the web container. Compose assigns different host ports automatically (e.g., 5000, 5001, 5002) if you don’t specify a fixed port mapping. Alternatively, use a load balancer like Nginx or Traefik in your stack.

Set CPU and memory limits:

services:
  web:
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 256M
        reservations:
          cpus: '0.25'
          memory: 128M

Note: deploy is mostly for Docker Swarm mode, but docker compose respects limits in v3.8 for local use.

Step 9: Troubleshooting Common Issues

  1. Port already in use: Stop the conflicting service or change the host port mapping.
  2. Container exits immediately: Check logs with docker compose logs . Common causes: missing dependencies, environment variable typos, or code errors.
  3. Cannot connect to database: Ensure the service name is used as host, and that both services are on the same network. Verify the database is healthy.
  4. Volume data persists unexpectedly: Use docker compose down -v to remove volumes along with containers. Be careful—this deletes persistent data.
  5. Build fails: Check your Dockerfile syntax and ensure all referenced files exist inside the build context.
  6. Environment variables not substituted: Make sure the .env file is in the same directory as docker-compose.yml and is not a hidden file with a different name.

Step 10: Useful Docker Compose Commands Reference

  • docker compose up: Create and start containers.
  • docker compose down: Stop and remove containers, networks.
  • docker compose ps: List running containers.
  • docker compose logs -f: Tail logs from all services.
  • docker compose exec : Execute a command in a running container.
  • docker compose build: Build or rebuild images.
  • docker compose pull: Pull latest images for services.
  • docker compose restart: Restart all services.
  • docker compose config: Validate and view the composed configuration.
  • docker compose top: Display running processes in each service.

Leave a Comment