How to Configure Docker Rotate Log on Amazon Linux 2

A growing e-commerce company needed to manage logs from its Docker containers on Amazon Linux 2. With thousands of orders processed daily, server logs grew quickly and became hard to handle.

This tutorial shows how to configure Docker Rotate Log on Amazon Linux 2 using a practical example. You will learn the concept and why log rotation matters for keeping systems healthy.

The E-Commerce Company’s Challenge

The e-commerce company’s microservices architecture used multiple Docker containers on Amazon Linux 2 instances. The business processed thousands of transactions daily, generating massive logs from payment processing, product catalog updates, and user authentication services.

Without proper log rotation, several issues can occur:

1. Insufficient disk space
2. Longer backup times
3. Reduced performance from increased I/O operations
4. Difficulty identifying and troubleshooting issues

The company needed an automated solution to manage these logs, so they configured Docker Rotate Log.

Configuring Docker Rotate Log on Amazon Linux 2

The e-commerce company solved this problem by configuring log rotation using Docker daemon options with --log-opt on their Amazon Linux 2 instances.

Follow these steps:

Step 1: Update Your System

Update your Amazon Linux 2 system packages:

sudo yum update -y

Step 2: Install and Start Docker

Install Docker:

sudo amazon-linux-extras install docker -y

Start the Docker service and enable it at system boot:

sudo systemctl start docker
sudo systemctl enable docker

Step 3: Configure Docker Daemon

Create the Docker daemon configuration file at /etc/docker/daemon.json:

sudo touch /etc/docker/daemon.json

Open the file in your preferred text editor and add:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "5"
  }
}

This configuration sets up a JSON log driver with two options:

• max-size: The maximum size of each log file before rotation. Set to 10 MB.
• max-file: The maximum number of log files to retain. Set to 5 files.

Adjust these values based on your requirements.

Step 4: Restart Docker Service

Restart the Docker service to apply your changes:

sudo systemctl restart docker.service

This approach shows how rotating logs helps maintain a healthy system. By implementing Docker Rotate Log on Amazon Linux 2, you get an automated solution that efficiently manages server logs from your services.

With this configuration, DevOps engineers can maintain system performance and troubleshoot issues without being overwhelmed by massive log volumes.

Alternative: Configuring Log Rotation Using logrotate for Docker Logs on Amazon Linux 2

You can also use logrotate instead of modifying the Docker daemon configuration. This method creates a custom logrotate configuration file for Docker container logs.

Step 1: Install logrotate

Update your Amazon Linux 2 system packages:

sudo yum update -y

Install logrotate:

sudo yum install logrotate -y

Step 2: Create a Custom Logrotate Configuration File for Docker

Create the file /etc/logrotate.d/docker-containers:

sudo touch /etc/logrotate.d/docker-containers

Open the file and add:

/var/lib/docker/containers/*/*.log {
    rotate 7
    daily
    compress
    size=50M
    missingok
    delaycompress
    copytruncate
}

This configuration includes:

• rotate: Number of times to rotate each log file before removal. Set to 7.
• daily: Rotate logs daily.
• compress: Compress rotated files.
• size=50M: Maximum size of each log file before rotation. Set to 50 MB.
• missingok: Do not report an error if a log file is missing.
• delaycompress: Postpone compression until after the second rotation.
• copytruncate: Truncate the original log file in place after making a copy.

Adjust these values based on your requirements.

Step 3: Check Log Rotation Configuration

Test your configuration with the --debug flag:

sudo logrotate --debug /etc/logrotate.conf

Check the output for any issues or errors.

Step 4: Force Log Rotation (Optional)

To force an immediate log rotation:

sudo logrotate --force /etc/logrotate.conf

Conclusion

This tutorial covered two solutions for managing Docker container logs on Amazon Linux 2: configuring log rotation using Docker daemon --log-opt options, and setting up log rotation with logrotate. Both methods work well for managing log files in production environments.

Choose the method that fits your project requirements. Configuring Docker daemon settings gives you direct control over log rotation through Docker. Using logrotate offers more flexibility and customization without changing Docker configurations.

Proper log rotation helps maintain system performance, optimize storage, troubleshoot issues effectively, and avoid problems from large logs.

Apply these concepts to your projects to keep your infrastructure healthy and manageable.