tooolbox/blog/Fixing Docker Daemon Startup Failures on Linux
devops2025-06-15ยท8ยท20/348

Fixing Docker Daemon Startup Failures on Linux

Troubleshooting guide for Docker daemon startup failures on Linux, including daemon.json issues, storage driver problems, and network configuration errors.

Fixing Docker Daemon Startup Failures on Linux

When the Docker daemon fails to start, all containers become unreachable. I encountered this on a production server after a kernel update left containerd in an inconsistent state. This guide covers the most common Docker daemon issues and their solutions.

Environment

  • OS: Ubuntu 22.04 LTS
  • Docker version: 24.0.5
  • containerd version: 1.7.2
  • Storage driver: overlay2
$ docker --version
Docker version 24.0.5, build ccd99bc

$ containerd --version
containerd containerd.io 1.7.2 07d38ee

The Problem

$ sudo systemctl status docker
โ— docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sat 2025-06-15 10:00:00 WET; 5min ago
Triggered by: โ— docker.socket - Docker Socket for the API
    Process: 1234 ExecStartPre=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
    Process: 1235 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)

Jun 15 10:00:00 server systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Jun 15 10:00:00 server dockerd[1234]: failed to start daemon: Error initializing network controller: error creating default bridge network: 
Jun 15 10:00:00 server dockerd[1234]: unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid character '}' looking for beginning of value

Analysis

Step 1: Check Docker Daemon Logs

# Check detailed logs
$ sudo journalctl -u docker.service --no-pager -n 50
-- Logs begin at Sat 2025-06-15 08:00:00 WET. --
Jun 15 10:00:00 server dockerd[1234]: time="2025-06-15T10:00:00Z" level=error msg="failed to start daemon"
Jun 15 10:00:00 server dockerd[1234]: Error initializing network controller: error creating default bridge network
Jun 15 10:00:00 server dockerd[1234]: unable to configure the Docker daemon with file /etc/docker/daemon.json
Jun 15 10:00:00 server dockerd[1234]: invalid character '}' looking for beginning of value

Step 2: Check daemon.json

$ cat /etc/docker/daemon.json
{
  "storage-driver": "overlay2",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "dns": ["8.8.8.8", "8.8.4.4"]
}    <-- Extra closing brace
}

There is a syntax error - an extra closing brace.

Step 3: Check containerd Status

$ sudo systemctl status containerd
โ— containerd.service - containerd container runtime
     Loaded: loaded (/lib/systemd/system/containerd.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2025-06-15 08:00:00 WET; 2h ago

# Check containerd socket
$ sudo ls -la /run/containerd/containerd.sock
srw-rw---- 1 root root 0 Jun 15 08:00 /run/containerd/containerd.sock

Step 4: Check System Resources

# Check disk space
$ df -h /var/lib/docker
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       50G   48G    2G  96% /var/lib/docker

# Disk almost full - this could cause issues

Solution

Fix 1: Correct daemon.json Syntax

# Fix the JSON syntax
$ sudo nano /etc/docker/daemon.json

# Corrected content:
{
  "storage-driver": "overlay2",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "dns": ["8.8.8.8", "8.8.4.4"]
}

# Validate JSON
$ python3 -m json.tool /etc/docker/daemon.json
{
    "storage-driver": "overlay2",
    ...
}

# Restart Docker
$ sudo systemctl restart docker

Fix 2: Free Disk Space

# Check Docker disk usage
$ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          15        5         8.5GB     6.2GB (73%)
Containers      10        5         2.1GB     1.5GB (71%)
Local Volumes   8         3         1.2GB     800MB (66%)
Build Cache     25        0         3.4GB     3.4GB (100%)

# Clean up unused resources
$ docker system prune -a --volumes
WARNING! This will remove:
  - all stopped containers
  - all networks not used by at least one container
  - all images without at least one container associated to them
  - all build cache

# Verify space freed
$ df -h /var/lib/docker
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       50G   35G   15G  70% /var/lib/docker

Fix 3: Reset containerd State

# Stop Docker
$ sudo systemctl stop docker

# Stop containerd
$ sudo systemctl stop containerd

# Remove containerd state
$ sudo rm -rf /run/containerd/*
$ sudo rm -rf /var/lib/containerd/*

# Restart containerd
$ sudo systemctl start containerd

# Restart Docker
$ sudo systemctl start docker

Fix 4: Check and Fix Network Configuration

# Remove problematic Docker networks
$ sudo ip link show docker0
# Check if bridge exists

# Remove stale bridge
$ sudo ip link delete docker0 2>/dev/null

# Restart Docker to recreate
$ sudo systemctl restart docker

# Verify new bridge
$ docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
abc123def456   bridge    bridge    local

Fix 5: Reinstall Docker (Nuclear Option)

# Stop Docker
$ sudo systemctl stop docker
$ sudo systemctl stop containerd

# Remove Docker packages
$ sudo apt purge -y docker docker-engine docker.io containerd runc

# Remove Docker data (WARNING: destroys all containers)
$ sudo rm -rf /var/lib/docker
$ sudo rm -rf /var/lib/containerd

# Reinstall
$ sudo apt update
$ sudo apt install -y docker.io containerd

# Configure and start
$ sudo systemctl enable --now docker containerd

Verify Docker is Running

$ sudo systemctl status docker
โ— docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2025-06-15 10:30:00 WET; 2min ago

# Test with a container
$ docker run --rm hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.

Prevention

# Always validate daemon.json before restarting
$ python3 -c "import json; json.load(open('/etc/docker/daemon.json'))"

# Monitor disk usage
$ docker system prune -af --volumes --filter "until=168h"

# Set up disk alerts
#!/bin/bash
USAGE=$(df /var/lib/docker --output=pcent | tail -1 | tr -d '% ')
if [ "$USAGE" -gt 85 ]; then
    echo "Docker disk usage at ${USAGE}%" | mail -s "Docker Disk Alert" admin@example.com
fi

Lessons Learned

  1. Always validate JSON before restarting Docker - use python3 -m json.tool.
  2. Monitor /var/lib/docker disk usage - Docker images accumulate quickly.
  3. Keep containerd healthy - Docker depends on it, and a broken containerd means broken Docker.
  4. Use docker system prune regularly to prevent disk exhaustion.
  5. Test daemon.json changes with dockerd --validate before restarting the service.

This blog does not accept any external sponsorships, affiliate marketing, or ad revenue.