Docker errors - Sometimes it doesn't work as expected...

mail

Error response from daemon: Get https://registry-1.docker.io/v2/: proxyconnect tcp: dial tcp 127.0.0.1:8080: connect: connection refused

Situation

I got this error while installing Docker on Debian Buster, at the "check" stage, i.e. after running docker run hello-world : 
docker: Error response from daemon: Get https://registry-1.docker.io/v2/: proxyconnect tcp: dial tcp 127.0.0.1:8080: connect: connection refused

Details

Here's my /etc/systemd/system/docker.service.d/proxy.conf : 
[Service]
Environment="http_proxy=http://localhost:8080/" "https_proxy=http://localhost:8080/" "ftp_proxy=http://localhost:8080"
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS
I don't know whether this is the result :
  • of a default install
  • or of previous changes/hacks made by my old self/someone else

Solution

Just specify the existing http_proxy and https_proxy environment variables into the unit file, as root : 
  1. cat << EOF > /etc/systemd/system/docker.service.d/proxy.conf
[Service]
Environment="http_proxy=$http_proxy" "https_proxy=$https_proxy" "ftp_proxy=http://localhost:8080"
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// \$DOCKER_OPTS
EOF
  2. reload the updated unit file :
    systemctl daemon-reload
  3. restart the Docker daemon :
    systemctl restart docker
  4. this should work, now :
    docker run hello-world
mail

Command "daemon" is deprecated, and will be removed in Docker 17.12. Please run `dockerd` directly

Situation

  1. Everything looks fine when starting Docker :
    systemctl status docker.service
    ● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─proxy.conf
   Active: active (running) since Wed 2018-01-24 12:46:59 CET; 11s ago		use this with --since below
     Docs: https://docs.docker.com
 Main PID: 4777 (dockerd)
    Tasks: 17
   Memory: 30.6M
      CPU: 362ms
   CGroup: /system.slice/docker.service
           ├─4777 dockerd -H fd://
           └─4786 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libcontainerd/container

Jan 24 12:46:59 myServer docker[4777]: time="2018-01-24T12:46:59.199165764+01:00" level=warning msg="Your kernel does not support swap memory limit"
Jan 24 12:46:59 myServer docker[4777]: time="2018-01-24T12:46:59.199209411+01:00" level=warning msg="Your kernel does not support cgroup rt period"
Jan 24 12:46:59 myServer docker[4777]: time="2018-01-24T12:46:59.199219535+01:00" level=warning msg="Your kernel does not support cgroup rt runtime"
Jan 24 12:46:59 myServer docker[4777]: time="2018-01-24T12:46:59.199544739+01:00" level=info msg="Loading containers: start."
Jan 24 12:46:59 myServer docker[4777]: time="2018-01-24T12:46:59.378657775+01:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip ca
Jan 24 12:46:59 myServer docker[4777]: time="2018-01-24T12:46:59.443060341+01:00" level=info msg="Loading containers: done."
Jan 24 12:46:59 myServer docker[4777]: time="2018-01-24T12:46:59.453716413+01:00" level=info msg="Daemon has completed initialization"
Jan 24 12:46:59 myServer docker[4777]: time="2018-01-24T12:46:59.453767567+01:00" level=info msg="Docker daemon" commit=89658be graphdriver=overlay version=17.05.0-ce
Jan 24 12:46:59 myServer docker[4777]: time="2018-01-24T12:46:59.462163955+01:00" level=info msg="API listen on /var/run/docker.sock"
Jan 24 12:46:59 myServer systemd[1]: Started Docker Application Container Engine.
  2. ... but :
    journalctl -u docker --since '2018-01-24 12:46'
    Jan 24 12:46:58 myServer systemd[1]: Starting Docker Application Container Engine...
Jan 24 12:46:58 myServer docker[4777]: Command "daemon" is deprecated, and will be removed in Docker 17.12. Please run `dockerd` directly.

Solution

  1. You'll have to edit a configuration file which is read by systemd while starting Docker. On my side, this was /etc/systemd/system/docker.service.d/proxy.conf (not installed / configured by myself), but /etc/systemd/system/docker.service.d/whatever.conf should do the trick
  2. Change the line : 
    ExecStart=/usr/bin/docker daemon -H fd:// $DOCKER_OPTS
    into : 
    ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS
  3. Make systemd aware of the change :
    systemctl daemon-reload
  4. Restart Docker (date is there to filter logs at the next step) :
    date; systemctl restart docker.service
    Wed Jan 24 12:49:53 CET 2018
    Nothing more, meaning Docker started successfully
  5. journalctl -u docker --since '2018-01-24 12:49' | grep deprecated
    Nothing found : the warning has disappeared.
mail

Error starting daemon: error initializing graphdriver: driver not supported

Situation

systemctl start docker.service
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
systemctl status docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─proxy.conf
   Active: failed (Result: exit-code) since Wed 2018-01-24 12:09:21 CET; 15s ago		use this with --since below
     Docs: https://docs.docker.com
  Process: 4216 ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS (code=exited, status=1/FAILURE)
 Main PID: 4216 (code=exited, status=1/FAILURE)
      CPU: 86ms

Jan 24 12:09:20 myServer systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Jan 24 12:09:20 myServer systemd[1]: Failed to start Docker Application Container Engine.
Jan 24 12:09:20 myServer systemd[1]: docker.service: Unit entered failed state.
Jan 24 12:09:20 myServer systemd[1]: docker.service: Failed with result 'exit-code'.
Jan 24 12:09:21 myServer systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jan 24 12:09:21 myServer systemd[1]: Stopped Docker Application Container Engine.
Jan 24 12:09:21 myServer systemd[1]: docker.service: Start request repeated too quickly.
Jan 24 12:09:21 myServer systemd[1]: Failed to start Docker Application Container Engine.
Jan 24 12:09:21 myServer systemd[1]: docker.service: Unit entered failed state.
Jan 24 12:09:21 myServer systemd[1]: docker.service: Failed with result 'exit-code'.
journalctl -u docker --since '2018-01-24 12:09'
Jan 24 12:09:17 myServer systemd[1]: Starting Docker Application Container Engine...
Jan 24 12:09:17 myServer dockerd[4176]: time="2018-01-24T12:09:17.260224422+01:00" level=info msg="libcontainerd: new containerd process, pid: 4183"
Jan 24 12:09:18 myServer dockerd[4176]: time="2018-01-24T12:09:18.264676211+01:00" level=warning msg="failed to rename /var/lib/docker/tmp for background deletion: %!s(<nil>). Deleting synchr
Jan 24 12:09:18 myServer dockerd[4176]: time="2018-01-24T12:09:18.278198186+01:00" level=error msg="[graphdriver] prior storage driver aufs failed: driver not supported"
Jan 24 12:09:18 myServer dockerd[4176]: Error starting daemon: error initializing graphdriver: driver not supported
Jan 24 12:09:18 myServer systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Jan 24 12:09:18 myServer systemd[1]: Failed to start Docker Application Container Engine.
Jan 24 12:09:18 myServer systemd[1]: docker.service: Unit entered failed state.
Jan 24 12:09:18 myServer systemd[1]: docker.service: Failed with result 'exit-code'.
Jan 24 12:09:18 myServer systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jan 24 12:09:18 myServer systemd[1]: Stopped Docker Application Container Engine.
 same block repeated 3 times, then :
Jan 24 12:09:21 myServer systemd[1]: docker.service: Start request repeated too quickly.
Jan 24 12:09:21 myServer systemd[1]: Failed to start Docker Application Container Engine.
Jan 24 12:09:21 myServer systemd[1]: docker.service: Unit entered failed state.
Jan 24 12:09:21 myServer systemd[1]: docker.service: Failed with result 'exit-code'.

Details

Looks like a Debian bug (source) :

Unfortunately, this is because as soon as the kernel went to 4.0, overlayfs was merged upstream, and the Debian kernel team dropped the AUFS patches, so 4.0+ kernels in Debian no longer support AUFS and there's not a lot we can do to change or fix that (nor much upstream can do), and probably not much they should do, arguably (since the AUFS patches aren't exactly a source of warm fuzzies with upstream kernel developers). Even migration of the data would be nearly impossible without access to a kernel that did have AUFS long enough to transfer the data into a different storage driver.

Solution

  1. Add to /etc/docker/daemon.json (create it if missing) (source (with obsolete method), up-to-date method) : 
    {
    "storage-driver": "overlay"
}
  2. date; systemctl start docker.service
    date will help filter logs hereafter
    Wed Jan 24 12:46:58 CET 2018
  3. see also