HA (and plugins) not restarting after docker-ce update

Elmardus · May 17, 2022, 9:31am

After every update of Docker-CE, the homeassistant container and plugin containers do not restart automatically. The observer and supervisor (and misc containers like dns, cli, etc.) are running normally.

I simulated a docker-ce update by running:
$ sudo apt install --reinstall docker-ce

This results in the following running/stopped containers:

The supervisor logs are included below:

22-05-17 08:51:28 INFO (MainThread) [supervisor.bootstrap] Initializing Supervisor Sentry
22-05-17 10:51:28 INFO (MainThread) [supervisor.bootstrap] Seting up coresys for machine: qemux86-64
22-05-17 10:51:28 INFO (SyncWorker_0) [supervisor.docker.supervisor] Attaching to Supervisor homeassistant/amd64-hassio-supervisor with version 2022.05.2
22-05-17 10:51:29 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.INITIALIZE
22-05-17 10:51:29 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-05-17 10:51:29 INFO (MainThread) [__main__] Setting up Supervisor
22-05-17 10:51:29 INFO (MainThread) [supervisor.api] Starting API on 172.30.32.2
22-05-17 10:51:30 INFO (MainThread) [supervisor.hardware.monitor] Started Supervisor hardware monitor
22-05-17 10:51:30 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface io.hass.os
22-05-17 10:51:30 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.systemd1
22-05-17 10:51:30 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.login1
22-05-17 10:51:30 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.hostname1
22-05-17 10:51:30 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.timedate1
22-05-17 10:51:31 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.NetworkManager
22-05-17 10:51:31 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface de.pengutronix.rauc
22-05-17 10:51:31 WARNING (MainThread) [supervisor.dbus.manager] Can't load dbus interface de.pengutronix.rauc: The name de.pengutronix.rauc was not provided by any .service files
22-05-17 10:51:31 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.resolve1
22-05-17 10:51:31 INFO (MainThread) [supervisor.host.info] Updating local host information
22-05-17 10:51:31 INFO (MainThread) [supervisor.host.services] Updating service information
22-05-17 10:51:31 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information
22-05-17 10:51:31 INFO (MainThread) [supervisor.host.manager] Host information reload completed
22-05-17 10:51:31 INFO (MainThread) [supervisor.host.network] Updating local network information
22-05-17 10:51:32 WARNING (MainThread) [supervisor.host.network] Requested to update interface enp2s0 which does not exist or is disabled.
22-05-17 10:51:32 INFO (MainThread) [supervisor.host.apparmor] Loading AppArmor Profiles: {'hassio-supervisor'}
22-05-17 10:51:32 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/amd64-hassio-cli with version 2022.05.0
22-05-17 10:51:32 INFO (MainThread) [supervisor.plugins.cli] Starting CLI plugin
22-05-17 10:51:32 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning hassio_cli application
22-05-17 10:51:33 INFO (SyncWorker_0) [supervisor.docker.cli] Starting CLI ghcr.io/home-assistant/amd64-hassio-cli with version 2022.05.0 - 172.30.32.5
22-05-17 10:51:33 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/amd64-hassio-dns with version 2022.04.1
22-05-17 10:51:33 INFO (MainThread) [supervisor.plugins.dns] Starting CoreDNS plugin
22-05-17 10:51:33 INFO (SyncWorker_1) [supervisor.docker.interface] Cleaning hassio_dns application
22-05-17 10:51:34 INFO (SyncWorker_1) [supervisor.docker.dns] Starting DNS ghcr.io/home-assistant/amd64-hassio-dns with version 2022.04.1 - 172.30.32.3
22-05-17 10:51:34 INFO (MainThread) [supervisor.plugins.dns] Updated /etc/resolv.conf
22-05-17 10:51:34 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/amd64-hassio-audio with version 2022.05.0
22-05-17 10:51:34 INFO (MainThread) [supervisor.plugins.audio] Starting Audio plugin
22-05-17 10:51:34 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning hassio_audio application
22-05-17 10:51:35 INFO (SyncWorker_0) [supervisor.docker.audio] Starting Audio ghcr.io/home-assistant/amd64-hassio-audio with version 2022.05.0 - 172.30.32.4
22-05-17 10:51:35 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/amd64-hassio-observer with version 2021.10.0
22-05-17 10:51:35 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/amd64-hassio-multicast with version 2022.02.0
22-05-17 10:51:35 INFO (MainThread) [supervisor.plugins.multicast] Starting Multicast plugin
22-05-17 10:51:35 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning hassio_multicast application
22-05-17 10:51:36 INFO (SyncWorker_0) [supervisor.docker.multicast] Starting Multicast ghcr.io/home-assistant/amd64-hassio-multicast with version 2022.02.0 - Host
22-05-17 10:51:36 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
22-05-17 10:51:36 INFO (SyncWorker_2) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/qemux86-64-homeassistant with version 2022.5.4
22-05-17 10:51:36 INFO (MainThread) [supervisor.homeassistant.secrets] Loaded 14 Home Assistant secrets
22-05-17 10:51:36 INFO (MainThread) [supervisor.os.manager] No Home Assistant Operating System found
22-05-17 10:51:38 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/git/a0d7b954 repository
22-05-17 10:51:38 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/git/45df7312 repository
22-05-17 10:51:38 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/git/cebe7a76 repository
22-05-17 10:51:38 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/core repository
22-05-17 10:51:40 INFO (MainThread) [supervisor.store] Loading add-ons from store: 65 all - 65 new - 0 remove
22-05-17 10:51:40 INFO (MainThread) [supervisor.addons] Found 7 installed add-ons
22-05-17 10:51:40 INFO (SyncWorker_2) [supervisor.docker.interface] Attaching to homeassistant/amd64-addon-mosquitto with version 6.1.2
22-05-17 10:51:40 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/hassio-addons/tasmoadmin/amd64 with version 0.17.0
22-05-17 10:51:40 INFO (SyncWorker_3) [supervisor.docker.interface] Attaching to ghcr.io/hassio-addons/vscode/amd64 with version 5.0.3
22-05-17 10:51:40 INFO (SyncWorker_2) [supervisor.docker.interface] Attaching to homeassistant/amd64-addon-rpc_shutdown with version 2.2
22-05-17 10:51:40 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/hassio-addons/node-red/amd64 with version 11.1.2
22-05-17 10:51:40 INFO (SyncWorker_3) [supervisor.docker.interface] Attaching to zigbee2mqtt/zigbee2mqtt-amd64 with version 1.25.1-1
22-05-17 10:51:40 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to sabeechen/hassio-google-drive-backup-amd64 with version 0.107.2
22-05-17 10:51:40 INFO (MainThread) [supervisor.backups.manager] Found 12 backup files
22-05-17 10:51:40 INFO (MainThread) [supervisor.discovery] Loaded 1 messages
22-05-17 10:51:40 INFO (MainThread) [supervisor.ingress] Loaded 0 ingress sessions
22-05-17 10:51:40 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.SETUP
22-05-17 10:51:40 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-05-17 10:51:40 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.SETUP
22-05-17 10:51:40 WARNING (MainThread) [supervisor.resolution.evaluations.base] Detected unsupported OS: Debian GNU/Linux 10 (buster) (more-info: https://www.home-assistant.io/more-info/unsupported/os)
22-05-17 10:51:40 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-05-17 10:51:40 INFO (MainThread) [supervisor.jobs] 'ResolutionFixup.run_autofix' blocked from execution, system is not running - CoreState.SETUP
22-05-17 10:51:40 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.SETUP
22-05-17 10:51:40 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-05-17 10:51:40 INFO (MainThread) [__main__] Running Supervisor
22-05-17 10:51:40 WARNING (MainThread) [supervisor.core] System running in a unsupported environment!
22-05-17 10:51:40 INFO (MainThread) [supervisor.jobs] 'OSManager.mark_healthy' blocked from execution, no Home Assistant OS available
22-05-17 10:51:40 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.INITIALIZE' starting 0 add-ons
22-05-17 10:51:40 INFO (MainThread) [supervisor.core] Supervisor reboot detected
22-05-17 10:51:40 INFO (MainThread) [supervisor.misc.tasks] All core tasks are scheduled
22-05-17 10:51:40 INFO (MainThread) [supervisor.core] Supervisor is up and running
22-05-17 10:51:40 INFO (MainThread) [supervisor.host.info] Updating local host information
22-05-17 10:51:40 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
22-05-17 10:51:40 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.DNS_SERVER_IPV6_ERROR/ContextType.DNS_SERVER
22-05-17 10:51:40 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-05-17 10:51:40 INFO (MainThread) [supervisor.host.services] Updating service information
22-05-17 10:51:41 INFO (MainThread) [supervisor.host.network] Updating local network information
22-05-17 10:51:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-05-17 10:51:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-05-17 10:51:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.DNS_SERVER_FAILED/ContextType.DNS_SERVER
22-05-17 10:51:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-05-17 10:51:41 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-05-17 10:51:41 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-05-17 10:51:42 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information
22-05-17 10:51:42 INFO (MainThread) [supervisor.host.manager] Host information reload completed
22-05-17 10:51:43 WARNING (MainThread) [supervisor.resolution.evaluations.base] Found unsupported images: {'linuxserver/sonarr', 'vaultwarden/server', 'haugene/transmission-openvpn', 'hkotel/mealie', 'nextcloud', 'linuxserver/tautulli', 'jc21/nginx-proxy-manager', 'ghcr.io/linuxserver/wireguard', 'plexinc/pms-docker', 'mysql', 'ghcr.io/linuxserver/mariadb', 'b4bz/homer', 'portainer/portainer-ce', 'linuxserver/bazarr', 'sctx/overseerr', 'bitnami/redis', 'lscr.io/linuxserver/prowlarr', 'saspus/duplicacy-web', 'itzg/minecraft-server', 'wordpress', 'redis', 'lscr.io/linuxserver/healthchecks', 'mariadb', 'lukechannings/moviematch', 'netdata/netdata', 'linuxserver/radarr', 'authelia/authelia'} (more-info: https://www.home-assistant.io/more-info/unsupported/software)
22-05-17 10:51:43 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-05-17 10:51:43 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-05-17 10:51:43 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete

When restarting the whole machine, the containers are all restarted as expected, this issue only occurs with a docker-ce update.
As a workaround I manually restart the homeassistant container, after which I can start the plugins using the plugin screen.

Is this a configuration issue? The supervisor logs don’t seem to indicate a direct reason why the containers are not restarted.

Elmardus · May 23, 2022, 12:58pm

I have updated the host machine to Debian 11 (bullseye) to be sure to exclude this as a cause for HA not rebooting automatically, but unfortunately it did not resolve the issue. Any other ideas what I can try to debug?

Tryfos · October 19, 2022, 8:10pm

Has this ever been addressed? I see a similar behavior after every docker-ce update.
After every docker-ce update all supervisor and containers seem to be running (I see that by doing a simple docker ps command). But I cannot connect to the GUI, ESPHome nodes do not connect etc.
I’ve tried:
ha supervisor restart (completes successfully, everything running-cannot connect to GUI)
sudo systemctl restart docker.service docker.socket containerd.service (the same as above)
docker restart (name of container). The same, the containers run fine but nothing more than that.
Only a complete reboot of the host seems to fix the problem after a docker-ce update. After the reboot everything runs fine and I can connect to the GUI.
How can I update docker-ce with HA (supervised) running fine even after the update, without having to reboot the host afterwards?

Hadatko · October 20, 2022, 6:40am

It looks like still an issue

tux1984 · December 7, 2022, 1:42pm

Same here with update of containerd. HA didn’t start properly, however all docker containers were running. Had to reboot the whole vm to make it work again. Any best practice command I could run to trigger a real restart of the whole HA stack?

dopey1620 · May 8, 2023, 9:19pm

I ran into the same issue and took me a while to figure out what’s going on. It looks like ha core container and the ha addons containers for some reason don’t restart (race condition or start order problem?)

I was able to recover without rebooting with the following

ha core restart
for addon in $(ha addon --raw-json | jq -r '.data.addons[].slug'); do ha addon restart ${addon}; done

above assumes you have jq installed to parse the json. It also restarts all addons - not just containerized ones. I’m not sure what data is in the addon response that would identify container addons. Maybe something with the state field.

I also did
docker restart $(docker ps -qa)
to restart all the docker containers and that seemed to work?
I’m not sure if there are any negative side effects to that solution though.