Sudden Catastrophic Failure

CO_4X4 · December 21, 2021, 2:28pm

Got up this morning and no home automation was working, when I looked at the HA web interface all my devices were grayed out. I SSH into the HA rPi and try to restart the core and got:

Error: Can't create container from homeassistant: 409 Client Error for http+docker://localhost/v1.41/containers/create?name=homeassistant: Conflict ("Conflict. The container name "/homeassistant" is already in use by container "a19d8f32328b9d41a6bc2e54d61d917b0427ae249c52f2f1e0f7a55cc8af453d". You have to remove (or rename) that container to be able to reuse that name.")

Doing ha host reboot knocked me off the SSH immediately but wouldn’t reboot the system, I had to physically pull the plug on the rPi to get it back up. It is back up now but started out extremely sluggish and is starting to settle down again. Any idea what could cause this?

It’s not on SD, it’s booting from a brand new SSD drive, so it’s not an SD failure.

From the logs it occurred about 3am, when no automations are running and nothing is triggering anything and the logs look 100% normal, it just crashed.

e-raser · January 7, 2023, 6:10pm

Same for me when trying to update from 2022.10.5 using “ha core update --version=2022.11.5”.

Nothing new:

github.com/home-assistant/supervisor

Container in use errors when updating HA core version or add-ons

opened 03:40AM - 19 Jul 21 UTC

closed 11:28PM - 14 Sep 21 UTC

sparkydave1981

**Hardware Environment**  - [x] Generic x86-64 (Intel NUC) **Home Assistant OS release:** - [x] 2021.06.8  ``` arch: amd64 audio_input: None audio_output: None boot: true image: ghcr.io/home-assistant/generic-x86-64-homeassistant ip_address: 172.30.32.1 last_version: 2021.7.3 machine: generic-x86-64 port: 8123 ssl: false update_available: false version: 2021.7.3 version_latest: 2021.7.3 wait_boot: 600 watchdog: true ``` **Supervisor logs:**  ``` 21-07-19 11:26:43 INFO (SyncWorker_5) [supervisor.docker.interface] Attaching to ghcr.io/hassio-addons/unifi/amd64 with version 0.23.2 21-07-19 11:26:43 INFO (SyncWorker_10) [supervisor.docker.interface] Attaching to ghcr.io/hassio-addons/motioneye/amd64 with version 0.15.0 21-07-19 11:26:43 INFO (SyncWorker_11) [supervisor.docker.interface] Attaching to homeassistant/amd64-addon-zwave_js with version 0.1.29 21-07-19 11:26:43 INFO (SyncWorker_9) [supervisor.docker.interface] Attaching to ghcr.io/einschmidt/caddy-2/amd64 with version 1.0.3 21-07-19 11:26:43 INFO (SyncWorker_8) [supervisor.docker.interface] Attaching to f8f3f8ff/amd64-addon-gmusicproxy with version 0.5.1 21-07-19 11:26:43 INFO (MainThread) [supervisor.snapshots] Found 5 snapshot files 21-07-19 11:26:43 INFO (MainThread) [supervisor.discovery] Loaded 3 messages 21-07-19 11:26:43 INFO (MainThread) [supervisor.ingress] Loaded 3 ingress sessions 21-07-19 11:26:43 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.SETUP 21-07-19 11:26:43 INFO (MainThread) [supervisor.resolution.check] System checks complete 21-07-19 11:26:43 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.SETUP 21-07-19 11:26:43 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete 21-07-19 11:26:43 INFO (MainThread) [supervisor.jobs] 'ResolutionFixup.run_autofix' blocked from execution, system is not running - CoreState.SETUP 21-07-19 11:26:43 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.SETUP 21-07-19 11:26:43 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete 21-07-19 11:26:43 INFO (MainThread) [__main__] Running Supervisor 21-07-19 11:26:43 INFO (MainThread) [supervisor.hassos] Rauc: B - marked slot kernel.1 as good 21-07-19 11:26:43 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.INITIALIZE' starting 0 add-ons 21-07-19 11:26:43 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.SYSTEM' starting 2 add-ons 21-07-19 11:26:43 WARNING (MainThread) [supervisor.addons.options] Option 'anonymous' does not exist in the schema for Mosquitto broker (core_mosquitto) 21-07-19 11:26:43 INFO (SyncWorker_11) [supervisor.docker.interface] Cleaning addon_core_mosquitto application 21-07-19 11:26:43 WARNING (SyncWorker_11) [supervisor.addons.options] Option 'anonymous' does not exist in the schema for Mosquitto broker (core_mosquitto) 21-07-19 11:26:43 INFO (SyncWorker_11) [supervisor.docker.addon] Starting Docker add-on homeassistant/amd64-addon-mosquitto with version 6.0.1 21-07-19 11:26:43 INFO (SyncWorker_4) [supervisor.docker.addon] Starting Docker add-on homeassistant/amd64-addon-mariadb with version 2.4.0 21-07-19 11:26:44 INFO (MainThread) [supervisor.services.modules.mqtt] Set core_mosquitto as service provider for mqtt 21-07-19 11:26:45 INFO (MainThread) [supervisor.services.modules.mysql] Set core_mariadb as service provider for MySQL 21-07-19 11:26:46 INFO (MainThread) [supervisor.auth] Auth request from 'core_mosquitto' for 'hassio_mqtt' 21-07-19 11:26:46 INFO (MainThread) [supervisor.auth] Home Assistant not running, checking cache 21-07-19 11:26:48 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.SERVICES' starting 6 add-ons 21-07-19 11:26:48 INFO (SyncWorker_11) [supervisor.docker.interface] Cleaning addon_core_ssh application 21-07-19 11:26:48 INFO (SyncWorker_11) [supervisor.docker.addon] Starting Docker add-on homeassistant/amd64-addon-ssh with version 9.1.3 21-07-19 11:26:49 INFO (SyncWorker_4) [supervisor.docker.interface] Cleaning addon_a0d7b954_unifi application 21-07-19 11:26:49 INFO (SyncWorker_4) [supervisor.docker.addon] Starting Docker add-on ghcr.io/hassio-addons/unifi/amd64 with version 0.23.2 21-07-19 11:26:49 INFO (SyncWorker_9) [supervisor.docker.interface] Cleaning addon_a0d7b954_vscode application 21-07-19 11:26:49 INFO (SyncWorker_9) [supervisor.docker.addon] Starting Docker add-on ghcr.io/hassio-addons/vscode/amd64 with version 3.6.0 21-07-19 11:26:49 INFO (SyncWorker_5) [supervisor.docker.interface] Cleaning addon_6a3d001f_assistant_relay application 21-07-19 11:26:49 INFO (SyncWorker_5) [supervisor.docker.addon] Starting Docker add-on apipa169/amd64-hassio-assistant_relay with version None 21-07-19 11:26:49 WARNING (MainThread) [supervisor.addons.options] Option 'interface' does not exist in the schema for Samba share (core_samba) 21-07-19 11:26:49 INFO (SyncWorker_1) [supervisor.docker.interface] Cleaning addon_core_samba application 21-07-19 11:26:49 WARNING (SyncWorker_1) [supervisor.addons.options] Option 'interface' does not exist in the schema for Samba share (core_samba) 21-07-19 11:26:49 INFO (SyncWorker_1) [supervisor.docker.addon] Starting Docker add-on homeassistant/amd64-addon-samba with version 9.5.1 21-07-19 11:26:50 INFO (SyncWorker_11) [supervisor.docker.addon] Starting Docker add-on homeassistant/amd64-addon-zwave_js with version 0.1.29 21-07-19 11:26:50 INFO (MainThread) [supervisor.api.middleware.security] /network/interface/default/info access from core_samba 21-07-19 11:26:55 INFO (MainThread) [supervisor.core] Start Home Assistant Core 21-07-19 11:26:55 INFO (MainThread) [supervisor.homeassistant.module] Update pulse/client.config: /data/tmp/homeassistant_pulse 21-07-19 11:26:55 INFO (SyncWorker_9) [supervisor.docker.interface] Cleaning homeassistant application 21-07-19 11:26:58 ERROR (SyncWorker_9) [supervisor.docker] Image ghcr.io/home-assistant/generic-x86-64-homeassistant not exists for homeassistant 21-07-19 11:26:58 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.APPLICATION' starting 5 add-ons 21-07-19 11:26:58 INFO (SyncWorker_5) [supervisor.docker.interface] Cleaning addon_cebe7a76_hassio_google_drive_backup application 21-07-19 11:26:58 INFO (SyncWorker_5) [supervisor.docker.addon] Starting Docker add-on sabeechen/hassio-google-drive-backup-amd64 with version 0.104.3 21-07-19 11:26:58 INFO (SyncWorker_10) [supervisor.docker.interface] Cleaning addon_15ef4d2f_esphome application 21-07-19 11:26:58 INFO (SyncWorker_10) [supervisor.docker.addon] Starting Docker add-on esphome/esphome-hassio-amd64 with version 1.19.4 21-07-19 11:26:59 INFO (SyncWorker_11) [supervisor.docker.interface] Cleaning addon_00185a40_simplescheduler application 21-07-19 11:26:59 INFO (SyncWorker_11) [supervisor.docker.addon] Starting Docker add-on 00185a40/amd64-addon-simplescheduler with version 0.30.05 21-07-19 11:26:59 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning addon_a0d7b954_nodered application 21-07-19 11:26:59 INFO (SyncWorker_0) [supervisor.docker.addon] Starting Docker add-on ghcr.io/hassio-addons/node-red/amd64 with version 9.2.0 21-07-19 11:26:59 WARNING (SyncWorker_11) [supervisor.docker.addon] Run On Startup.d running with disabled protected mode! 21-07-19 11:26:59 INFO (SyncWorker_11) [supervisor.docker.interface] Cleaning addon_a1308ec6_run_on_startup_addon application 21-07-19 11:26:59 INFO (MainThread) [supervisor.api.middleware.security] /core/info access from cebe7a76_hassio_google_drive_backup 21-07-19 11:26:59 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/info access from cebe7a76_hassio_google_drive_backup 21-07-19 11:26:59 INFO (SyncWorker_11) [supervisor.docker.addon] Starting Docker add-on a1308ec6/amd64-addon-run_on_startup_addon with version 0.10 21-07-19 11:26:59 INFO (MainThread) [supervisor.api.middleware.security] /snapshots access from cebe7a76_hassio_google_drive_backup 21-07-19 11:26:59 INFO (MainThread) [supervisor.api.middleware.security] /snapshots/bc1a0f81/info access from cebe7a76_hassio_google_drive_backup 21-07-19 11:26:59 INFO (MainThread) [supervisor.api.middleware.security] /snapshots/e87a2030/info access from cebe7a76_hassio_google_drive_backup 21-07-19 11:26:59 INFO (MainThread) [supervisor.api.middleware.security] /snapshots/106a1d22/info access from cebe7a76_hassio_google_drive_backup 21-07-19 11:26:59 INFO (MainThread) [supervisor.api.middleware.security] /snapshots/971441c3/info access from cebe7a76_hassio_google_drive_backup 21-07-19 11:26:59 INFO (MainThread) [supervisor.api.middleware.security] /snapshots/ffc4ba25/info access from cebe7a76_hassio_google_drive_backup 21-07-19 11:27:04 INFO (MainThread) [supervisor.misc.tasks] All core tasks are scheduled 21-07-19 11:27:04 INFO (MainThread) [supervisor.core] Supervisor is up and running 21-07-19 11:27:04 INFO (MainThread) [supervisor.host.info] Updating local host information 21-07-19 11:27:04 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING 21-07-19 11:27:04 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON 21-07-19 11:27:04 WARNING (MainThread) [supervisor.addons.options] Option 'anonymous' does not exist in the schema for Mosquitto broker (core_mosquitto) 21-07-19 11:27:04 INFO (MainThread) [supervisor.host.services] Updating service information 21-07-19 11:27:04 INFO (MainThread) [supervisor.host.network] Updating local network information 21-07-19 11:27:04 WARNING (MainThread) [supervisor.addons.options] Option 'interface' does not exist in the schema for Samba share (core_samba) 21-07-19 11:27:04 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM 21-07-19 11:27:04 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE 21-07-19 11:27:04 INFO (MainThread) [supervisor.resolution.check] System checks complete 21-07-19 11:27:04 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING 21-07-19 11:27:05 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information 21-07-19 11:27:05 INFO (MainThread) [supervisor.host] Host information reload completed 21-07-19 11:27:08 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete 21-07-19 11:27:08 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING 21-07-19 11:27:08 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete 21-07-19 11:29:00 INFO (MainThread) [supervisor.api.middleware.security] /network/info access from core_ssh 21-07-19 11:29:00 INFO (MainThread) [supervisor.api.middleware.security] /network/info access from core_ssh 21-07-19 11:29:00 INFO (MainThread) [supervisor.api.middleware.security] /host/info access from core_ssh 21-07-19 11:29:00 INFO (MainThread) [supervisor.api.middleware.security] /core/info access from core_ssh 21-07-19 11:29:05 INFO (MainThread) [supervisor.api.middleware.security] /core/info access from core_ssh 21-07-19 11:29:23 INFO (MainThread) [supervisor.api.middleware.security] /core/stop access from core_ssh 21-07-19 11:29:32 INFO (MainThread) [supervisor.api.middleware.security] /core/update access from core_ssh 21-07-19 11:29:32 INFO (MainThread) [supervisor.homeassistant.core] Updating Home Assistant to version 2021.7.3 21-07-19 11:29:32 INFO (SyncWorker_7) [supervisor.docker.interface] Updating image ghcr.io/home-assistant/generic-x86-64-homeassistant:2021.7.2 to ghcr.io/home-assistant/generic-x86-64-homeassistant:2021.7.3 21-07-19 11:29:32 INFO (SyncWorker_7) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/generic-x86-64-homeassistant with tag 2021.7.3. 21-07-19 11:29:37 INFO (MainThread) [supervisor.homeassistant.core] Successful started Home Assistant 2021.7.3 21-07-19 11:34:41 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/info access from core_ssh 21-07-19 11:35:16 INFO (MainThread) [supervisor.api.middleware.security] /core/info access from core_ssh 21-07-19 11:36:09 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/logs access from core_ssh ``` **Journal logs:**  **Kernel logs:**  **Description of problem:**  Whenever I try to update HA Core to a new version I have major issues. I get errors relating to the Container being is use / name already existing. The same happens if I try to update an Add-on which is running. The only solution (temporary) has been to stop HA (or the add-on), update, then start again. In this case it sometimes works however it also requires a Host Reboot.

@CO_4X4 How did you solve it? Hardware issue? Or also Supervisor going crazy?

Update: Nothing helped.

Even ha su repair timed out with Post "http://supervisor/supervisor/repair": context deadline exceeded (Client.Timeout exceeded while awaiting headers).
Also ha host reboot gave Post "http://supervisor/host/reboot": context deadline exceeded (Client.Timeout exceeded while awaiting headers) so I really had to shutdown the host the hard way (power cable - pull, wait, plug) for the first time in more than 2 years.

Lesson learned: Dear Supervisor, if you’re screwed, everything is screwed.