Docker install upgrade 2023.2 -> 2023.5 fails

Installation
#1

Hi,
I am running HA in a docker setup with /config mounted as a docker volume. Until 2023.2 my working upgrade process always was

  • shutdown current HA
  • copy volume
  • start new HA container, with copied /config volume

when I try this to update to 2023.5 (also tried 2023.3 and 2023.4) the new HA comes up in one of two messed up states.
It either
directs me to the onboarding page and after that to an empty HA.
or
brings up the login page (old credentials work), but dashboards and devices/entities/integrations (none of which are listed in the breaking changes) are gone or in a broken state.

I already disabled everything in custom_components to without any improvement. The logs do not show any indication of failure.
I am out of ideas how to fix this upgrade and appreciate any help, thanks!

#2

Did you try mounting your volume in, e.g., a busybox container to check its content?

#3

The volumes are accessible to me on the host filesystem. I can see/edit files on the host, this is also how I copy the contents of the old volume to the new one.
I can also see the new HA instance working on the existing database (migrating), writing logs and reading my configuration.yaml .
So seeing this and since it is a plain host file copy I am quite confident that the files are ok.

#4

Ok. I though you were using docker volumes rather than bind mounted ones.
Same setup, and no issue in years…

Maybe (re)try a sequential one, taking a new 2023.2 backup, and upgrading to 2023.3.
Be sure to check the ha / docker logs

#5

What I am using should be volume mounts, I am managing them through portainer and they are attached by volume mount not bind (with driver&scope local). I already tried to install 2023.3 but was not sure if I had the configuration completely stripped down at that point so I retried just now.
Again the same result (currently the onboarding screen).

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun home-assistant (no readiness notification)
s6-rc: info: service legacy-services successfully started
2023-06-07 07:16:35.187 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Lampen Thomas' failed to setup triggers and has been disabled: Unknown device '1fa06357f3a98a3d2cf7cca10cb2b261'
2023-06-07 07:16:35.192 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Wohnzimmerlampe Zig' failed to setup triggers and has been disabled: Unknown device 'd01ca01ccfd036bb997e5bc3be2e3f97'
2023-06-07 07:16:35.192 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Wohnzimmer und Treppe aus' failed to setup triggers and has been disabled: Unknown device 'd01ca01ccfd036bb997e5bc3be2e3f97'
2023-06-07 07:16:35.193 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'MarkiseZig' failed to setup triggers and has been disabled: Unknown device 'ff08ece1c0f7bb1caf22264149e7b29a'
2023-06-07 07:16:35.193 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Treppenlicht' failed to setup triggers and has been disabled: Unknown device 'd01ca01ccfd036bb997e5bc3be2e3f97'
2023-06-07 07:16:35.337 WARNING (MainThread) [homeassistant.components.homeassistant.triggers.numeric_state] Error initializing 'Ebike Ladegeraet Ausschalten' trigger: In 'numeric_state' condition: unknown entity sensor.dose_garage_energy_power
2023-06-07 07:16:35.338 WARNING (Recorder) [homeassistant.components.recorder.migration] Database is about to upgrade from schema version: 33 to: 35
2023-06-07 07:16:35.342 WARNING (MainThread) [homeassistant.components.homeassistant.triggers.numeric_state] Error initializing 'co2thomasCalibrate' trigger: In 'numeric_state' condition: unknown entity sensor.co2thomas_ppm
2023-06-07 07:16:35.347 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding columns created_ts, start_ts, last_reset_ts to table statistics. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 07:16:35.350 WARNING (MainThread) [homeassistant.components.automation] Error evaluating condition in 'sync_solar_retained':
In 'condition':
  In 'or' (item 1 of 2):
    In 'state':
      In 'state' condition: unknown entity sensor.soxsolar_yieldtotal
  In 'or' (item 2 of 2):
    In 'state':
      In 'state' condition: unknown entity sensor.soxsolar_yieldtotal
2023-06-07 07:16:35.360 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding columns created_ts, start_ts, last_reset_ts to table statistics_short_term. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 07:16:35.373 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_statistics_start_ts` to database. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 07:16:35.958 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_statistics_statistic_id_start_ts` to database. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 07:16:36.722 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_statistics_short_term_start_ts` to database. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 07:16:36.941 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_statistics_short_term_statistic_id_start_ts` to database. Note: this can take several minutes on large databases and slow computers. Please be patient!

The errors concerning the automations seem to be related to zigbee devices connected through ZHA.
sync_solar_retained refers to a template sensor which I commented out to prevent errors due to changes in the template value processing in one of the updates.

#6

IIRC, there was a recent DB upgrade which really took a long time.
Maybe let it start for a while without trying to get in…

#7

I’ll try that thanks. My current DB is some 1.1 Gig .

I just hope I can manage to properly isolate the updating instance from the live one without interfering too much :smiley: . I put it in its own network now with no ports mapped to external network.

Amazing how much stuff I accumulated in HA over the years.

#8

Mmm… What do you mean. Using some “blue/green” setup?
Is your DB the default SQLite, or?

#9

I try to keep my current instance up while running the new one on a copy of the data in an isolated network. So yeah kind of blue/green because I do not want HA to be down so long during the day :smiley: .

Yes, my DB is the default SQlite.

– edit:
so far

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun home-assistant (no readiness notification)
s6-rc: info: service legacy-services successfully started
2023-06-07 12:23:12.178 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-06-07 12:23:12.179 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration weback_vacuum which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-06-07 12:23:12.179 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration govee which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-06-07 12:23:24.347 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Lampen Thomas' failed to setup triggers and has been disabled: Unknown device '1fa06357f3a98a3d2cf7cca10cb2b261'
2023-06-07 12:23:24.350 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Wohnzimmerlampe Zig' failed to setup triggers and has been disabled: Unknown device 'd01ca01ccfd036bb997e5bc3be2e3f97'
2023-06-07 12:23:24.353 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Wohnzimmer und Treppe aus' failed to setup triggers and has been disabled: Unknown device 'd01ca01ccfd036bb997e5bc3be2e3f97'
2023-06-07 12:23:24.354 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'MarkiseZig' failed to setup triggers and has been disabled: Unknown device 'ff08ece1c0f7bb1caf22264149e7b29a'
2023-06-07 12:23:24.354 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Treppenlicht' failed to setup triggers and has been disabled: Unknown device 'd01ca01ccfd036bb997e5bc3be2e3f97'
2023-06-07 12:23:25.656 WARNING (Thread-2 (run_forever)) [websocket] websocket connected
2023-06-07 12:23:26.580 WARNING (MainThread) [homeassistant.components.homeassistant.triggers.numeric_state] Error initializing 'Ebike Ladegeraet Ausschalten' trigger: In 'numeric_state' condition: unknown entity sensor.dose_garage_energy_power
2023-06-07 12:23:26.581 WARNING (Recorder) [homeassistant.components.recorder.migration] Database is about to upgrade from schema version: 33 to: 41
2023-06-07 12:23:26.584 WARNING (MainThread) [homeassistant.components.homeassistant.triggers.numeric_state] Error initializing 'co2thomasCalibrate' trigger: In 'numeric_state' condition: unknown entity sensor.co2thomas_ppm
2023-06-07 12:23:26.588 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding columns created_ts, start_ts, last_reset_ts to table statistics. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:26.593 WARNING (MainThread) [homeassistant.components.automation] Error evaluating condition in 'sync_solar_retained':
In 'condition':
  In 'or' (item 1 of 2):
    In 'state':
      In 'state' condition: unknown entity sensor.soxsolar_yieldtotal
  In 'or' (item 2 of 2):
    In 'state':
      In 'state' condition: unknown entity sensor.soxsolar_yieldtotal
2023-06-07 12:23:26.603 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding columns created_ts, start_ts, last_reset_ts to table statistics_short_term. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:26.614 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_statistics_start_ts` to table `statistics`. Note: this can take several minutes on large databases and slow computers. Please be patient!

3-06-07 12:23:27.210 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_statistics_statistic_id_start_ts` to table `statistics`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:27.965 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_statistics_short_term_start_ts` to table `statistics_short_term`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:28.188 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_statistics_short_term_statistic_id_start_ts` to table `statistics_short_term`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:40.504 WARNING (Recorder) [homeassistant.components.recorder.migration] Upgrade to version 34 done
2023-06-07 12:23:40.506 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_statistics_statistic_id_start` from table `statistics`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:40.663 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_statistics_short_term_statistic_id_start` from table `statistics_short_term`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:40.757 WARNING (Recorder) [homeassistant.components.recorder.migration] Upgrade to version 35 done
2023-06-07 12:23:40.757 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding columns context_id_bin, context_user_id_bin, context_parent_id_bin to table states. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:40.761 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding columns context_id_bin, context_user_id_bin, context_parent_id_bin to table events. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:40.764 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_events_context_id_bin` to table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:40.772 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_states_context_id_bin` to table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:42.838 WARNING (Recorder) [homeassistant.components.recorder.migration] Upgrade to version 36 done
2023-06-07 12:23:42.839 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding columns event_type_id to table events. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:42.840 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_events_event_type_time_fired_ts` from table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:42.844 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_events_event_type_id_time_fired_ts` to table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:42.855 WARNING (Recorder) [homeassistant.components.recorder.migration] Upgrade to version 37 done
2023-06-07 12:23:42.855 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding columns metadata_id to table states. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:42.857 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_states_metadata_id_last_updated_ts` to table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.706 WARNING (Recorder) [homeassistant.components.recorder.migration] Upgrade to version 38 done
2023-06-07 12:23:45.707 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_events_event_type_time_fired_ts` from table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.708 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_events_event_type` from table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.709 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_events_event_type_time_fired` from table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.710 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_events_time_fired` from table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.712 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_events_context_user_id` from table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.713 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_events_context_parent_id` from table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.714 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_entity_id_last_updated` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.716 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_last_updated` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.717 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_entity_id` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.719 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_context_user_id` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.720 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_context_parent_id` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.722 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_created_domain` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.723 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_entity_id_created` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.724 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `states__state_changes` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.726 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `states__significant_changes` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.727 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_entity_id_created` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.728 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_statistics_statistic_id_start` from table `statistics`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.729 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_statistics_short_term_statistic_id_start` from table `statistics_short_term`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.731 WARNING (Recorder) [homeassistant.components.recorder.migration] Upgrade to version 39 done
2023-06-07 12:23:45.732 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_events_event_type_id` from table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.733 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_metadata_id` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.734 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_statistics_metadata_id` from table `statistics`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.785 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_statistics_short_term_metadata_id` from table `statistics_short_term`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.818 WARNING (Recorder) [homeassistant.components.recorder.migration] Upgrade to version 40 done
2023-06-07 12:23:45.819 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_event_types_event_type` to table `event_types`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.820 WARNING (Recorder) [homeassistant.components.recorder.migration] Index ix_event_types_event_type already exists on event_types, continuing
2023-06-07 12:23:45.820 WARNING (Recorder) [homeassistant.components.recorder.migration] Adding index `ix_states_meta_entity_id` to table `states_meta`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:45.822 WARNING (Recorder) [homeassistant.components.recorder.migration] Index ix_states_meta_entity_id already exists on states_meta, continuing
2023-06-07 12:23:45.824 WARNING (Recorder) [homeassistant.components.recorder.migration] Upgrade to version 41 done
2023-06-07 12:23:53.040 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_statistics_start` from table `statistics`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:53.097 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_statistics_short_term_start` from table `statistics_short_term`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:23:54.877 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_events_context_id` from table `events`. Note: this can take several minutes on large databases and slow computers. Please be patient!
2023-06-07 12:27:25.233 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_context_id` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!

3-06-07 12:28:35.406 WARNING (Recorder) [homeassistant.components.recorder.migration] Dropping index `ix_states_entity_id_last_updated_ts` from table `states`. Note: this can take several minutes on large databases and slow computers. Please be patient!
#10

So it’s indeed still migrating after 5 minutes :wink:

#11

So I just switched off the old instance just now and brought the updated one into my network. The result is again some weird limbo state where my user still works but all the rest is gone entities/devices/dashboards :frowning:

#12

Did you try to start your “backup” with the same version as the “original”, i.e. without any upgrade involved?
If that doesn’t work, it might hint to an issue with your procedure…

1 Like
#13

wow ok that at least was unexpected…it actually seems to be a problem with the procedure… currently I do not know what should be the issue, but my clone instance (all the same as original, working on /config copy) just sent me to the onboarding page.

…

some tests later I found that copying the volume contents using cp -rp [src] [dst] is not the correct way…using cp -a [src] [dst] produced a working copy.
Seems like this was the first time I tried an update since moving from synology-docker to portainer.

Thank you so much for your patience and good hints @koying

#14

My process on Synology DSM Docker with /config mapped to /volume1/docker/homeassistant/config on the NAS is:

  • download latest HA image
  • stop container
  • reset container (often generates a timeout error on first try)
  • restart container

On Portainer, it is:

  • stop container
  • recreate container inc pull new image
  • start container

Both of these obviously result in some downtime, but usually work without any issues.

image
image694Ă—161 3.03 KB

#15

In your case you do not have a working backup in case something is wrong with the update process or even the new version. I always like to have my last working version as a standby which is why I run the new version on a copy of the data - so the “old” setup can always be restored.

1 Like
#16

Understood — I’ve never had a problem yet, and the /config folder is backed up to Amazon S3 via cloud sync, but I take your point.

#17

:champagne:

Smells like your .storage hidden directory might have been left out, which contains most of the meat of the configuration nowadays.