Docker migration problem from Raspberry Pi to NUC

klaaskox · June 30, 2024, 5:51am

I have been running HomeAssistant for years on a Raspberry PI 4. It runs in a docker container managed by Portainer. It also runs a zwavejs2mqtt container.

Now I want to migrate everything to a NUC. Here is what I did:

I installed docker
I installed portainer
copied all config volumes to the new system
created the docker containers with the correct volume mapping

Now, when I open the zwjs2mqtt admin panel on the NUC, it shows the current information. However, when I open Home Assistant, it shows the onboarding process. It doesn’t seem to recognize the config folder I copied and mapped to the container.

What could be the cause of this problem?

fleskefjes · June 30, 2024, 6:38am

To me it sounds like you haven’t done the mapping correctly. One way to test this is to stop the new container, clean out all files in the folder you have mapped, and then start it again. If the mapping is correct it should fill the folder with files.

klaaskox · June 30, 2024, 7:06am

Thanks for your response. I tried this, and now the empty config directory fills with HA-files. I guess this means the mapping is correct.

In both cases I get this onboarding screen. I assume I should not get this when I copied the existing config folder?

fleskefjes · June 30, 2024, 7:12am

That is correct. Can you show the contents of the folder after you’ve copied over the data?

klaaskox · June 30, 2024, 7:13am

 28K -rwxr-xr-x  1 root     root      25K Jun 30 08:56 automations.yaml
4.0K drwxr-xr-x  2 root     root     4.0K Jun 30 08:56 backups
4.0K drwxr-xr-x  4 root     root     4.0K Jun 30 08:56 blueprints
4.0K drwxr-xr-x  2 root     root     4.0K Jun 30 08:56 .cloud
4.0K -rwxr-xr-x  1 root     root     3.1K Jun 30 08:56 configuration.yaml
4.0K drwxr-xr-x  4 root     root     4.0K Jun 30 08:56 custom_components
4.0K drwxr-xr-x  3 root     root     4.0K Jun 30 08:56 custom_zha_quirks
4.0K -rwxr-xr-x  1 root     root     2.4K Jun 30 08:56 dashboard-home.yaml
4.0K drwxr-xr-x  2 root     root     4.0K Jun 30 08:56 deps
4.0K -rwxr-xr-x  1 root     root        8 Jun 30 08:56 .HA_VERSION
 35M -rw-r--r--  1 root     root      35M Jun 30 08:56 home-assistant.log
908K -rw-r--r--  1 root     root     906K Jun 30 08:56 home-assistant.log.1
   0 -rw-r--r--  1 root     root        0 Jun 30 08:56 home-assistant.log.fault
1.3G -rwxr-xr-x  1 root     root     1.3G Jun 30 08:56 home-assistant_v2.db
 64K -rwxr-xr-x  1 root     root      64K Jun 30 08:56 home-assistant_v2.db-shm
4.0M -rwxr-xr-x  1 root     root     4.0M Jun 30 08:56 home-assistant_v2.db-wal
4.0K drwxr-xr-x  3 root     root     4.0K Jun 30 08:56 my_dashboard
 16K -rwxr-xr-x  1 root     root      14K Jun 30 08:56 scenes.yaml
4.0K drwxr-xr-x  3 root     root     4.0K Jun 30 08:56 scripts
4.0K -rwxr-xr-x  1 root     root      157 Jun 30 08:56 scripts.yaml
4.0K -rwxr-xr-x  1 root     root      161 Jun 30 08:56 secrets.yaml
4.0K drwxr-xr-x  3 root     root     4.0K Jun 30 08:56 .storage
4.0K drwxr-xr-x  3 root     root     4.0K Jun 30 08:56 themes
4.0K drwxr-xr-x  2 root     root     4.0K Jun 30 08:56 tts
 44K -rwxr-xr-x  1 root     root      41K Jun 30 08:56 ui-lovelace.yaml
 16K -rwxr-xr-x  1 root     root      13K Jun 30 08:56 wled.yaml
4.0K drwxr-xr-x  6 root     root     4.0K Jun 30 08:56 www
508K -rwxr-xr-x  1 root     root     508K Jun 30 08:56 zigbee.db

fleskefjes · June 30, 2024, 7:17am

That looks alright - what does your logs say? I notice that your logfile is huge (35MB)

klaaskox · June 30, 2024, 7:43am

Maybe these errors have something to do with it?

2024-06-30 09:40:41.701 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved (None)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/homekit/__init__.py", line 363, in _async_start_homekit
    await homekit.async_start()
  File "/usr/src/homeassistant/homeassistant/components/homekit/__init__.py", line 858, in async_start
    loaded_from_disk = await self.hass.async_add_executor_job(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/homekit/__init__.py", line 580, in setup
    self.driver.load()
  File "/usr/local/lib/python3.12/site-packages/pyhap/accessory_driver.py", line 669, in load
    self.encoder.load_into(file_handle, self.state)
  File "/usr/local/lib/python3.12/site-packages/pyhap/encoder.py", line 97, in load_into
    loaded = json.load(fp)
             ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/__init__.py", line 293, in load
    return loads(fp.read(),
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2024-06-30 09:40:41.708 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved (None)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/homekit/__init__.py", line 363, in _async_start_homekit
    await homekit.async_start()
  File "/usr/src/homeassistant/homeassistant/components/homekit/__init__.py", line 858, in async_start
    loaded_from_disk = await self.hass.async_add_executor_job(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/homekit/__init__.py", line 580, in setup
    self.driver.load()
  File "/usr/local/lib/python3.12/site-packages/pyhap/accessory_driver.py", line 669, in load
    self.encoder.load_into(file_handle, self.state)
  File "/usr/local/lib/python3.12/site-packages/pyhap/encoder.py", line 97, in load_into
    loaded = json.load(fp)
             ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/__init__.py", line 293, in load
    return loads(fp.read(),
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

wrongisthenwright · June 30, 2024, 11:43am

Permission problems? When you started the container with empty config folders what where the files ownership and permissions?

tmjpugh · June 30, 2024, 4:58pm

There are 2 possible causes

1.you did not copy over contents of /haconfig/.storage folder. The folder exists in the screenshot but it is impossible to know if this was created upon start of new container, existing but empty, or correctly copied. need to verify this but only after check 2

2.Incorrect file permissions. unless you set user/group HA runs as almost always the permission are wrong when copied to new system. when migrating from system A to system B the best method is to

setup new system >> copy files to /whatever_you_named_HA_config >> create folder /HAtemp >> run HA docker pointing to empty HAtemp folder >> verify onboarding is shown at HA IP:port >> check folder/file permission of /HAtemp >> match permission in your copied /whatever_you_named_HA_config folder >> stop and restart HA docker pointing to permission corrected /whatever_you_named_HA_config folder

klaaskox · July 1, 2024, 10:35am

Thanks, I will try this again!

klaaskox · July 3, 2024, 6:01pm

When I don’t bind a config directory, all files/folders are root:root

However, I still get the onboarding screen when I use the copied config folder from my RPI4 and root:root.

I looked into the docker container log file, and found that it does find my configuration, but I assume it somehow stops/crashes and falls back to the onboarding process.

Can this have something to do with it?

2024-07-03 19:56:58.996 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry  for hacs
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 594, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/hacs/__init__.py", line 220, in async_setup_entry
    setup_result = await async_initialize_integration(hass=hass, config_entry=config_entry)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/hacs/__init__.py", line 206, in async_initialize_integration
    await async_try_startup()
  File "/config/custom_components/hacs/__init__.py", line 193, in async_try_startup
    startup_result = await async_startup()
                     ^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/hacs/__init__.py", line 171, in async_startup
    hass.config_entries.async_setup_platforms(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ConfigEntries' object has no attribute 'async_setup_platforms'. Did you mean: 'async_unload_platforms'?

tmjpugh · July 3, 2024, 6:40pm

It it a cached screen. Can you actually setup onboarding?

Did you check #1 ? /config/.storage is where user info is stored. This must be present with correct group:user and r/w/x permissions on files.

Are you migrating from:to same version of HA? Looks like one of your custom components is having issue. If you change version prior to getting things functional you will have issues and add an additional problem to check.

klaaskox · July 4, 2024, 5:43am

The problem with the hacs setup caused the problem.

I was migrating from:to the same version of HA. I upgraded hacs on the RPI4, en performed the migration again, now the error disappeared and I could login to HA.

Thanks @fleskefjes and @tmjpugh for your support!