Hanging During Startup - Need Help

Sock · September 8, 2022, 10:23pm

Home Assistant OS VM on Proxmox using Maria DB

Today I upgraded from 2022.8.7 to 2022.9.1. The system restarted and was cycling through the “Starting ___, not everything will be available until it is finished” messages when the web interface became unresponsive. I rebooted, same thing. There is nothing glaring in the HomeAssistant.log file.

I restored my VM-level backup from last night. Same behavior.

I built an entirely new HAOS VM, created / logged in with a dummy account, downgraded it to 2022.8.7, and restored a full HASS backup from overnight. Same behavior.

Each time the interface hangs, it lists a different component in the “Starting ___” message, so I don’t think the component being shown is the issue.

I can still access the CLI and the SAMBA share. I can’t find anything from the CLI that shows an obvious problem.

I need some help on what to look for. I’ve had this instance running for years, and I can’t imagine starting over.

2022-09-08 18:06:47.438 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration cryptostate which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.438 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration bhyve which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.439 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration ui_lovelace_minimalist which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.440 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration watchman which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.440 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration my_unifi which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.440 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration cupertino which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.441 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration nest_protect which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.442 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration eufy_vacuum which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.442 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration lovelace_gen which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.443 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration unifigateway which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.444 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration tesla_custom which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.444 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration dwains_dashboard which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.445 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration mila which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.445 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration nicehash which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.446 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration fontawesome which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.447 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration localtuya which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.447 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration climacell which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.447 ERROR (SyncWorker_0) [homeassistant.loader] The custom integration 'climacell' does not have a version key in the manifest file and was blocked from loading. See https://developers.home-assistant.io/blog/2021/01/29/custom-integration-changes#versions for more details
2022-09-08 18:06:47.447 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration govee which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.449 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration solaredge_modbus which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.449 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration wyzesense which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.450 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration weatherapi which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.450 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.450 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration powercalc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.452 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration holidays which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.453 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration alexa_media which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.453 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration blueiris which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:47.454 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration monitor_docker which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:06:48.252 WARNING (Recorder) [homeassistant.components.recorder.util] Ended unfinished session (id=452 from 2022-09-08 20:39:52)
2022-09-08 18:06:49.253 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from supervisor (172.30.32.2). Requested URL: '/api/config'. (HomeAssistantSupervisor/2022.08.6 aiohttp/3.8.1 Python/3.10)
2022-09-08 18:06:49.261 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from supervisor (172.30.32.2). Requested URL: '/api/config'. (HomeAssistantSupervisor/2022.08.6 aiohttp/3.8.1 Python/3.10)
2022-09-08 18:06:49.263 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from supervisor (172.30.32.2). Requested URL: '/api/config'. (HomeAssistantSupervisor/2022.08.6 aiohttp/3.8.1 Python/3.10)
2022-09-08 18:06:51.273 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from goldenmonkey.lan (192.168.0.111). Requested URL: '/api/websocket'. (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36 Edg/104.0.1293.63)
2022-09-08 18:07:20.439 ERROR (MainThread) [homeassistant.components.nws] Error requesting NWS forecast station KOSU data: 500, message='Internal Server Error', url=URL('https://api.weather.gov/gridpoints/ILN/82,84/forecast')
2022-09-08 18:07:29.598 WARNING (MainThread) [homeassistant.setup] Setup of automation is taking over 10 seconds.
2022-09-08 18:07:48.603 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: mjpeg, apple_tv, automation
2022-09-08 18:08:45.589 ERROR (MainThread) [homeassistant.components.rest] Unexpected error fetching rest data data: Invalid type for url.  Expected str or httpx.URL, got <class 'NoneType'>: None
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 205, in _async_refresh
    self.data = await self._async_update_data()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 164, in _async_update_data
    return await self.update_method()
  File "/usr/src/homeassistant/homeassistant/components/rest/data.py", line 60, in async_update
    response = await self._async_client.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1514, in request
    request = self.build_request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 344, in build_request
    url = self._merge_url(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 374, in _merge_url
    merge_url = URL(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urls.py", line 102, in __init__
    raise TypeError(
TypeError: Invalid type for url.  Expected str or httpx.URL, got <class 'NoneType'>: None
2022-09-08 18:08:46.780 WARNING (MainThread) [custom_components.govee] API is back online.
2022-09-08 18:08:46.921 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Z-Wave JS' for zwave_js integration not ready yet: None; Retrying in background
2022-09-08 18:08:51.478 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: mjpeg, apple_tv, automation, mqtt, plex, synology_dsm
2022-09-08 18:09:02.908 WARNING (MainThread) [homeassistant.components.integration.sensor] sensor.old_office_lamp_energy could not restore last state unknown: [<class 'decimal.ConversionSyntax'>]

mobile.andrew.jones · September 8, 2022, 10:45pm

Sock:

2022-09-08 18:06:49.253 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from supervisor (172.30.32.2). Requested URL: '/api/config'. (HomeAssistantSupervisor/2022.08.6 aiohttp/3.8.1 Python/3.10)
2022-09-08 18:06:49.261 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from supervisor (172.30.32.2). Requested URL: '/api/config'. (HomeAssistantSupervisor/2022.08.6 aiohttp/3.8.1 Python/3.10)
2022-09-08 18:06:49.263 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from supervisor (172.30.32.2). Requested URL: '/api/config'. (HomeAssistantSupervisor/2022.08.6 aiohttp/3.8.1 Python/3.10)
2022-09-08 18:06:51.273 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from goldenmonkey.lan (192.168.0.111). Requested URL: '/api/websocket'. (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36 Edg/104.0.1293.63)

That part kind of looks like a problem actually, I’d maybe temporarily remove the http_ban from your config, in case the supervisor has been banned from accessing home assistant!

Sock · September 8, 2022, 10:53pm

Thanks for catching that.
I can’t seem to determine where it would be enabled - there are no http or ip_ban settings in my configuration.yaml.

FWIW, those specific errors don’t appear in the log of my latest reboot.

mobile.andrew.jones · September 8, 2022, 10:55pm

You might need to post the entire log then - from startup, until it crashes.

Sock · September 8, 2022, 10:59pm

Here is the entire log from the latest reboot.

2022-09-08 18:55:43.700 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration cryptostate which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.707 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration bhyve which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.709 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration ui_lovelace_minimalist which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.714 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration watchman which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.715 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration my_unifi which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.717 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration cupertino which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.721 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration nest_protect which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.727 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration eufy_vacuum which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.735 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration lovelace_gen which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.737 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration unifigateway which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.739 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration tesla_custom which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.742 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration dwains_dashboard which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.744 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration mila which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.749 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration nicehash which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.752 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration fontawesome which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.766 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration localtuya which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.767 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration climacell which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.767 ERROR (SyncWorker_1) [homeassistant.loader] The custom integration 'climacell' does not have a version key in the manifest file and was blocked from loading. See https://developers.home-assistant.io/blog/2021/01/29/custom-integration-changes#versions for more details
2022-09-08 18:55:43.771 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration govee which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.773 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration solaredge_modbus which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.775 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration wyzesense which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.776 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration weatherapi which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.777 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.779 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration powercalc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.780 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration holidays which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.781 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration alexa_media which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.786 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration blueiris which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:55:43.799 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration monitor_docker which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-09-08 18:56:04.707 WARNING (MainThread) [custom_components.govee] API is back online.
2022-09-08 18:56:04.817 ERROR (MainThread) [homeassistant.components.rest] Unexpected error fetching rest data data: Invalid type for url.  Expected str or httpx.URL, got <class 'NoneType'>: None
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 205, in _async_refresh
    self.data = await self._async_update_data()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 164, in _async_update_data
    return await self.update_method()
  File "/usr/src/homeassistant/homeassistant/components/rest/data.py", line 60, in async_update
    response = await self._async_client.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1514, in request
    request = self.build_request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 344, in build_request
    url = self._merge_url(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 374, in _merge_url
    merge_url = URL(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urls.py", line 102, in __init__
    raise TypeError(
TypeError: Invalid type for url.  Expected str or httpx.URL, got <class 'NoneType'>: None
2022-09-08 18:56:12.131 WARNING (MainThread) [homeassistant.components.integration.sensor] sensor.old_office_lamp_energy could not restore last state unknown: [<class 'decimal.ConversionSyntax'>]
2022-09-08 18:56:14.472 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform mqtt is taking over 10 seconds.

mobile.andrew.jones · September 8, 2022, 11:05pm

Anything in /config/home-assistant.log.fault ?

Sock · September 8, 2022, 11:08pm

Nothing - that file is empty.

Here’s what I’m able to see in the Supervisor log. The “Error on call http:” lines start after it becomes unresponsive.

mobile.andrew.jones · September 8, 2022, 11:13pm

What about dmesg on the VM itself? Just to check Home Assistant didn’t get killed with OOM or something?

Sock · September 8, 2022, 11:27pm

I don’t see anything obvious there, just startup stuff. Nothing in the log after about 75s post-startup.

Sock · September 8, 2022, 11:34pm

I disconnected my USB devices: ZigBee + Z-Wave stick, Bluetooth receiver, WYZE hub and the system now starts completely. This at least gives me further direction to keep hunting.

Sock · September 9, 2022, 12:30am

Ultimately, I had to reboot the Proxmox host. Then I could add the USB devices back to the Home Assistant VM and everything worked.

My Linux expertise is pretty weak, so thanks for the help @mobile.andrew.jones!

mobile.andrew.jones · September 9, 2022, 12:37am

Little bit apprehensive about updating my Proxmox HA instance now, but it’s still on 2022.9.0 right now, and I always wait until the first point release before updating