[homeassistant.setup] Setup of recorder is taking over 10 seconds

Hi,
after a hard reset of my raspberry pi2, home assistant doesn’t load and it remains stuck on this warning:

WARNING (MainThread) [homeassistant.setup] Setup of recorder is taking over 10 seconds.

I tried to delete the .db file but it doesn’t start.
The only way to have HA working is putting and error typo in the recorder configuration, like:
recorder:
purge_days: 5

HA doesn’t recognize purge_days and I don’t know for which reason, HA starts but notifies me that logbook, history and recorder could not be set up. This is my configuration for those components:

history:
logbook:
recorder:

Do you know how to help me in fixing those components?
Thanks

I have the same error being reported. I have restarted several times and I can’t fix it. Some investigation has led me to believe that this source of this error is a corrupt file, but I don’t know which 1 or how to fix it.

The error log contains the following:

Error during connection setup: (sqlite3.OperationalError) disk I/O error
(Background on this error at: Error Messages — SQLAlchemy 2.0 Documentation) (retrying in 3 seconds)

Does anyone have any recommendations on how to fix this?

At the end I was able to fix everything only using a backup of the entire operating system. Then, everything worked.

In my case was the db file corrupted…

2 Likes

I just guessed that the datbase file was corrupted. So, I deleted it and restarted hassio. I lost my history, but I didn’t have to re-build and reconfigure my entire home assistant. :+1:

rm /config/home-assistant_v2.db

1 Like

Yes also on my case the dB file was corrupted but even deleting it, the file wasn’t created again. So only using the backup I restored everything.

1 Like

old news but deleting the .db file worked for me.

Are there any downsides to deleting that DB?

Sounds like a nuclear option.

your history gets wiped

I saw this problem when I upgraded to 0.114.

I suspected that it might be related to invalid entity ID’s that I had excluded under Recorder (which can happen if you rename/delete entities and forget to update Recorder). When I deleted the invalid entities then HA restarted normally without the error.

I am thinking that previously invalid entities were ignored but now they are causing the startup to hang. If this is the case then perhaps the configuration validation should be picking the errors up.

Any hint how to identify such entities? My HA starts, but set up of recorder tooks 282 secons and I don’t want lost history :frowning:

Don’t think it is that. I still have nonexisting entity under exclude: in recorder. But this reminds me I have to clean this up.

Reading through the release notes there might be a fix for this in 0.114.2

OK, but it is hard to explain this - I updated to 0.114.1, removed the comments from around 100 excluded entities under recorder and a restart took almost 9 minutes as per the log below:

2020-08-17 19:28:27 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for sonoff which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:28:27 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for anniversaries which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:28:27 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for meross_cloud which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:28:27 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for shelly which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:28:27 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:28:27 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for momentary which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:28:38 WARNING (MainThread) [homeassistant.setup] Setup of recorder is taking over 10 seconds.
2020-08-17 19:29:28 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder
2020-08-17 19:30:28 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder
2020-08-17 19:31:28 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder
2020-08-17 19:32:28 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder
2020-08-17 19:33:28 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder
2020-08-17 19:34:28 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder
2020-08-17 19:35:08 WARNING (MainThread) [homeassistant.components.webhook] Received message for unregistered webhook cc78bc3ec834b1c671f61f2247282af59e3d6ad1d3044dfbaa8ba717e6eb649c from 127.0.0.1
2020-08-17 19:36:18 WARNING (MainThread) [homeassistant.setup] Setup of default_config is taking over 10 seconds.
2020-08-17 19:36:27 WARNING (MainThread) [custom_components.meross_cloud] 
===============================
Meross Cloud Custom component
Developed by Alberto Geniola
Low level library version: 0.4.0.0
-------------------------------
This custom component is under development and not yet ready for production use.
In case of errors/misbehave, please report it here: 
https://github.com/albertogeniola/meross-homeassistant/issues

If you like this extension and you want to support it, please consider donating.
-------------------------------
List of devices reported by HTTP API:
- meross01 (mss210) - OnlineStatus.ONLINE
- meross02 (mss210) - OnlineStatus.ONLINE
===============================
2020-08-17 19:36:27 WARNING (Thread-39) [meross_iot.manager] Subscribed to topics, updating state for already known devices...
2020-08-17 19:36:52 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: default_config
2020-08-17 19:37:16 WARNING (MainThread) [custom_components.meross_cloud.sensor] Found 0 devices that require polling. Setting polling interval to 0:00:30

Previously under version 0.114.0 I would have received a persistent notification to say that Logbook, Recorder, and Default Config could not be loaded (this did not happen this time under version 0.114.1).

I then re-commented out all the 100 entities under Recorder and a restart took less than 2 mins as per this log:

2020-08-17 19:41:16 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for anniversaries which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:41:16 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for meross_cloud which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:41:16 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:41:16 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for momentary which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:41:16 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for shelly which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:41:16 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for sonoff which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2020-08-17 19:41:27 WARNING (MainThread) [homeassistant.setup] Setup of recorder is taking over 10 seconds.
2020-08-17 19:42:16 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder
2020-08-17 19:42:58 WARNING (MainThread) [custom_components.meross_cloud] 
===============================
Meross Cloud Custom component
Developed by Alberto Geniola
Low level library version: 0.4.0.0
-------------------------------
This custom component is under development and not yet ready for production use.
In case of errors/misbehave, please report it here: 
https://github.com/albertogeniola/meross-homeassistant/issues

If you like this extension and you want to support it, please consider donating.
-------------------------------
List of devices reported by HTTP API:
- meross01 (mss210) - OnlineStatus.ONLINE
- meross02 (mss210) - OnlineStatus.ONLINE
===============================
2020-08-17 19:42:58 WARNING (Thread-19) [meross_iot.manager] Subscribed to topics, updating state for already known devices...
2020-08-17 19:43:00 WARNING (MainThread) [custom_components.meross_cloud.sensor] Found 0 devices that require polling. Setting polling interval to 0:00:30

For now I am happy to leave everything being recorded as the performance limitations do not seem to be there since the recent changes to restructure how the database functions.

Aha! I should check if startup performance improves with and without excludes.

I just started up with 0.114.2, and it does contain a fix, but there’s an important caveat: the reason I’m seeing this issue appears to be that my DB is really big, not that it’s corrupt (I’m using the default auto-purge settings). The DB is about 1.8G right now, and it took the best part of an hour to run a full integrity check on it.

The difference in 0.114.2 appears to be that the recorder no longer gives up after a few minutes of trying to start up. I was stuck on “Waiting on integrations to complete setup: recorder” every minute for about half an hour, but it did eventually start up correctly and I didn’t see the notification for logbook/history in the UI. Because it takes so long, the fix might give the impression that the problem has gotten even worse.

I need to get into the DB and see if there’s some entity I added recently that’s been spamming the crap out of the recorder.

This might make sense ie. once you have properly completed an integrity check then maybe you will no longer see the problem. My DB is over 4.5GB and my last restart earlier today only took about 1.5 minutes.

Are you using the standard db, or running a db externally outside of HA?
I experience extreme long startup time, with these lines in the log. It takes about 20 minutes to do a restart, and I cannot figure this out…

Setup of recorder is taking over 10 seconds.
Waiting on integrations to complete setup: recorder```

Hi, I am just running the standard db but have some more info I can provide …

My db had grown to 4.5GB as I had commented out all the Recorder excludes thinking it would help, however I think this actually caused an additional delay as HA performs new database consistency checks since version 0.114. The interesting thing is that the long delays only occur sporadically i.e. once or maybe twice a day on HA restart. What I have seen on many occasions is that the first restart is very slow but then subsequent restarts are much quicker. I do not know what exactly triggers a database consistency check but it may be time based.

As a result I have taken the comments back out of the Recorder excludes, peformed a purge, and my db is down to around 300MB.

I have just done a first restart of the day as a test and it took 1min 40 sec, I then did a second restart and it took 21 sec. This is slower than it used to be on my setup but I can live with it.

I am still on version 0.114.3.

1 Like

I ended up deleting the home-assistant_v2.db file, and now restart takes less than a minute. Probably corrupt database somehow.

My startup delays seem to have disappeared since upgrading to 0.115.1 :smiley: