I had similar issues with MySQL on another server. After 17hrs (23Gb database) i pulled the plug. After that HA couldn’t start the recorder anymore.
I deleted the database, still HA report recorder couldn’t be started.
Then I restored a snapshot of 2021.3.4, and.
new tables where created by HA, new data was generated, mysql was working again.
Upgraded to 2021.4.2 again, same thing happened, got notified in dashboard that recorder couldn’t be started. No other logs show any warnings 
Dear community devs and sr. members - Thanks for yet another stellar release, however I wanted to ask if anyone running the ZwaveJS2MQTT integration using direct ws:// websocket integration, are seeing problems with it post-upgrade?
After upgrading from core-2021.3.4 to core-2021.4.2, it seems the ZwaveJS integration breaks: All entities from the zwave devices are unavailable. The integration tile has a Configure option which under normal circumstances allows to add/remove nodes. That part is gone as well as the Reload option on the trippledot menu.
I did read the breaking changes in the release notes however they do not seem to apply as this is happening to all zwave devices. I also let it sit for half an hour to allow for any residual database operations to complete
Will be happy to help troubleshoot this as I have a pre-upgrade VM snapshot I can roll revert to and retry the upgrade any tine. Second I can open a bug report, just wanted to ask around first if anyone else are seeing similar?
Darned, 1 hour into the DB upgrade…
Got it on mysql, and I have 1 week in recorder, and it’s going through the different steps like it should, but the statement ‘might take some minuttes’ is some understatement 
EDIT: finally done, took 1 hour and 10 mins 
The updater tells me that 2021.4.3 is ready, but when checking Docker HUB there is no 2021.4.3 ?!?
probably still be built, there is always a delay
@mobile.andrew.jones Found the issues, it is in mysql command trying to modify a table.
When testing…version 2021.3.4 was able the recreate the sql tables immediately without any issues and it started filling them again. Only when i updated to 2021.4.2 it failed again. This time it didn’t have to update a 23Gb database, but only a few Kb’s (actually 467 events and 743 states) on a new different database as the one i used initially.
For some reason HA 2021.4.1/2 cannot update my database while 2021.3.4 can🤔
@fribse only one week should not take long
Since I’m convinced now there’s something wrong with Mysql integration i raised a ticket on github
Error during connection setup to mysql://hassuser:[email protected]/hass_db?charset=utf8: (MySQLdb._exceptions.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6), MODIFY created DATETIME(6)' at line 1") [SQL: ALTER TABLE events MODIFY time_fired DATETIME(6), MODIFY created DATETIME(6)] (Background on this error at: http://sqlalche.me/e/13/f405) (retrying in 3 seconds)
Hi,
my issue was solved when I switched ESPhome from dev to “normal”. I believe I had to change it for the tag reader some time ago.
It’s now working on the standard release and all is fine.
(I use
System Health
| version | core-2021.4.2 |
|---|---|
| installation_type | Home Assistant OS |
| dev | false |
| hassio | true |
| docker | true |
| virtualenv | false |
| python_version | 3.8.7 |
| os_name | Linux |
| os_version | 5.4.109 |
| arch | x86_64 |
| timezone | Europe/Vienna |
Home Assistant Community Store
| GitHub API | ok |
|---|---|
| Github API Calls Remaining | 4946 |
| Installed Version | 1.11.3 |
| Stage | running |
| Available Repositories | 842 |
| Installed Repositories | 4 |
Home Assistant Cloud
| logged_in | true |
|---|---|
| subscription_expiration | 20. April 2021, 2:00 |
| relayer_connected | true |
| remote_enabled | true |
| remote_connected | true |
| alexa_enabled | false |
| google_enabled | false |
| can_reach_cert_server | ok |
| can_reach_cloud_auth | ok |
| can_reach_cloud | ok |
Home Assistant Supervisor
| host_os | Home Assistant OS 5.13 |
|---|---|
| update_channel | stable |
| supervisor_version | supervisor-2021.03.9 |
| docker_version | 19.03.15 |
| disk_total | 234.0 GB |
| disk_used | 17.0 GB |
| healthy | true |
| supported | true |
| board | intel-nuc |
| supervisor_api | ok |
| version_api | ok |
| installed_addons | Visual Studio Code (3.3.0), Terminal & SSH (9.1.0), Samba share (9.3.1), InfluxDB (4.0.4), Grafana (6.2.0), AppDaemon 4 (0.5.1), MariaDB (2.3.0), Home Assistant Google Drive Backup (0.103.1), phpMyAdmin (0.2.0), Check Home Assistant configuration (3.6.0), Mosquitto broker (5.1.1), ESPHome (1.16.2) |
Lovelace
| dashboards | 2 |
|---|---|
| resources | 0 |
| views | 6 |
| mode | storage |
Kind regards
Albert
1 Like
The addon container may run python 3.6.9.
EDIT I checked, it does. From within the addon container:
root@a0d7b954-esphome:~# python3 --version
Python 3.6.9
root@a0d7b954-esphome:~#
If you click the link of the issue, you find my config of both the grouped trackers I use to mitigate the single WiFi trackers issue, and the individual ping config I use, which is pretty default.
Another way to avoid all this is the use of Phil’s Composite tracker.
And topping that, instead of using individual trackers in the core Person integration, I use the Composite tracker in the person config…
Btw, the Ping issue is fixed and merged in the current 2021.4.3
1 Like
I solved it by adding the additional dev repo… now have 2 instances of esphome
I have not progressed beyond 2021.4.0 and have no issue at all.
Does the zwave js control panel UI load properly?
btw, did anyone see the 2021.4.3 release yet https://www.home-assistant.io/blog/2021/04/07/release-20214/#release-202143---april-10 ?
my system alerted 5 hours ago…? (need the Ping fix…)
I’d guess the build failed and everyone who can kick it is sleeping
It seems to me that in case of DB migration being required process should look like:
- First HA should try to estimate how long it could take to migrate DB (not sure how, but perhaps performing few sample record changes on temporary created ‘test’ tables?) and evaluating number of records in DB. This should give at least some order of magnitude view on how long the process might take for particular setup.
- Then we should be given options:
a) Migrate to new DB, discarding all of historical data, but process would be instant.
b) Migrate with DB being converted in background, X hours of data will be lost during conversion. HA starts instantly and no service disruption. Data starts to recorded once DB update is completed.
c) Perform full migration, no data lost but HA not available for X hours.
Strangely difference between option b) and c) is that while we anyhow loose X hours of data (either data not being recorded during conversion or HA not working at all), but we have or not HA providing us with some core services… so my preference anyhow would be to start HA and agree to some data loss 
Another question to DB gurus… would below process work?
- at start of conversion HA renames old tables to not use them for data recording.
- creates new tables, with new structure in the same DB instance, with old names
- HA starts recording new info instantly to new tables
- background process reads data from old tables, converts as required and writes to new tables
- old tables are deleted after completion
This way we have no data loss and HA services available during conversion. I’m not sure about consistency of data for some queries, if only recent data is available, but historical might be incomplete.
And another side remark. during my ~2 years journey with HA it is only second time that I recall when DB structure was updated. First time it was ~30 minutes for me (but then my setup was way simpler). So with this frequency of DB changes I’m not really sure if this situation shouldn’t be just accpted. If it would be repeated every HA update, this would be another story.
It would obviously help to change wording from ‘could take few minutes’ to ‘could take several hours’, just to set proper expectations At least this could easy stress related to not knowing what is going on… if this is still conversion or something failed…
1 Like
Seriously? I can remember more than that in the last few months… it gets migrated a fair bit.
heck yes, it was twice already during the last beta 
1 Like
Then it was even more transparrent! Seriously I recall only case from the past, when DB schemas were changed and migration was causing some delay in startup. If there were any more it was fully transparrent to me. Migration to 2021.4 was the first case that really scared me…
Well, I’m not participating in beta! This might explain why I did not noticed any more changes!
No there have been more, not just in beta either.
I would say, at least in my case - after looong time (almost 24 hrs) - it seems that conversion was performed (in db apparently startded to appear new writes), SAMBA works - can connect to config, addons, backup etc., MQTT (Mosquitto) works - ongoing traffic visible in mqtt explorer, but there is no connection to HA. Anybody else met the same?