2024.7: Time to resize your cards!

pops106 · July 6, 2024, 7:20am

Database is currently not locked - then no logging until i reboot. This is in the system logs, reboot fixes the issue.

I am seeing this in the logs at 5am which is when the google backup runs, i have tried manually doing a HA backup and also a google backup which doesnt seem to cause the problem.

I have disabled the backup and see if it occours overnight.

I have already updated to .1 as well.

2024-07-06 05:01:16.292 WARNING (MainThread) [homeassistant.components.recorder.core] Database currently not locked
2024-07-06 05:10:50.330 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 2508266610, dts = 3590302486
2024-07-06 05:13:03.869 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 10197540, dts = -6030
2024-07-06 05:49:52.247 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 196322760, dts = 4292992516
2024-07-06 06:03:25.623 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 6349912, dts = -71720
2024-07-06 06:15:24.970 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 63082080, dts = 4293580756
2024-07-06 06:42:20.249 WARNING (Thread-4409) [custom_components.dreame_vacuum.dreame.protocol] Error while executing request: https://de.api.io.mi.com/app/v2/home/rpc/466929656 HTTPSConnectionPool(host='de.api.io.mi.com', port=443): Read timed out. (read timeout=5)
2024-07-06 07:05:16.954 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 267399630, dts = -240030
2024-07-06 08:08:26.252 WARNING (MainThread) [homeassistant.core] Timed out waiting for final writes to complete, the shutdown will continue
2024-07-06 08:08:26.252 WARNING (MainThread) [homeassistant.core] Shutdown stage 'final write': still running: <Task pending name='Task-430763' coro=<Recorder._async_shutdown() running at /usr/src/homeassistant/homeassistant/components/recorder/core.py:480> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.12/asyncio/futures.py:387, <1 more>, Task.task_wakeup()]> cb=[set.remove()]>
2024-07-06 08:08:56.422 WARNING (MainThread) [homeassistant.core] Timed out waiting for close event to be processed, the shutdown will continue
2024-07-06 08:08:56.423 WARNING (MainThread) [homeassistant.core] Shutdown stage 'close': still running: <Task pending name='Task-430763' coro=<Recorder._async_shutdown() running at /usr/src/homeassistant/homeassistant/components/recorder/core.py:480> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.12/asyncio/futures.py:387, <1 more>, Task.task_wakeup()]> cb=[set.remove()]>
2024-07-06 08:08:56.423 WARNING (MainThread) [homeassistant.core] Shutdown stage 'close': still running: <Task pending name='Task-430797' coro=<Recorder._async_close() running at /usr/src/homeassistant/homeassistant/components/recorder/core.py:472> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.12/asyncio/futures.py:387, <1 more>, Task.task_wakeup()]> cb=[set.remove()]>
2024-07-06 08:08:56.604 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_33] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker
    work_item = work_queue.get(block=True)
2024-07-06 08:08:56.623 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_54] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
2024-07-06 08:08:56.639 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_40] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
2024-07-06 08:08:56.966 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_54] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
2024-07-06 08:08:57.293 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_40] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
2024-07-06 08:09:06.443 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_54] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
2024-07-06 08:09:06.460 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_40] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
2024-07-06 08:09:06.959 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_54] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
2024-07-06 08:09:07.458 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_40] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):

So backup starts at 5:00am and at 5:01am it does the “Database currently not locked” message. Then no other messages until I restart HA at 8:08am where it looks like it is waiting for the DB to finishing performing writes to shut it down cleanly.

So I am guessing Google Backup kicks off a normal HA backup which pauses / quiesce the DB but is not releasing it.

Aussie_Adam · July 6, 2024, 8:03am

Whilst that works, it doesn’t show a on / off status on the button so you can’t tell which activity is running, also it means another click won’t turn it off again. I would also need to add a power off button just to be able to stop the activity and again, I can’t see which activity is running as the buttons don’t show a on/off state. With select, also changing from one activity to another does not turn off the previous like the switches so when changing from say xbox to tv, the xbox remains on. So I still need to create input booleans and scripts/automations to make buttons display which activity is on, turn off again and then just a minor thing, to get voice commands to work again which worked before just by exposing the automatically created switch. It was a couple mouse clicks to get working, and now it’s gone from easy to a right royal pain to get things working again.

My point I guess is, why should I need to do this as it all seems like a lot of work to get things running again, especially when it all worked all fine and dandy before, and the API hasn’t changed?

I suppose it’s just because someone decided to remove the perfectly working code that automatically created switches because they personally couldn’t see a need to have multiple ways to select a activity. Again, why when it works, and the API isn’t changing, was it even changed and removed?? As I said in my original post, if it ain’t broke, why “fix it”??

It just shouldn’t be this hard and even more so when it worked already! Useability is out the window with this change and the exact opposite of the HA ethos…

mrgrlscz · July 6, 2024, 8:06am

In my configuration, this is not working since 2024.7.0 neither in 7.1

card_mod:
  style: |
    :host {
      position: sticky;
    }

In chromes developper tools I can see that the position: sticky element is OK. But the card itself isn’t.

zigomatic · July 6, 2024, 8:09am

Reboot is just a workaround that close all connections to the DB. But problem remains. At 4am, there is a cleanup action to purge the DB.
I guess something is locking the DB.

FGOD1983 · July 6, 2024, 8:20am

having the same and since 2024.7.1 it seems the recorder stopped recording history data too…

Logger: homeassistant.core
Bron: core.py:1559
Eerst voorgekomen: 10:13:10 (1880 gebeurtenissen)
Laatst gelogd: 10:14:52

Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.<locals>._event_listener at 0x7fa0cc53f380>>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
    self._hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
    if entity_filter(entity_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/places/recorder_history_prefilter/recorder_prefilter.py", line 123, in entity_filter
    return recorder_entity_filter(entity_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

The time of when the line stops is when I updated to the 2024.7.1 version while current status is still there in the entity.

At first I thought it was an issue with power saving on my zigbee devices, but at further inspection it seems to be an issue with the recorder

Bieniu · July 6, 2024, 8:32am

zigomatic · July 6, 2024, 8:40am

@Bieniu
Yes I already see this but I don’t have a lot of hacs integration and none of these.
I suppose that having localy the repository and not using it don’t give any issues.
So I counted only:

hacs
remote HA connection
music assistant
Watchman
Meross Lan

So I deactivate them all. and put info log on homeassistant.components.recorder

My question is how can I identify or find part of the code in those repo which prevent recorder working properly ?

[edited:]
it’s due to my old HA with a legacy foreign key in db.
Thanks to bdraco for founding this and for the migration path forward.
I will wait the migration in 2024.8

keithr59 · July 6, 2024, 9:02am

Sounds like Purge causes recorder to stop writing to the DB until HA is restarted · Issue #117263 · home-assistant/core (github.com)

Which is due to be fixed 2024.8.0 Remove legacy foreign key constraint from sqlite states table by bdraco · Pull Request #120779 · home-assistant/core (github.com)

stefan1982 · July 6, 2024, 9:05am

I’m getting some error after upgrading in the logs from logger home assistant.util.loop, source "util/loop.py:77, right after the reboot:

Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pytz/zoneinfo/Africa/Abidjan', 'rb') in /usr/local/lib/python3.12/site-packages/pytz/__init__.py, line 108: return open(filename, 'rb') inside the event loop; This is causing stability issues. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#open Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module> sys.exit(main()) File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main exit_code = runner.run(runtime_conf) File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 960, in _read_ready self._read_ready_cb() File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 1017, in _read_ready__data_received self._protocol.data_received(data) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 133, in data_received self.handle_telegram(telegram) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 159, in handle_telegram parsed_telegram = self.telegram_parser.parse(telegram) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 95, in parse dsmr_object = object["value_parser"].parse(match) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 242, in parse values=self._parse(line) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 219, in _parse return self._parse_values(values) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 190, in _parse_values return [self.value_formats[i].parse(value) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 399, in parse value = self.coerce_type(value) if value is not None else value File "/usr/local/lib/python3.12/site-packages/dsmr_parser/value_types.py", line 24, in timestamp local_tz = pytz.timezone('Europe/Amsterdam') File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 181, in timezone if zone in all_timezones_set: # noqa File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 154, in _lazy for i in fill_iter.pop(): File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 101, in _lazy list.extend(self, fill_iter.pop()) File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 1115, in <genexpr> tz for tz in _all_timezones_unchecked if resource_exists(tz)) File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 121, in resource_exists open_resource(name).close() File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 108, in open_resource return open(filename, 'rb')

tom_l · July 6, 2024, 9:15am

Did you create a bug report like it said to?

stefan1982 · July 6, 2024, 9:35am

I’m searching at the moment if this bug/error is already reported or not. If not, I’ll make a report of it…

jeremypsl · July 6, 2024, 10:09am

Same here… waiting for a fix !

madelena · July 6, 2024, 11:12am

Stay tuned for the dashboard live stream, where we will discuss how we can coordinate our contributor efforts! We’d love to have your keen eye on fixing the details.

smithbill17 · July 6, 2024, 11:14am

My particular issue with History Graphs not updating, seems to have been caused by my use of the iCloud3 integration which I think was failing to start or causing some problems due to changes in v2024.7.0 to do with recorder_prefilter.py

I’m not sure of the technical details, but I used the temporary fix for iCloud3 mentioned here:

github.com/gcobb321/icloud3_v3

HA 2024.7.0 Recorder Error

opened 05:52AM - 04 Jul 24 UTC

scotty1395

Running iCloud3 v3.0.5.2. After upgrading to HA 2024.7.0 I get the below error.… Recorder seems to lock up. Disabling iCloud3 and restarting HA fixes the issue. Enabling iCloud3 triggers the error instantly. ``` Logger: homeassistant.core Source: core.py:1559 First occurred: 3:12:49 pm (336 occurrences) Last logged: 3:13:24 pm Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.<locals>._event_listener at 0x7fdd0b5b3060>> Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal self._hass.async_run_hass_job(job, event) File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job hassjob.target(*args) File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener if entity_filter(entity_id): ^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/icloud3/support/recorder_prefilter.py", line 123, in entity_filter return recorder_entity_filter(entity_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not callable ```

Fanful · July 6, 2024, 11:19am

When is this going to happen? If there’s no date fixed yet, some estimation, like in the next couple of weeks or later?

swoop124 · July 6, 2024, 12:55pm

after i restored a backup to go back to 2014.6.4, i tried today to update to 2024.7.1 but again i experience the same errors as with 2024.7.0 (see 2024.7: Time to resize your cards! - #68 by swoop124) so i’am restoring back to 2024.6.4.
hope a future Update will work for me.

123 · July 6, 2024, 1:42pm

It appears to be complaining about a problem with one of your custom integrations.

config/custom_components/icloud3

Another user also reported a problem involving “icloud3”. You should report it to the custom integration’s author.

Gav_in · July 6, 2024, 1:42pm

Been mentioned a load of times in the thread…icloud3 is causing issues with the database / recorder…you’ve mentioned that Integration in your log.
Update this and you should be fine

Have a read here too PSA: 2024.7 recorder problems

edit: Taras types faster than I do

AlesAlitis · July 6, 2024, 3:49pm

Same issue here. Pretty much all my sensors flat-line from 4 AM to about 2 PM. I just updated to 2024.07.1 but I’ll have to wait until tomorrow to see if it’s fixed

petro · July 6, 2024, 4:08pm

Please take time to read the full release thread. History issues have been mentioned multiple times and there are custom integrations that are causing the problems.