I have got stats /recording problems since 2024.7
Energy board not working anymore.
Historical data of sensors is not working anymore too.
Each day at the same hour it stops recording values :
It works again when i reload configuration or restart, and then again stops recording starting at the same hour in the morning.
Database is currently not locked - then no logging until i reboot. This is in the system logs, reboot fixes the issue.
I am seeing this in the logs at 5am which is when the google backup runs, i have tried manually doing a HA backup and also a google backup which doesnt seem to cause the problem.
I have disabled the backup and see if it occours overnight.
I have already updated to .1 as well.
2024-07-06 05:01:16.292 WARNING (MainThread) [homeassistant.components.recorder.core] Database currently not locked
2024-07-06 05:10:50.330 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 2508266610, dts = 3590302486
2024-07-06 05:13:03.869 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 10197540, dts = -6030
2024-07-06 05:49:52.247 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 196322760, dts = 4292992516
2024-07-06 06:03:25.623 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 6349912, dts = -71720
2024-07-06 06:15:24.970 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 63082080, dts = 4293580756
2024-07-06 06:42:20.249 WARNING (Thread-4409) [custom_components.dreame_vacuum.dreame.protocol] Error while executing request: https://de.api.io.mi.com/app/v2/home/rpc/466929656 HTTPSConnectionPool(host='de.api.io.mi.com', port=443): Read timed out. (read timeout=5)
2024-07-06 07:05:16.954 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Timestamp discontinuity detected: last dts = 267399630, dts = -240030
2024-07-06 08:08:26.252 WARNING (MainThread) [homeassistant.core] Timed out waiting for final writes to complete, the shutdown will continue
2024-07-06 08:08:26.252 WARNING (MainThread) [homeassistant.core] Shutdown stage 'final write': still running: <Task pending name='Task-430763' coro=<Recorder._async_shutdown() running at /usr/src/homeassistant/homeassistant/components/recorder/core.py:480> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.12/asyncio/futures.py:387, <1 more>, Task.task_wakeup()]> cb=[set.remove()]>
2024-07-06 08:08:56.422 WARNING (MainThread) [homeassistant.core] Timed out waiting for close event to be processed, the shutdown will continue
2024-07-06 08:08:56.423 WARNING (MainThread) [homeassistant.core] Shutdown stage 'close': still running: <Task pending name='Task-430763' coro=<Recorder._async_shutdown() running at /usr/src/homeassistant/homeassistant/components/recorder/core.py:480> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.12/asyncio/futures.py:387, <1 more>, Task.task_wakeup()]> cb=[set.remove()]>
2024-07-06 08:08:56.423 WARNING (MainThread) [homeassistant.core] Shutdown stage 'close': still running: <Task pending name='Task-430797' coro=<Recorder._async_close() running at /usr/src/homeassistant/homeassistant/components/recorder/core.py:472> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.12/asyncio/futures.py:387, <1 more>, Task.task_wakeup()]> cb=[set.remove()]>
2024-07-06 08:08:56.604 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_33] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
self._bootstrap_inner()
File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker
work_item = work_queue.get(block=True)
2024-07-06 08:08:56.623 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_54] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
self._bootstrap_inner()
File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
work_item.run()
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.12/threading.py", line 1147, in join
self._wait_for_tstate_lock()
File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
if lock.acquire(block, timeout):
2024-07-06 08:08:56.639 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_40] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
self._bootstrap_inner()
File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
work_item.run()
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.12/threading.py", line 1147, in join
self._wait_for_tstate_lock()
File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
if lock.acquire(block, timeout):
2024-07-06 08:08:56.966 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_54] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
self._bootstrap_inner()
File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
work_item.run()
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.12/threading.py", line 1147, in join
self._wait_for_tstate_lock()
File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
if lock.acquire(block, timeout):
2024-07-06 08:08:57.293 WARNING (Thread-5087 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_40] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
self._bootstrap_inner()
File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
work_item.run()
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.12/threading.py", line 1147, in join
self._wait_for_tstate_lock()
File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
if lock.acquire(block, timeout):
2024-07-06 08:09:06.443 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_54] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
self._bootstrap_inner()
File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
work_item.run()
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.12/threading.py", line 1147, in join
self._wait_for_tstate_lock()
File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
if lock.acquire(block, timeout):
2024-07-06 08:09:06.460 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_40] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
self._bootstrap_inner()
File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
work_item.run()
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.12/threading.py", line 1147, in join
self._wait_for_tstate_lock()
File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
if lock.acquire(block, timeout):
2024-07-06 08:09:06.959 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_54] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
self._bootstrap_inner()
File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
work_item.run()
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.12/threading.py", line 1147, in join
self._wait_for_tstate_lock()
File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
if lock.acquire(block, timeout):
2024-07-06 08:09:07.458 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_40] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
self._bootstrap_inner()
File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
work_item.run()
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.12/threading.py", line 1147, in join
self._wait_for_tstate_lock()
File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
if lock.acquire(block, timeout):
So backup starts at 5:00am and at 5:01am it does the āDatabase currently not lockedā message. Then no other messages until I restart HA at 8:08am where it looks like it is waiting for the DB to finishing performing writes to shut it down cleanly.
So I am guessing Google Backup kicks off a normal HA backup which pauses / quiesce the DB but is not releasing it.
Whilst that works, it doesnāt show a on / off status on the button so you canāt tell which activity is running, also it means another click wonāt turn it off again. I would also need to add a power off button just to be able to stop the activity and again, I canāt see which activity is running as the buttons donāt show a on/off state. With select, also changing from one activity to another does not turn off the previous like the switches so when changing from say xbox to tv, the xbox remains on. So I still need to create input booleans and scripts/automations to make buttons display which activity is on, turn off again and then just a minor thing, to get voice commands to work again which worked before just by exposing the automatically created switch. It was a couple mouse clicks to get working, and now itās gone from easy to a right royal pain to get things working again.
My point I guess is, why should I need to do this as it all seems like a lot of work to get things running again, especially when it all worked all fine and dandy before, and the API hasnāt changed?
I suppose itās just because someone decided to remove the perfectly working code that automatically created switches because they personally couldnāt see a need to have multiple ways to select a activity. Again, why when it works, and the API isnāt changing, was it even changed and removed?? As I said in my original post, if it aināt broke, why āfix itā??
It just shouldnāt be this hard and even more so when it worked already! Useability is out the window with this change and the exact opposite of the HA ethosā¦
In my configuration, this is not working since 2024.7.0 neither in 7.1
card_mod:
style: |
:host {
position: sticky;
}
In chromes developper tools I can see that the position: sticky element is OK. But the card itself isnāt.
Reboot is just a workaround that close all connections to the DB. But problem remains. At 4am, there is a cleanup action to purge the DB.
I guess something is locking the DB.
having the same and since 2024.7.1 it seems the recorder stopped recording history data tooā¦
Logger: homeassistant.core
Bron: core.py:1559
Eerst voorgekomen: 10:13:10 (1880 gebeurtenissen)
Laatst gelogd: 10:14:52
Error running job: <Job listen * HassJobType.Callback <function Recorder.async_initialize.<locals>._event_listener at 0x7fa0cc53f380>>
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal
self._hass.async_run_hass_job(job, event)
File "/usr/src/homeassistant/homeassistant/core.py", line 938, in async_run_hass_job
hassjob.target(*args)
File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 330, in _event_listener
if entity_filter(entity_id):
^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/places/recorder_history_prefilter/recorder_prefilter.py", line 123, in entity_filter
return recorder_entity_filter(entity_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
The time of when the line stops is when I updated to the 2024.7.1 version while current status is still there in the entity.
At first I thought it was an issue with power saving on my zigbee devices, but at further inspection it seems to be an issue with the recorder
@Bieniu
Yes I already see this but I donāt have a lot of hacs integration and none of these.
I suppose that having localy the repository and not using it donāt give any issues.
So I counted only:
So I deactivate them all. and put info log on homeassistant.components.recorder
My question is how can I identify or find part of the code in those repo which prevent recorder working properly ?
[edited:]
itās due to my old HA with a legacy foreign key in db.
Thanks to bdraco for founding this and for the migration path forward.
I will wait the migration in 2024.8
Which is due to be fixed 2024.8.0 Remove legacy foreign key constraint from sqlite states table by bdraco Ā· Pull Request #120779 Ā· home-assistant/core (github.com)
Iām getting some error after upgrading in the logs from logger home assistant.util.loop, source "util/loop.py:77, right after the reboot:
Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pytz/zoneinfo/Africa/Abidjan', 'rb') in /usr/local/lib/python3.12/site-packages/pytz/__init__.py, line 108: return open(filename, 'rb') inside the event loop; This is causing stability issues. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#open Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module> sys.exit(main()) File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main exit_code = runner.run(runtime_conf) File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 960, in _read_ready self._read_ready_cb() File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 1017, in _read_ready__data_received self._protocol.data_received(data) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 133, in data_received self.handle_telegram(telegram) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 159, in handle_telegram parsed_telegram = self.telegram_parser.parse(telegram) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 95, in parse dsmr_object = object["value_parser"].parse(match) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 242, in parse values=self._parse(line) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 219, in _parse return self._parse_values(values) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 190, in _parse_values return [self.value_formats[i].parse(value) File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 399, in parse value = self.coerce_type(value) if value is not None else value File "/usr/local/lib/python3.12/site-packages/dsmr_parser/value_types.py", line 24, in timestamp local_tz = pytz.timezone('Europe/Amsterdam') File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 181, in timezone if zone in all_timezones_set: # noqa File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 154, in _lazy for i in fill_iter.pop(): File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 101, in _lazy list.extend(self, fill_iter.pop()) File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 1115, in <genexpr> tz for tz in _all_timezones_unchecked if resource_exists(tz)) File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 121, in resource_exists open_resource(name).close() File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 108, in open_resource return open(filename, 'rb')
Did you create a bug report like it said to?
Iām searching at the moment if this bug/error is already reported or not. If not, Iāll make a report of itā¦
Same hereā¦ waiting for a fix !
Stay tuned for the dashboard live stream, where we will discuss how we can coordinate our contributor efforts! Weād love to have your keen eye on fixing the details. 
My particular issue with History Graphs not updating, seems to have been caused by my use of the iCloud3 integration which I think was failing to start or causing some problems due to changes in v2024.7.0 to do with recorder_prefilter.py
Iām not sure of the technical details, but I used the temporary fix for iCloud3 mentioned here:
When is this going to happen? If thereās no date fixed yet, some estimation, like in the next couple of weeks or later?
after i restored a backup to go back to 2014.6.4, i tried today to update to 2024.7.1 but again i experience the same errors as with 2024.7.0 (see 2024.7: Time to resize your cards! - #68 by swoop124) so iāam restoring back to 2024.6.4.
hope a future Update will work for me.
It appears to be complaining about a problem with one of your custom integrations.
config/custom_components/icloud3
Another user also reported a problem involving āicloud3ā. You should report it to the custom integrationās author.
Been mentioned a load of times in the threadā¦icloud3 is causing issues with the database / recorderā¦youāve mentioned that Integration in your log.
Update this and you should be fine
Have a read here too PSA: 2024.7 recorder problems
edit: Taras types faster than I do 
