2024.2 onwards: recorder and history sometimes not recording some sensors

[jchh (my username is my initials (and my first name isn't John))]

I have been having an issue since 2024.2 whereby some entities are no longer being reliably recorded.

Example 1:
input_select.house_mode is Awake, yet it’s history shows it as being Asleep

Only the to: 'Asleep' is being recorded, but not the To: 'Awake'. The last 2 House_mode ➜ Awake have not been recorded:

It starts to show correctly after a restart (albeit the history of when it actually went to Awake is lost.
This is not a permanent fix as as I restarted yesterday as well.

Example 2:
These 4 timers are running right now…

I have tried removing the entities, restarting, adding them back and restarting again, but it makes no difference.

Before I downgrade back to 2024.1 I thought i’d check here to see if anyone else is observing this.

I am running HA Container (docker on an RPi).

I have added this info to this GitHub issue but no-one has taken ownership of it (yet).

[Edwin_D]

I checked but see no similar problems in my instance. What database do you use? could it be there’s something broken there? Maybe the recorder database shows corruption or the connection to the database is unreliable? Is there anything in the logs?

I would be surprised if it is only input_select, I guess the effect is easier to catch there. Do you see the same in binary sensors?

[jchh (my username is my initials (and my first name isn't John))]

Hi,

Thanks for replying!

I am using the HA database. How can I check if there is something broken? If it were broken, why would it update correctly after a restart, always record one of the states but then not record another?

I have only noticed it so far in some (but not all) input_select and timers - but that is because I am monitoring some to check/analyse/prove a new automation.

I am running 2 instances and have just tried running a timer on the 2nd one. It is also not recording when the state is ‘active’. I noticed that it does show correctly in ‘history-graph’ but disappears when the screen is refreshed. Both the following screenshots were taken a second apart. The first before the resters, the 2nd after.

Before:

After:

edit: I will now check some binary_sensors

edit2: Forgot to add that I do have errors in my logs, but not sure they are relevant to this - what should I be looking for there?

[jchh (my username is my initials (and my first name isn't John))]

OK, binary_sensors are also no longer recording.

This is my recorder.yaml - everything was recording just fine in 2024.1

recorder.yaml

commit_interval: 60 # default is 5
purge_keep_days: 7  # default is 10

include:
  domains:
    - automation
    - camera
    - climate
    - cover
    - fan
    - input_select
    - light
    - person
    - script
    - timer
  entity_globs:
    # Aqara
    - binary_sensor.*_contact
    # Hive
    - binary_sensor.hive*
    - sensor.hive*
    # Hue
    - binary_sensor.*_occupancy
    - sensor.*_temp
    - switch.*_smartplug
    # tapo
    - binary_sensor.tapo_*
    - select.tapo_*
    - sensor.tapo_*

[Edwin_D]

I’m no expert on how to check for curruption. But if it is the default db, you do not need to look at the external dbms. The commit interval is a minute. If during that minute the system reboots you may lose values from the last minute, but I think I can safely assume you’re not rebooting every 55 secs.

Are you using an SD card for the recorder or a SSD? Maybe you’re slowly watching your SD card dying?

[jchh (my username is my initials (and my first name isn't John))]

I’m using an SSD.

Note that I have not changed anything other than upgrading and I am seeing this same behaviour on 2 different instances; the only commonality is that they both use HA Container (docker).

[Edwin_D]

That is weird. It does seem to imply it is not just specific to your instance. I’m running on an RPI on HAOS so that may be a relevant difference.

[jchh (my username is my initials (and my first name isn't John))]

not fixed in 2024.2.3

[Ildar_Gabdullin]

Same happens on my 2024.2.5 setup with all “timer” entities.
Even w/o “recorder.yaml” - i.e. with default settings.
posted here

It shows a history first:

but after refresh it is lost:

Simply does not write states to DB, same “idle”:

Update:
Deleted all timers, created new ones (same names) - they do have a persistent history, absolutely weird.
Update:
After a couple of “timer start” - again no History.

[NzKyle (Kyle)]

There are a couple of issues on GitHub related to this - #111375 and #110495 - but there hasn’t been any movement on them.

My Python isn’t the best - but when I have some time I’ll dig around the code a bit and see if anything untoward jumps out at me.

[jchh (my username is my initials (and my first name isn't John))]

Fixed in 2024.3.1