Next Generation Recorder

[00lex (lex)]

Hi! 6 months have passed since I switched to Home Assistant and by now I feel at home with the system.

However, two things are still bothering me. Here I would like to talk about the “Recorder”. I’ve read a lot about “Recorder”, “Database” and how users try to keep their database lean. Unfortunately, there was nothing for me. Even if you set the “recorder” very individually, there are still too many compromises or unnecessary data in the end.

I know it’s always easy to make suggestions when you don’t know the technical background, and maybe it’s not even possible to implement with home assistant as it is. On the other hand, the team has already implemented other great things.

To the point:
Why not per domain, entity, entity_globs etc. individual “commit_interval” and “keep/expire times” ?
that would be so awesome!

recorder:
  commit_interval: 60
  db_url: sqlite:////home/homeassistant/.homeassistant/backups/db.tar/ha.db # exclude from backup ;)
  expire: 365
  rules:
    entities:                     # , commit_interval, expire
      - binary_sensor.workday_sensor, 1m             , 30d
      - sensor.benz					, 1d             , 1y
      - sensor.consumption_2        , 5m             , 180d
      - sensor.l_main               , 5s             , 9M
      - device_tracker.watch_becks  , 10m            , 6h
      - switch.cam_terrase_detection, 1m             , 3M
    domains:
      - binary_sensor               , 5m             , 30d
      - sensor                      , 5m             , 30d
    entity_globs:
      - binary_sensor.p_*           , 5m             , 30d
      - sensor.p_*_battery          , 2h             , 30d

Would something like this fail due to lack of interest or the technical underpinnings of Home assistant?

[CaptTom (Tom)]

This is similar to other FR’s:

Recorder Retention Period By Entity

I’d like to be able to set a Recorder keep_days value for each entity. Background: New and non-technical users are steered toward running HA on a RPi with an SD card. Yet, if they take the defaults, every event and state change is recorded in the SQL database on the SD card, leading to a bloated database file and shortened SD card life. Yes, I know there are lots of ways to change these defaults, including using other databases and database locations. But for most beginners, it’s a steep learn…

Make Recorder Logbook and History Includes avaialable for editing on the entities page

Like this: [Untitled]

And others I don’t have handy. Unfortunately, as you suggest, they’ve gotten no interest from developers yet. But there is hope. I’ve noticed the last few releases included some efficiency updates to the Recorder database. I take this as a sign that at least some devs are OK doing the less glamorous work on the important parts of the project, rather than making high-visibility design changes to the UI.

[bdraco (J. Nick Koston)]

You could accomplish what you are asking for with an automation if we added keep_days to the recorder.purge_entities service

You could set expire days to the maximum you want for everything in the yaml config and than create automations to purge specific groups of entities with a keep_days value

The commit interval is per transaction it wouldn’t make much sense to have it per entity id as you could end up writing more data because you couldn’t group it as well. The added complexity likely would not be a net benefit

[00lex (lex)]

Thank you. This is again a relatively complex solution where you work in parallel in the UI and in the config and in the end you still don’t have the database you want.

I will play around with it though. But how to setup multiple Domains/Entity Globs? With a comma, semicolons? Do I need parentheses? This is probably basic knowledge but I can not find the information

Not here: Recorder - Home Assistant
and not here: Service Calls - Home Assistant

service: recorder.purge_entities
data:
  entity_globs: binary_sensor.p_*, binary_sensor.c_*
target:
  entity_id:
    - binary_sensor.workday_sensor
    - input_boolean.schlafparadies
    - input_boolean.schlafsalon
    - input_boolean.schlafschlaf
    - input_boolean.workday
    - input_number.daytimevar
    - sensor.consumption_2
    - sensor.ts_kuche_battery
    - sensor.ts_paradies_battery
    - sensor.ts_salon_battery
    - sensor.ts_schlafzimmer_battery
    - device_tracker.watch_becks
    - switch.cam_terrase_detection

[123 (Taras)]

@00lex

But how to setup multiple Domains/Entity Globs? With a comma, semicolons? Do I need parentheses? This is probably basic knowledge but I can not find the information

It’s explained in the first documentation link you posted.

The value you provide should be a YAML list.

Refer to the example here.

[00lex (lex)]

thank you. I need a manual for the documentation