Recorder - Home Assistant (home-assistant.io)
Like to include or exclude sensor based on area or label that is just added to HA.
Excluding by label (to avoid listing them all out in configuration. yaml) would make it so much easier to discard data you know you will never want to see again other than in real time. I know my DB would be a lot smaller if I had this ability.
On top of excluding, it would also be nice if one could set a shorter retention period for entities based on a label.
1 Like
There are so many popular threads about database activity and size, that the inclusion, exclusion process should really be made more user-friendly and accessible.
However, whilst I wait for this long-overdue feature to be implemented…
I have recently implemented the basics of getting this done directly in HA. I am currently just needing to write some automations, but the framework is in place.
Here is an example…
I have a macro that returns an entity_id list of all entities matching any of the labels I provide
This macro lives in /config/custom_templates/labels.jinja
{#-
# Returns: a comma separated list of *ANY* entities matching any of the labels provided in label_list
# Usage: {{label_entities_any(['Temperature-Climate','Humidity-Climate']).split(",")}}
-#}
{%- macro label_entities_any(label_list) -%}
{{- label_list | map('label_entities') | sum(start=[]) | sort | join(",") -}}
{%- endmacro -%}
I have another macro that uses this first macro.
It returns me a list of every entity_id I want to not record.
You could just use a single label ‘No-Recorder’ and tag everything you want excluded.
{%- macro recorder_excluded_entities() -%}
{%- set excluded = [
'Weather-Wind-Speed',
'Weather-Wind-Gust',
'Weather-Wind-Gust-Max',
'Weather-Air-Quality',
'Weather-Temperature-Feels-Like',
'Weather-Temperature',
'Weather-Humidity',
'Weather-Dew-Point',
'Weather-Rain-Hourly',
'Weather-Rain-Daily',
'Weather-Rain-Weekly',
'Weather-Rain-Monthly',
'Weather-Rain-Yearly',
'Weather-Relative-Pressure',
'Weather-UV-Index',
] -%}
{%- set entities = label_entities_any(excluded)
.split(",")
| list
| sort
| unique -%}
{{- entities| join(",") -}}
{%- endmacro -%}
I have a YAML package (recorder_exclude.yaml) that takes that list of entities and writes it out to a YAML file (which is read in by my configuration).
So for changes to take effect…
- you must first run the script (I have an automation that does this every 6 hours and on shutdown)
- you need to restart HA
# ############################################################
# Notify file service - enables writing to a file (it appends)
# ############################################################
notify:
- platform: file
name: file_log_weather_recorder_test
filename: /config/packages/climate/weather_recorder_test.yaml
timestamp: false
# ############################################################
# Shell-Commands to manage the recorder configuration file
# ############################################################
shell_command:
# Remove the old file
recorder_exclude_remove: rm /config/db1138/climate/weather_recorder_test.yaml
# Create the replacement file
recorder_exclude_create: touch /config/db1138/climate/weather_recorder_test.yaml
# Remove the first two lines from the file 'Because STUPID' @see below
recorder_exclude_clean: sed -i '1,2d' /config/packages/climate/weather_recorder_test.yaml
# ############################################################
# Script that creates the YAML file
# ############################################################
script:
write_recorder_exclude:
alias: write_recorder_exclude
mode: single
variables:
recorder_entities: >
{%- from "labels.jinja" import recorder_excluded_entities -%}
{%- set recorder_entities = recorder_excluded_entities().split(",") -%}
{{- recorder_entities -}}
recorder_output: >
{%- set e = "" -%}
{%- set sep = "\n - " -%}
{%- if recorder_entities | count -%}
{%- set e = sep + recorder_entities | join(sep) -%}
{%- endif -%}
{%- set rv = [
'',
'# Generated by "script.write_recorder_exclude"',
'recorder:',
' exclude:',
' entities:' + e,
] -%}
{{- rv | join("\n") -}}
sequence:
- if:
- condition: template
value_template: "{{- (recorder_entities | count) > 0 -}}"
then:
- service: shell_command.recorder_exclude_remove
data: { }
alias: 'Clears the contents of the recorder file'
- delay: "00:00:01"
- service: shell_command.recorder_exclude_create
data: { }
alias: 'Creates a new empty recorder file'
- delay: "00:00:01"
- service: notify.file_log_weather_recorder_test
data:
message: |
{{- recorder_output -}}
alias: 'Write the contents to the recorder file'
- delay: "00:00:01"
# Home Assistant notifications (Log started: 2024-05-26T07:53:15.380423+00:00)
# --------------------------------------------------------------------------------
- service: shell_command.recorder_exclude_clean
data: { }
alias: 'Removes the STUPID first two lines that get added to all notify.file files'
1 Like
You can modify the macros and script above to collect all entities with a ‘Record-24-Hours’ label and then run the recorder.purge_entities service with the results.
Manual:
service: recorder.purge_entities
data:
keep_days: 1
entity_id:
- sensor.weather_daily_rain
- sensorweather_dew_point
- sensor.weather_feels_like
Automated: (make sure to add a condition to check you have some entities in recorder_excluded_entities() before calling the service)
service: recorder.purge_entities
data:
keep_days: 1
entity_id: >
{%- from "labels.jinja" import recorder_excluded_entities -%}
{%- set recorder_entities = recorder_excluded_entities().split(",") -%}
{%- set purge_entities = "" -%}
{%- set sep = "\n - " -%}
{%- if recorder_entities | count -%}
{%- set purge_entities = sep + recorder_entities | join(sep) -%}
{%- endif -%}
{{- e -}}
1 Like
Hello @teskanoo
I had added the label purge to all the entities that I don’t want to keep so I thought why not use the following approach:
{% set list = label_entities("purge")%}
{% set todo = "- " + list | join("\n- ")%}
{{-todo-}}
the above returns:
- sensor.adguard_home_average_processing_speed
- sensor.adguard_home_dns_queries
so running the action:
action: recorder.purge_entities
data:
keep_days: 5
entity_id: >
{% set list = label_entities("purge")%}
{% set todo = "- " + list | join("\n- ")%}
{{-todo-}}
fails with the following message:
Failed to perform the action recorder.purge_entities. Entity ID - sensor.adguard_home_average_processing_speed - sensor.adguard_home_dns_queries is an invalid entity ID for dictionary value @ data['entity_id']. Got None
What am I doing wrong?
Looks like you’re trying to turn your list into how it appears when it’s pure YAML text
What you want is a simple comma separated list
action: recorder.purge_entities
data:
keep_days: 5
entity_id: >
{{- label_entities("purge") | join(",") -}}
1 Like
@teskanoo thank you so much. That was it. Your code worked!
I assumed that my script should have worked since the manual format is:
action: recorder.purge_entities
data:
keep_days: 1
entity_id:
- sensor.mfc_l8690cdw_status
- sensor.mfc_l8690cdw_page_counter
- sensor.mfc_l8690cdw_b_w_counter
- sensor.mfc_l8690cdw_color_counter
- sensor.mfc_l8690cdw_duplex_unit_pages_counter
- sensor.mfc_l8690cdw_drum_remaining_life
- sensor.mfc_l8690cdw_belt_unit_remaining_life
- sensor.mfc_l8690cdw_fuser_remaining_life
- sensor.mfc_l8690cdw_laser_remaining_life
- sensor.mfc_l8690cdw_pf_kit_1_remaining_life
- sensor.mfc_l8690cdw_pf_kit_mp_remaining_life
1 Like
Instead of listing all entities or renaming the entities in order to be able to use a wildcard, I’d like to use something like this:
recorder:
include: # or exclude
labels:
- Keep History
# and the same for the influxdb integration
influxdb:
[...]
include: # or exclude
labels:
- Keep History
btw, this may be a duplicate: Allow for use of labels in recorder