Some sensors keep long term data, some don't, how to configure this?

Hi,
I try to compare some temperature data (e.g.)


but as you can see, not all data is kept for more than 10 days, which seems to be the default.

The long kept data in this scenario is from the “Airthings” integration. Which I simply klicked, and I am sure I did not configure anything. The missing data is from my own wheather station integration, where I do

sensor:
  - name: davis_vantage_pro_curr
    platform: rest
    resource: "http://192.168.90.146/v1/current_conditions"
    json_attributes:
      - data
    value_template: "Vantage Pro"
  - platform: template
    sensors:
      temp:
        friendly_name: "Temperatur"
        device_class: temperature
        unit_of_measurement: "°F"
        value_template: '{{ ((state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["temp"] | float * 100)) | round / 100  }}'
        unique_id: vantage_temp

My question: I found many recommandations here, that say to install extra databases and some other software to get long term sensor data, but to me it feels, that this is already "there "somewhere, but I have to enable it.

How can I enable long term sensor data for e.g. my weather station (Also other sensors are effected, but I hoped thas this example would work.)

thanks
Juergen

LTS are kept for any sensor with a valid state_class.

The legacy template sensor platform does not support state_class and will not be getting any updates to enable this. You should be using the the template integration which does support this.

Hi,
thank you for the reply.

I found this
Template Weather Provider - Home Assistant and I am totally stuck and overwelmed. No idea what to change.

I try this

but this seems to use the same wrong strategy.

Than I found this

but the “add integration” HomeAssistant configuration does not work.

Than I found

and it says “during the installation you are asked”, … but I wasn’t.

I asked there for help.

thanks
Juergen

Delete this:

  - platform: template
    sensors:
      temp:
        friendly_name: "Temperatur"
        device_class: temperature
        unit_of_measurement: "°F"
        value_template: '{{ ((state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["temp"] | float * 100)) | round / 100  }}'
        unique_id: vantage_temp

Create this:

# configuration.yaml
template:
  - sensor:
      - name: "Temperatur"
        unique_id: vantage_temp
        device_class: temperature
        state_class: measurement # <- generates LTS (long term statistics)
        unit_of_measurement: "°F"
        state: '{{ (state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["temp"] | float * 100) | round / 100  }}'
        availability: '{{ state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["temp"] | is_number  }}'

You will lose all your history for the sensor Temperatur and the new entity id will be sensor.temperatur but it will record LTS forever from now on.

Hi Tom,
thanks a bunch!! So I will loose my 10 days :slight_smile: of recording.

Changed it with the temperature and it worked lilke a charme. I will now try to change the rest of the vantage sensors.

My question: I am a relative newbie (got the HA recommendation from a friend about 8 month ago). I dig thru a lot of HA informations, and as I wrote above there seems to be “at least” some configuration that is way outdated.

Is there a way to suggest to mark such information as “outdated”, and link to a better (more relevant) information?

thanks
Juergen

I wish. But sadly no.

I will occasionally get a notification someone liked an old post of mine that is out of date and will then post an update warning them.

Your best course of action is to look at the date the post was made, if it is older than say 6 months or so (things change fast) then check the examples in the official documentation to make sure it is still current.

Hi Tom,
problem with this “6 month” is, that sometime, the only thing that works, is older. I really dig around, and as a “non programmer”, I at least want to give ack something avoiding that others have troubles like me, as it is really annoying.

To keep it short: Maybe there is a way to foster this. I am willing to help, but have not enough knowledge to say, that “this implementation” is not recommended.
I’ll post my learnings here, and try to find a way to make the worls a little better.

Thank you very very much for your help. I also changed some other sensor data, I wanted to keep for a longer time period.

all the best
Juergen
P.S.: I’ll wrap this up with t little “solution”, and than would close this issue.

Here is the solution to keep “HomeMade” sensor data for a longer time. Motivation: I have my own dragino Lorawan Sensor configurations and my weatherstation (vantage pro) and wanted to keep this data longer than the default ten days. You find a lot of outdated informations for older HA Versions, that recommend installing new databses, different systems, whatever. This might be a way, but I think it is outdated. With the help of Tom, I did the following for my Vantage Weatherstation and my dragino Lorawan devices:
To get a better overview over such configutations, I suggest to split them up in “Packages”, as shown here Packages - Home Assistant and NOT filling up the configuration.yaml more and more, as still described in many places.

So my configuration.yaml has this

homeassistant:
  packages: !include_dir_named "packages"

in the very first lines, and than I have the directory “packages”, which looks like
screen-2024-08-15-15-41-34

The configuration for “some” (the ones I need) sensors of the Vantage pro is in the vantage_pro.yaml (Beware, if you are like me and don’t read all information immediately, there HAS to be the underscore in the filename, and “-” (minus) is forbidden!

In my scenario my Vantage is reachable thru the ethernet wifi gateway, this is reachable at IP http://192.168.90.146. Change this below accordingly. You might call the url below, but depending on the webbrowser you won’t see anything, or get the json data to see.

The content of this file is:

# Description of Vantage pro weather staion API 
# https://weatherlink.github.io/weatherlink-live-local-api/

sensor:
  - name: davis_vantage_pro_curr
    platform: rest
    resource: "http://192.168.90.146/v1/current_conditions"
    json_attributes:
      - data
    value_template: "Vantage Pro"

template:
  - sensor:
      - name: "Vantage Temperatur Heubergweg"
        unique_id: vantage_temp
        device_class: temperature
        state_class: measurement # <- generates LTS (long term statistics)
        unit_of_measurement: "°F"
        state: '{{ (state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["temp"] | float * 100) | round / 100  }}'
        availability: '{{ state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["temp"] | is_number  }}'

  - sensor:
      - name: "Wind-Gesch"
        unique_id: vantage_wind_speed
        device_class: wind_speed
        state_class: measurement # <- generates LTS (long term statistics)
        unit_of_measurement: "km/h"
        state: '{{ ((state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["wind_speed_avg_last_2_min"] | float * 160.934)) | round / 100}}'
        availability: '{{ state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["temp"] | is_number  }}'

  - sensor:
      - name: "Regen (1 Min)"
        unique_id: vantage_rain_1
        device_class: water
        state_class: measurement # <- generates LTS (long term statistics)
        unit_of_measurement: "L"
        state: '{{ ((state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["rain_rate_hi"] | float *  20)) | round / 100}}'
        
  - sensor:
      - name: "Regen (60 Min)"
        unique_id: vantage_rain_60
        device_class: water
        state_class: measurement # <- generates LTS (long term statistics)
        unit_of_measurement: "L"
        state: '{{ ((state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["rainfall_last_60_min"] | float *  20)) | round / 100}}'
        
  - sensor:
      - name: "Regen (24h)"
        unique_id: vantage_rain_24
        device_class: water
        state_class: measurement # <- generates LTS (long term statistics)
        unit_of_measurement: "L"
        state: '{{ ((state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["rainfall_last_24_hr"] | float *  20)) | round / 100}}'

  - sensor:
      - name: "Feuchtigkeit"
        unique_id: vantage_hum
        device_class: humidity
        state_class: measurement # <- generates LTS (long term statistics)
        unit_of_measurement: "%"
        state: '{{ ((state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["hum"] | float * 100)) | round / 100  }}'

  - sensor:
      - name: "Windchill"
        unique_id: vantage_wind_chill
        device_class: temperature
        state_class: measurement # <- generates LTS (long term statistics)
        unit_of_measurement: "°F"
        state: '{{ ((state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["wind_chill"] | float * 100)) |round / 100 }}'

which you might have to adapt, if you want non metric values. And which you definitely do have to translate, if you are a non germans speaker.

I hope this helps others.

Again: Many thanks Tom

all the best
Juergen

Hi Tom,
I changed evrything as shown above in the file. IMHO I did the LTS correct. This morning I checked me system, but it still has only the last 10 days. See screenshot (Same sensor (blue) as above). (now from 6th on and not from 5th on as yesterday.

You wrote, that the data will be compleately erased and kept new, which is not the case. Do I have to give the sensors a new ID, to make sure it reacts to the LTS?

thanks
Juergen

Oh. I forgot about the unique id. That means you wont lose your existing state data.

You won’t see LTS for another 10 days. It only started being generated when you created the sensor with the state class.

Hi Tom,
so to get the data for more than 10 days, I have to use “new unique_id”, which means the old sensors are “gone”, but the new ones will keep the data LTS?
Correct?

Any way to fillde in a database to avoid this (I admint, I am not really in the mood :slight_smile: ) just curious.

many thanks
Juergen

No not correct. All you have to to is wait ten days.

The LTS data created toady and yesterday will then not disappear after your 10 day purge interval.

Do nothing. Wait 10 days.

Hi Tom,
so the system purges the sensor data, that was collected before I changed to “LTS”, and the data from yesterday after the “LTS” change is kept, but I cannot “see” the difference as I kept the same unique_id?
Do I understand this correct?

thanks
Juergen
P.S.: Waiting is not my strongest virtue :slight_smile:

The system always purges state data. Your LTS data has only been collected for a couple of days. That wont show up on your graph until it is 10 days old.

You can see the LTS data now if you want by using a statistics graph instead of a history graph.

Hi @tom_l ,
today is the first day, where I can see half a day more of history data. Many thanks again.

I tried it with this sensor (which already show very old LTS data),
and this simply gives errors.

sensor:
    - platform: integration
      source: sensor.battery_power_charging
      name: "Total Battery Energy Charged"
      unique_id: 'battery_power_charging_uuid'
      unit_prefix: k
      method: left
    - platform: integration
      source: sensor.battery_power_discharging
      name: "Total Battery Energy Discharged"
      unique_id: 'battery_power_discharging_uuid'
      unit_prefix: k
      method: left
    - platform: integration
      source: sensor.power_photovoltaics
      name: "Total Photovoltaics Energy"
      unique_id: 'power_photovoltaics_uuid'
      unit_prefix: k
      method: left

I was not expecting this, so here for documentation purpose, that

state_class: measurement

does not work always, but seems to break in certain conditions, so to all others that read it: Be careful and test.

thanks
Juergen

No. As accumulators they would have the state_class total.

Hi @tom_l ,
since the last update (or already the one before?) I see the following error in the log file:

2024-09-18 09:41:00.794 WARNING (MainThread) [homeassistant.components.sensor] Entity sensor.rain_rate_hi (<class 'homeassistant.components.template.sensor.SensorTemplate'>) is using state class 'measurement' which is impossible considering device class ('water') it is using; expected None or one of 'total_increasing', 'total'; Please update your configuration if your entity is manually configured, otherwise create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue

YAML config:

template:
  - sensor:
      - name: "Regen (1 Min)"
        unique_id: vantage_rain_1
        device_class: water
        state_class: measurement # <- generates LTS (long term statistics)
        unit_of_measurement: "L"
        state: '{{ ((state_attr("sensor.davis_vantage_pro_curr", "data")["conditions"][0]["rain_rate_hi"] | float *  20)) | round / 100}}'

Can you give me a pointer what I do need to add?

many thanks
Juergen

state_class: total