Best practice for gathering sensor data

Configuration
1

I am getting the hang of connecting devices to home assistant, but am wondering what are some good practices for sensor data management?

Example: Monitoring a MQTT sensor every second, but taking the average over 10 minutes to actually report or display in HA. I figured out pretty quick that HA doesn’t like to make a graph for something reported every second. It slows down HA, makes a big database, ect.

An example of why I want/need this is this: A wind and direction sensor from wunderground can rapid fire updates every second or two, if HA only grabs one snapshot in time every 10 minutes, the reading could be vastly different than the average over the whole 10 min. A reading every second reported to HA for 10 min is a ton of readings, while readings taken every second, averaged, and only recorded once every 10 min is both accurate and efficient.

2

The database HA uses is a relational database.

In general, using a relational database for storing sensor data is not very efficient. If you are serious storing data at this sort of rate, and for a perioid of time, you should look to use a time series database, like influxdb or prometheus.

The other problem with storing this data in the HA database is that the HA database is occasionally corrupted, and can only realistically be recovered by deleting it and starting again - which would lose all your collected sensor data.

3

Then I’ll stick to that question.

I also have several sensors that send values via mqtt every 10 seconds. (temperature, humidity, brightness…)

Up to now I have edited the MQTT values via Node-Red. (make the average of the last 20 values and send them out via MQTT)

That’s exactly what I would like to do with HA.

4

The Statistics Sensor will do what you want.

5

I have the Statistics Sensor enabled for testing a sensor, but haven’t made anything reliable. The MQTT message still blasts HA every second, and the sampling rate isn’t consistent with how I set it.

Do you have an example of how I would accomplish my goal with the statistics sensor?

Node-Red sounds & looks interesting, going to look into it a bit…