Statistics Sensor data as Entity Attributes - removing the need for helpers and sensors in most cases

I suggest commonly used values, like Statistics, should live with the Entity itself for ease of use, ease of visibility, ease of manageability.

Why / Problem statement…
Part of HA’s complexity is that commonly needed features and functions are often ‘helpers’ or ‘template sensors’… which creates a lot more work to build and manage for the user.
This also greatly disperses (and often hides - sensor template do not show in the UI) the information/values into multiple different locations / entities. Just trying to remember where something is or what it is named can be difficult. I’m only 4 months in and I have 2400 entities and I avoid template sensors like the plague as they are super ‘hidden’ IMO. If I cant see it / search for it it in the UI I dont want it hiding in my system.

Implementation Proposal…

  1. Entity statistics off by default
  2. User provided an option inside all relevant Entity property pages to enable “Statistics” which would provide all the same values as the sensor example below can provide… just HA would be providing in the Attributes of the Entity.
sensor:
  - platform: statistics
    name: Living Room Temperature Stats
    entity_id: sensor.living_room_temperature
    state_characteristic: mean  # options: mean, median, min_value, max_value, etc.
    sampling_size: 20  # Number of samples
    max_age:
      minutes: 60  # Optional, to only include recent data

The dev team has been working for at least 2 years to remove large and dynamic attributes from entities’ state objects because of the negative impacts they were having on the database and overall performance… it seems very unlikely anything like this would happen.

image

While there may be issues with exactly how representative the data available from Analytics is, I think it’s a bit of a stretch to claim Statistics is commonly used. Even if you fudge it a bit and include Min/Max in there, and assume no one is using both… you’re only looking at 10% usage rate.

I understand your first point though that begs the question: Where is the sensor data is stored – I assumed it was in the same database, no?
… maybe a good solution is to have a separate database only for dynamic entries like this?

I cant tell if HA uses SQLlite to store entity values but I believe you can Attach Database so that a single query runs across them. I believe they run separate and I think performance should be maintained.

Or maybe there is a way to use a ‘static reference / link’ from the main entity database to the attributes in the smaller ‘dynamic’ database, so the code itself pulls the data from the dynamic db, based on the reference which lives in the entity (statically). Similar to the way Linux does Symbolic Links

Most users would only have a handful of entities they would use statistics on. Maybe 10 to 50, I’m guessing.

I’m definitely including min, max, mean, etc. As I’ve learned over the years, the easier you make something to use, usage will increase - sometimes greatly. The inverse is true also. I’d bet this use would increase a lot if it was just like any other attribute and just a checkbox to enable in the Entity’s settings.

I use min, max and mean for some of my entities, but I need some showing for 1 week, others for 2 weeks and some entities need both one for 1 hour, 1 day and 1 week.

Your suggestion does not seem to open up for such dynamic selection of entities.

on the contrary – you can store as many attributes (as pointers / symbolic links) as you’d want. My understanding is that 50-100 attributes is “safe” and its recommended to stay under 255. Is that enough for you :wink:

the Statistics code still does all the calcs, so you can set any of the settings you want, thus making the duration (max_age) whatever you choose.

It is just a mess in the GUI and harder to code.

This suggestion is what I call a laziness suggestion.
The devs have to do the hard work, so the user can relax.

The min, max, mean is a big mess to do, but it gets much worse as you add other functions.

Your comment is sad in many ways; and is the antithesis of what building applications is all about. Luckily in my entire career, I’ve never heard a developer say anything like that.

I am not sure what your career is, but it is the way it is.
It is either ioen up the possibilities with an open ended system or making it user friendly with a GUI abd limit the possibilities.
Just look at Apple vs. Linux or Apple vs. Android.

Your suggestion is not about new functionality, because HA can already do what you suggest.

30 year software career (Microsoft, Red Hat, AWS)

Your comment about open vs not is wrong is every way… and even your analogy is wrong. Android is user friendly; IMO as much as Apple (we have both in our house). I know just as many non-technical people who use Android as do Apple : my 77yr old dad and his 80 yr old wife both used both Apple and Android phones (many years for each)

If engineers (software and hardware) had your attitude most things, like the Internet, would not exist. I was around building computer hardware and programming (BBS software and more) before the internet existed too.

Yeah, the users of iPhone and Android might see a user friendly interface, but the ones that actually have to work with the OS is the developers of apps and they see a quite different approach between the two.

And it is funny that you mention Microsoft as an example, because they started out with only text on MSDOS and OS/2 before moving over to Windows, where the GUI became more and more dominant. The latest versions of Windows have however switched direction and more and more settings are handled by powershell commands, especially on the server versions.

And you say that the internet would not exist with this approach, but the internet was exactly left open, so all kind of different services could be build upon it.
If it was integrated and locked down with the apps onto of it, then we would still be using Gopher.

1 Like