Avoid missing values in sensors during restart

[123 (Taras)]

Sure, because it all it needs is just one sensor to be available for it to compute a value. Remember when I cautioned you about using this approach?

@123

if you exclude sensors from the calculation, it affects the computed average value. Worst case, if on startup none of the sensors are available, all will be excluded and the result of the calculation becomes zero (which will definitely affect the graph). Are you sure this how you want to handle the situation?

The only way to ensure it won’t compute anomalously low values is to restrict the calculation to when all sensors are available (not when some are and some aren’t).

average_house_temperature:
  friendly_name: 'Inside'
  unit_of_measurement: '°C'
  availability_template: >-
    {% set sensors = expand('group.indoor_temperature_sensors') | list %}
    {{ sensors | rejectattr('state', 'in', ['unknown', 'unavailable'])
       | list | count == sensors | count }}
  value_template: >-
    {% set sensors = expand('group.indoor_temperature_sensors')
        | map(attribute='state') | map('float') | list %}
    {{ (sensors | sum / sensors | count) | round(1) }}

[ZyberSE]

Unfortunately I get a similar result with that code.

The graph shows 3.9c at the bottom, I have around 25c on average right now and that’s over 7 sensors, seams to me like the value template returns true as all sensors are available but then when getting their values only one returns something, but still the count is 7 which would explain the low value.

[123 (Taras)]

You really need to determine the values reported by each sensor at startup. Without that information, this is all guesswork. Either examine each sensor’s history or the Logbook or, best of all, the values recorded in the database. It’s entirely possible they’re not unknown or unavailable but who knows what.

[ZyberSE]

Further investigation shows that my Zigbee and Z-Wave sensors with bogus values have state ‘restored’, but my SensorPush sensors do not. They instead are missing the attribute "attribution": "Data by SensorPush".

So I would need to exclude sensors that have state restored and those that miss the attribute "attribution": "Data by SensorPush". If any sensor comes through that filer I would like to calculate average temperature based on those. Which would mean a way to filter them out again the the value template and also count that filter.

This is beyond my templating skills so help is greatly appreciated.

[123 (Taras)]

@ZyberSE

Further investigation shows that my Zigbee and Z-Wave sensors with bogus values have state ‘restored’, but my SensorPush sensors do not. They instead are missing the attribute "attribution": "Data by SensorPush".

It’s easy to reject an entity if its state value is restored. It only requires a small modification in the template posted above:

rejectattr('state', 'in', ['unknown', 'unavailable', 'restored'])

What’s more challenging is the other part, where SensorPush sensors normally have an attribution attribute except when they have a bogus value. We can’t simply reject all sensors that are missing an attribution attribute because it’s normal for non-SensorPush sensors to lack that attribute.

Is there some other attribute that SensorPush sensors always have but your other sensors do not? We’re looking for a way to positively identify a SensorPush sensor and then reject it if it lacks an attribution attribute.

[ZyberSE]

Yeah that’s the tricky part. Here are the raw values from one of the sensors.

A dirty workaround could be looking at restart time, and delay the sensor 30sec. Would that be possible?

[123 (Taras)]

How do you envision incorporating delays based on restart time within a Template Sensor?

[123 (Taras)]

How many sensors are involved? How many of the total are SensorPush?

[ZyberSE]

It’s 7 in total and 5 SensorPush, I could exclude those that are not.

[123 (Taras)]

Add a custom attribute to the 2 non-SensorPush sensors called attribution. Assign it whatever value you want.

Now when everything is running normally, all 7 sensors have an attribution attribute. On startup, the 5 SensorPush sensors will be (potentially) missing their attribution attribute but the other 2 will still have it (because it’s a custom attribute). Now the template can safely reject any of the 7 sensors that lack an attribution attribute (and any whose state is restored).

[ZyberSE]

Any suggestions on how todo that?

[123 (Taras)]

I don’t understand your question; I just gave you a suggestion on how to do it.

[ZyberSE]

Yeah but I’m not that skilled with templating so I can to it practically.

[123 (Taras)]

This seems like a perfect opportunity to improve your templating skills. You’ve been given several examples that can now be enhanced in the suggested manner.

First step, is to create custom attributes for the two non-SensorPush sensors. You can do it via the UI or via customize.yaml. You only have two sensors so I would suggest adding the custom attributes via the UI.

Customizing entities

[ZyberSE]

Ahh. I thought those attributes came straight from the sensor gateway but they are merged with any customization done before they become available?

I did poke around a bit in the SensorPush integration code and from what I understand "attribution": "Data by SensorPush" is just a config passed to HA and nothing set by the integration itself?
If that’s the case those other values must come from somewhere else?

[123 (Taras)]

I don’t use the SensorPush integration but if the sensor entities it creates usually have an attribution attribute then it’s a safe assumption that the SensorPush integration is responsible for it.

Your observation that this attribute is missing from SensorPush sensors on startup (and/or whenever it generates bogus values) is some sort of strange quirk of the SensorPush integration. I have no idea if that’s by design or accident.

[ZyberSE]

After some excellent help from @123 I’m happy to report my template_sensor works perfectly now!

Here is what I ended up with. I decided to require at least two sensors as just one could deviate from the expected average value too much.

average_house_temperature:
  friendly_name: 'Inside'
  unit_of_measurement: '°C'
  device_class: temperature
  availability_template: >-
    {% set sensors = expand('group.house_temperature_sensors') | list %}
    {{ sensors
       | rejectattr('state', 'in', ['unknown', 'unavailable', 'restored'])
       | selectattr("attributes.attribution")
       | list | count > 1
    }}
  value_template: >-
    {% set sensors = expand('group.house_temperature_sensors')
       | rejectattr('state', 'in', ['unknown', 'unavailable', 'restored'])
       | selectattr("attributes.attribution")
       | map(attribute='state') | map('float') | list %}
    {{ (sensors | sum / sensors | count) | round(1) }}

[123 (Taras)]

@ZyberSE

decided to require at least two sensors

Where is that requirement in availability_template? What I see is that it accepts a minimum count of 1 sensor to declare availability.

count > 0

Or have I misunderstood what you wrote?

[ZyberSE]

@123

Or have I misunderstood what you wrote?

Yeah, you need to look again

[123 (Taras)]

Sneaky, but the post’s history revealed the alteration.