Avoid missing values in sensors during restart

I would like a smooth graph from sensor even over reboots.

Currently I get these dips:
image

Is there anyway to have the graph interpolate missing values?

For my House temperature I use a template sensor to calculate average of my seven different sensors. Even though I do my best to avoid incorrect values I still get some strange results during restarts as you can see in the graph. It looks like it sums a few sensors but dividing by more.
Is there anyway to optimize this template?

I’m thinking two things:

  1. Can I add a list of sensors to the template_sensor and reference those in the actual template instead of having them hard coded in the template?
  2. Instead of just looking for entity_id when counting them, can I verify that each sensor have a value?

image

This is my template_sensor for average house temp.

    average_house_temperature:
      friendly_name: 'Inside'
      unit_of_measurement: '°C'
      value_template: >-
        {{ ((    states('sensor.bedroom_temperature') | float
               + states('sensor.office_temperature') | float
               + states('sensor.bathroom_temperature') | float
               + states('sensor.shower_room_temperature') | float
               + states('sensor.kitchen_temperature') | float
               + states('sensor.entrance_motion_temperature') | float
               + states('sensor.living_room_motion_temperature') | float
            ) / states |
            selectattr('entity_id','in', [
              'sensor.bedroom_temperature',
              'sensor.office_temperature',
              'sensor.bathroom_temperature',
              'sensor.shower_room_temperature',
              'sensor.kitchen_temperature',
              'sensor.entrance_motion_temperature',
              'sensor.living_room_motion_temperature'
            ]) | list | count
           ) | round(1)
        }}

You need to add an availability template to your template sensor. Search the forum there are many examples.

2 Likes

This won’t fix the ‘missing values’ issue (like tom_l said, use availability_template) however it will simplify the template:

    average_house_temperature:
      friendly_name: 'Inside'
      unit_of_measurement: '°C'
      value_template: >-
        {% set sensors = expand('sensor.bedroom_temperature', 'sensor.office_temperature',
                                'sensor.bathroom_temperature', 'sensor.shower_room_temperature',
                                'sensor.kitchen_temperature', 'sensor.entrance_motion_temperature',
                                'sensor.living_room_motion_temperature')
                         | map(attribute='state') | map('float') | list %}
        {{ (sensors | sum / sensors | count) | round(1) }}

You can create a group containing all of the temperature sensors (group.all_temperature) and then reference the group entity in the template:

    average_house_temperature:
      friendly_name: 'Inside'
      unit_of_measurement: '°C'
      value_template: >-
        {% set sensors = expand('group.all_temperature') | map(attribute='state') | map('float') | list %}
        {{ (sensors | sum / sensors | count) | round(1) }}

Yes, that’s possible. However, if you exclude sensors from the calculation, it affects the computed average value. Worst case, if on startup none of the sensors are available, all will be excluded and the result of the calculation becomes zero (which will definitely affect the graph). Are you sure this how you want to handle the situation?

If that’s the way you want to handle it, then try this:

    average_house_temperature:
      friendly_name: 'Inside'
      unit_of_measurement: '°C'
      value_template: >-
        {% set sensors = expand('group.all_temperature')
            | rejectattr('state', 'in', ['unknown', 'unavailable'])
            | map(attribute='state') | map('float') | list %}
        {{ (sensors | sum / sensors | count) | round(1) }}

Thank you guys for some awesome suggestions!!
The code looks so much cleaner but unfortunatly I’m still getting the dpi/missing values. What have I missed?

GROUP

indoor_temperature_sensors:
  name: Indoor temperature sensors
  entities:
    - sensor.bedroom_temperature
    - sensor.office_temperature
    - sensor.bathroom_temperature
    - sensor.shower_room_temperature
    - sensor.kitchen_temperature
    - sensor.entrance_motion_temperature
    - sensor.living_room_motion_temperature

SENSOR

average_house_temperature:
  friendly_name: 'Inside'
  unit_of_measurement: '°C'
  availability_template: >-
    {% set sensors = expand('group.indoor_temperature_sensors')
        | rejectattr('state', 'in', ['unknown', 'unavailable'])
        | map(attribute='state') | map('float') | list %}
    {{ sensors | count > 0 }}
  value_template: >-
    {% set sensors = expand('group.indoor_temperature_sensors')
        | rejectattr('state', 'in', ['unknown', 'unavailable'])
        | map(attribute='state') | map('float') | list %}
    {{ (sensors | sum / sensors | count) | round(1) }}

I would have to say it’s a good solution because it’s a copy of what I posted … :wink:

Except for the availability_template. It contains the template I suggested for value_template and is not necessarily optimal for this application.

Replace it with this:

  availability_template: >-
    {{ expand('group.indoor_temperature_sensors')
        | rejectattr('state', 'in', ['unknown', 'unavailable'])
        | list | count > 0 }}

I did it that way first but I’m not sure if a sensor could become available before it has a value?

Your first version did take the float value into account, the later only checks for the sensor. But I’m not sure it’s a real problem or not.

Right after a restart.

image

Sure, because it all it needs is just one sensor to be available for it to compute a value. Remember when I cautioned you about using this approach?

The only way to ensure it won’t compute anomalously low values is to restrict the calculation to when all sensors are available (not when some are and some aren’t).

average_house_temperature:
  friendly_name: 'Inside'
  unit_of_measurement: '°C'
  availability_template: >-
    {% set sensors = expand('group.indoor_temperature_sensors') | list %}
    {{ sensors | rejectattr('state', 'in', ['unknown', 'unavailable'])
       | list | count == sensors | count }}
  value_template: >-
    {% set sensors = expand('group.indoor_temperature_sensors')
        | map(attribute='state') | map('float') | list %}
    {{ (sensors | sum / sensors | count) | round(1) }}

Unfortunately I get a similar result with that code.

The graph shows 3.9c at the bottom, I have around 25c on average right now and that’s over 7 sensors, seams to me like the value template returns true as all sensors are available but then when getting their values only one returns something, but still the count is 7 which would explain the low value.

You really need to determine the values reported by each sensor at startup. Without that information, this is all guesswork. Either examine each sensor’s history or the Logbook or, best of all, the values recorded in the database. It’s entirely possible they’re not unknown or unavailable but who knows what.

Further investigation shows that my Zigbee and Z-Wave sensors with bogus values have state ‘restored’, but my SensorPush sensors do not. They instead are missing the attribute "attribution": "Data by SensorPush".

So I would need to exclude sensors that have state restored and those that miss the attribute "attribution": "Data by SensorPush". If any sensor comes through that filer I would like to calculate average temperature based on those. Which would mean a way to filter them out again the the value template and also count that filter.

This is beyond my templating skills so help is greatly appreciated.

It’s easy to reject an entity if its state value is restored. It only requires a small modification in the template posted above:

rejectattr('state', 'in', ['unknown', 'unavailable', 'restored'])

What’s more challenging is the other part, where SensorPush sensors normally have an attribution attribute except when they have a bogus value. We can’t simply reject all sensors that are missing an attribution attribute because it’s normal for non-SensorPush sensors to lack that attribute.

Is there some other attribute that SensorPush sensors always have but your other sensors do not? We’re looking for a way to positively identify a SensorPush sensor and then reject it if it lacks an attribution attribute.

Yeah that’s the tricky part. Here are the raw values from one of the sensors.

A dirty workaround could be looking at restart time, and delay the sensor 30sec. Would that be possible?

How do you envision incorporating delays based on restart time within a Template Sensor?

How many sensors are involved? How many of the total are SensorPush?

It’s 7 in total and 5 SensorPush, I could exclude those that are not.

Add a custom attribute to the 2 non-SensorPush sensors called attribution. Assign it whatever value you want.

Now when everything is running normally, all 7 sensors have an attribution attribute. On startup, the 5 SensorPush sensors will be (potentially) missing their attribution attribute but the other 2 will still have it (because it’s a custom attribute). Now the template can safely reject any of the 7 sensors that lack an attribution attribute (and any whose state is restored).

Any suggestions on how todo that?

I don’t understand your question; I just gave you a suggestion on how to do it.