Timing of sensors issue (mathematical help needed)

I want to create a graph of the part of the power consumption in my house which is NOT monitored by any of my powerplugs. Creating the template sensor isn’t so much of an issue (find code below). The problem is that each powerplug submits its values at different timings so that I get big spike (both positive and negative ones) which prevents me from actually seeing the power consumption which isn’t monitored.

Can any one of you point me in the right direction for solving this issue?

{{ ((states.sensor.power_consumption_watts.state|float) - (states.sensor.cvketel_power.state|float) - (states.sensor.droger_power.state|float) - (states.sensor.eettafel_stopcontact_power.state|float) - (states.sensor.espressomachine_power.state|float) - (states.sensor.koelkast_power.state|float) - (states.sensor.koelvriezer_power.state|float) - (states.sensor.licht_badkamer_spiegel_power.state|float) - (states.sensor.licht_eettafel_group_power.state|float) - (states.sensor.licht_keuken_power.state|float) - (states.sensor.licht_schuur_poort_power_2.state|float) - (states.sensor.licht_tv_group_power.state|float) - (states.sensor.licht_voordeur_power.state|float) - (states.sensor.licht_zijmuur_power.state|float) - (states.sensor.oven_power.state|float) - (states.sensor.quooker_power.state|float) - (states.sensor.smarthome_power.state|float) - (states.sensor.tv_power.state|float) - (states.sensor.vaatwasser_power.state|float) - (states.sensor.wasmachine_power.state|float)) |round(2) }}

Resulting graph below:

The calc can only work with the figures supplied, in a similar situation I resolved this by having the graph ‘average’ this out over a longer timeframe, i.e. with 30 min. data I average it on 60 mins. This may not be perfect but then again, so is your data right now.
Another option is to try and change the time to match the others?

Data comes in in various different timings; some sensors send eveery 8 seconds, other every 10. I think synchronizing all powermeters isn’t possible (at least I have not seen anything like that in esphome). I am now trying to see if piping the values of the template sensor through a statistics (average_linear) produces anything meaningfull… Main (potentially only) issue are the spikes created by my boiling water tap; they show up both on the DSMR (full house consumption) and the individiual power meter…

Rather than averaging the result of the template, create a template sensor for your “known loads” and a filter sensor to average this. Also average your total load. A two minute average for each should work (~ 10 samples). Then use another template sensor to subtract these two averages.

Also for all these templates you need to supply default values for your float filters and you should use an availability template to protect against stupid values if one or more sensor becomes unknown or unavailable.

See: Template sensors this morning started calculating zero energy intermittently, causing energy usage to go haywire - #5 by tom_l

Thanks @Tom_in_HI! I have now created the following:

  - platform: statistics
    name: "Measured power mean"
    entity_id: sensor.measured_power
    state_characteristic: mean
    max_age:
      minutes: 2
    sampling_size: 100
  - platform: statistics
    name: "Power consumption watts mean"
    entity_id: sensor.power_consumption_watts
    state_characteristic: mean
    max_age:
      minutes: 2
    sampling_size: 50
  - platform: template
      averageddelta_power:
        friendly_name: "Averaged delta mean consumed VS mean measured"
        unit_of_measurement: 'W'
        value_template: "{{ ((states.sensor.power_consumption_watts_mean.state|float) - (states.sensor.measured_power_mean.state|float)) |round(2) }}"

The strange thing is that if I look at the averageddelta_power values they change every few seconds, whereas I would expect them to change between 1-2 minutes. Is that normal?

Not sure. I don’t trust the stats sensor as it does not take into account how long a state occurs for. A sensor could be 10 all day then one reading of 5 changes the mean to 7.5.

I would use the low pass filter instead:

I think the characteristic average_step should solve the problem that you described. Or do I understand it wrong?

1 Like

I changed from mean → average_step (and increasing the sampling_size to 2000) and I still see the resulting sensor changing values every few seconds. The powermeters supply values are coming in about every 8 seconds. With about 10 sensors feeding into the measured sensor I believe that a max_age of 2 minutes with a sampling size of 2000 should be more than enough to cover the timeframe of 2 minutes…

EDIT: also the resulting graph isn’t what I would expect (negative values should, in the case of proper averaging, not happen)

The negative values arise from the sequence of:

  1. big load turns off
  2. master power sensor updates, reporting lower value than the big load’s individual sensor
  3. template updates to negative value
  4. load’s power sensor updates, reporting new lower value
  5. template updates to positive value

Your averaging should cover several updates of the master power sensor.

I know… That’s also why am using a long period for averaging (2 minutes) and a big sample (2000) to ensure the max_age becomes leading. Within 2 minutes the master power sensors updates 4 times and all the individual sensors about 15 times. The key issue is now that either I don’t understand the statistics sensor OR the statistics sensor is not performing as expected…

Finally managed to make it work. Ha-average recently got updated and now works with the latest release. I used that with a timeframe of 5 minutes and now I get graphs which are meaningful. Now let’s track down where all that electricity is going to…