The new Untracked Consumption. Trying to add a custom energy sensor

Hi everyone. I noticed the new untracked consumption part added this last week under the Energy view. It’s a great addition. One thing i am trying to add though is a custom sensor, and i’m not having much luck, i’ve tried making a custom config for the sensors. I’ve tried Wh, W and kWh. I’ve tried basing it on Per minute, or per hour. Using the Utility meters and the config and also the Integral sensors. I just can’t seem to get it to work how i want it to though.

Basically with all the trackers off i have (Smart plugs and such). i can see this grey untracked.

What i am trying to do and what would be a great feature to add also, would be Assumed loads.

I have worked out all the items in the house and some, have the idle state. So this is the lowest base load.

So I have these in my Yaml. oh which I can get them to show on one of my charts fine.

  - platform: template
    sensors:
      battery_system_base_load_current_power:
        friendly_name: "Battery System base load current power"
        unit_of_measurement: "W"
        value_template: "{{ 17 }}"
        # Could be changed to 0.28 per minute.
        icon_template: mdi:lightning-bolt
        
      smart_plugs_base_load_current_power:
        friendly_name: "Smart Plugs Base Load current power"
        unit_of_measurement: "W"
        value_template: "{{ 29.44 }}"
        # Could be changed to 0.5 per minute.
        icon_template: mdi:lightning-bolt
        
      house_items_base_load_current_power:
        friendly_name: "House items base load current power"
        unit_of_measurement: "W"
        value_template: "{{ 31 }}"
        # Could be changed to 0.52 per minute.
        icon_template: mdi:lightning-bolt

But what i would really like is to get these stock/base rates to be totalling up per hour like the other energy devices, and then colour coding, and deducting from the Untracked Consumption. I know these are there. They are background lowest power min consumption devices. The untracked i’d like reserved for when things fire up, ie Vacuum, and the devices included in the above when they come out of idle etc and using power etc.

So it’s basically just a way to reduce the grey by adding in “known” fixed base loads".

Probably an improvement to submit somewhere. But a re-ordering ability and a colour selector for the energy devices would be a lovely addition.

If anyone knows who the chuff these need to be adding it would be amazing :slight_smile: Thankyou so much!

Assuming that I understand you correctly, you did create three artificial power sensors that mimic a constant base power usage, and you want to add these as base energy consumption to the Energy Dashboard.
The idea is that this constant base energy consumption is not part of the new Untracked Consumption, so that the Untracked Consumption mainly consists of the extra varying energy consumption.

For that you have to create Energy sensors that calculate the constantly increasing amount of energy that is consumed by these artificial power sensors.
You can do that using the Integral Integration.
This integration calculates the Riemann sum integral of power usage, which is energy.
This is also described in Integrating individual device energy usage and Creating an Energy Sensor out of a Power Sensor.

In your case the power sensors are given in Watt or W, so the simplest method is to create Energy sensors in Watt-hour or Wh.
In that case you don’t have to define a Metric prefix (the prefix is ‘None’).

It is possible to create these Integral sensors through the UI:

Settings → Devices & services → Helpers → Create Helper → Integral sensor

But it is also possible to create them in Yaml, like this:

sensor:
  - platform: integration
    source: sensor.battery_system_base_load_current_power
    name: "Battery System Base energy"
    round: 2
    max_sub_interval:
      minutes: 1

You can then add these Energy sensors to the Energy Dashboard as extra Individual Devices, that mimic the constantly increasing used energy from these base power usage sensors.

1 Like

Thanks for the reply!

Yes 100% it’s not untracked because i know exactly where it is.

I know that the 20 tapo plugs use xx amount, i know the Solar abttery system draws xx watts per hour and i know the core house devices xx

I’ve been trying to use that Integral sensor or the utility meter, but i’ve not found it to be accurate in knocking off the untracked consumption.

I tried the above ( I think i had before) and i see errors. Any ideas on this? Maybe i need to leave it a few hours and see if data is collected. I doubt it can back date.

image

Interesting!
Indeed it can take some time for the Integral sensor to start delivering data, depending on the max_sub_interval setting.
And indeed the Integral sensors cannot backdate: I think these start from scratch as well when modified.
And Home Assistant always has to be restarted when something is modified on these sensors.

Regarding the device_class: on the Integral Integration doc page it is stated:

This configuration will provide you with sensor.energy_spent which will have your energy in kWh, as a device_class of energy.

But your error message shows that in this case that apparently is not the case.
As a test I tried to add the device_class in the Yaml config like so:

sensor:
  - platform: integration
    source: sensor.base_power_one
    name: "Base Energy One"
    unique_id: uid_base_energy_one
    device_class: energy
    round: 2
    max_sub_interval:
      minutes: 1

But then the HA Configuration check did not accept this:

So then I removed the device_class again from the energy sensor, and added it to the artificial power sensor, and that did the trick:

template:
  - sensor:
    - name: "Base Power One"
      unique_id: uid_base_power_one
      device_class: power
      unit_of_measurement: "W"
      state: "{{ 17 }}"
      icon: mdi:lightning-bolt

The energy sensor is now accepted in the Energy Dashboard as Individual device.

Note that I did use the Modern format for the artificial power template sensor, and you are still using the Legacy format, but that should not make a difference.

Yea it just took a little while for the data to begin populating and yea the backdate isn’t a too big a deal thankfully.

The data is showing for the 3 colours now which means i can deduct it really well from the untracked now!

Yea I did try a little with device class but had similar to yourself also.

I’ve just updated my energy sensors to your format above for the state and also added unique id’s.

There was a lot of inverter battery activity last night, so i do wonder if the increased untracked energy is somehow related to that. But being able to remove any known power will help to pin point and locate missing energy.

I did do a soft/cofig restart for HA. But the errors still showed so now doing a full system restart to see if it clears those errors. However, i dont think it liked it as i got a similar error log.

  - platform: template
    sensors:
      battery_system_base_load_current_power:
        friendly_name: "Battery System base load current power"
        unique_id: uid_battery_base_power
        device_class: power
        unit_of_measurement: "W"
        state: "{{ 17 }}"
        # value_template: "{{ 17 }}"
        # Could be changed to 0.28 per minute.
        icon_template: mdi:lightning-bolt
        
      smart_plugs_base_load_current_power:
        friendly_name: "Smart Plugs Base Load current power"
        unique_id: uid_smart_plugs_base_power
        device_class: power
        unit_of_measurement: "W"
        state: "{{ 29.44 }}"
        # value_template: "{{ 29.44 }}"
        # Could be changed to 0.5 per minute.
        icon_template: mdi:lightning-bolt
        
      house_items_base_load_current_power:
        friendly_name: "House items base load current power"
        unique_id: uid_house_items_base_power
        device_class: power
        unit_of_measurement: "W"
        state: "{{ 31 }}"
        #value_template: "{{ 31 }}"
        # Could be changed to 0.52 per minute.
        icon_template: mdi:lightning-bolt

So just changed the state back to the value_template. And did another full restart.
Then it booted and no error showed up. Also the energy dasboard error was removed.
image

Fingers crossed those tweaks don’t upset anything and the data continues to roll in. :slight_smile: Thankyou so much for the help, your a wizard are checking and working that one out! I think i was also changing things and expecting updates pretty instantly and also i did soft config reloads and the full re-boot i think has helped also.

Now i just have to identify the weird large unknown use somehow!

So in case anyone needs to also use the code, the actual working part i added to my config was this.

  - platform: template
    sensors:
      battery_system_base_load_current_power:
        friendly_name: "Battery System base load current power"
        unique_id: uid_battery_base_power
        device_class: power
        unit_of_measurement: "W"
        value_template: "{{ 17 }}"
        icon_template: mdi:lightning-bolt
        
      smart_plugs_base_load_current_power:
        friendly_name: "Smart Plugs Base Load current power"
        unique_id: uid_smart_plugs_base_power
        device_class: power
        unit_of_measurement: "W"
        value_template: "{{ 29.44 }}"
        icon_template: mdi:lightning-bolt
        
      house_items_base_load_current_power:
        friendly_name: "House items base load current power"
        unique_id: uid_house_items_base_power
        device_class: power
        unit_of_measurement: "W"
        value_template: "{{ 31 }}"
        icon_template: mdi:lightning-bolt