Energy Disaggregation or Non-Intrusive Load Monitoring (NILM) blueprint

Background

I have a power meter measuring current (A), voltage (V), and frequency (Hz) for each of my 2 phases 10 times per second. I already had smart plugs for many loads to monitor energy and get notifications (e.g. washer finished) but I didn’t have any for the larger 240V loads such as dryer/oven/stove.

There is a lot of academic research on Energy Disaggregation or Non-Intrusive Load Monitoring (NILM). This picture from the original paper demonstrates the problem

The large loads of interest have a pretty distinctive signature, a sudden increase followed by an almost equal sudden decrease some time later, often many times until the appliance is off. Given that, I thought I should be able to achieve energy disaggregation only for resistive or similar loads using derivative sensors and just yaml code in Home Assistant.

Following image shows the current for each phase, their derivatives, and the estimated power of my dryer using the below setup:

Prerequisites

1. Create derivative sensors for each phase current.

Go to Settings > Devices & Services > Helpers. Click on Create helper. Select Derivative sensor. Give a Name, select the Input sensor with the phase current, set the Time window to e.g. 1 minute or 10 seconds and Time unit to minutes or seconds. You will have to experiment what value works for you. For me, 1 minute works well except for the rice cooker and microwave especially if the duration on the microwave is less than 1 minute. I’m currently using 1 min for all and 10 seconds for the rice cooker and microwave.

Appliances

For each appliance of interest:

1. Create a number helper to hold approximate power in W

Go to Settings > Devices & Services > Helpers. Click on Create helper. Select Number. Give a Name e.g. “Dryer power”, change Maximum value to something larger than the expected power of the appliance e.g. 6000 for dryer, and set Unit of measurement to W.

2. Create a timer helper to avoid thinking the appliance is running forever

Go to Settings > Devices & Services > Helpers. Click on Create helper. Select Timer. Give a Name e.g. “Dryer timer”, select Restore, and set a Duration to e.g. 5 minutes. This needs to be larger than the time the draw stays elevated. e.g. my dryer seems to draw maximum current for ~2 minutes, followed by very low current for ~1 minute and the cycle repeats until done. For EVSE you would need to set this to several hours depending on what’s the maximum time of you EV charging. This timer helper is needed if the automation misses the sudden decrease for whatever reason. At the end of the timer the automation sets the power of the appliance to 0.

3. Create an energy sensor

This is optional. Only if you want to track the energy in the energy dashboard under the individual devices.

After 2023.11.0, go to Settings > Devices & Services > Helpers. Click on Create helper. Select Integration - Riemann sum integral sensor. Give a Name e.g. “Dryer energy”, select Input sensor the number helper created at step 1, select Left Riemann sum. Finally add the created energy sensor in the individual devices of your energy configuration.

Before 2023.11.0, in /config/configuration.yaml add:

sensor:
  - platform: integration
    unique_id: dryer_energy
    name: "Dryer energy"
    source: input_number.dryer_power
    unit_prefix: k
    unit_time: h
    method: left

In /config/configuration.yaml add:

homeassistant:
  customize:
    sensor.dryer_energy:
      device_class: energy

4. Create a template binary sensor whether the appliance is running.

This is optional. Only if you want to have automations when the appliance finishes, e.g. notify when the dryer is done.
In /config/configuration.yaml add something like the following. The delay_off needs to be larger than the duration the appliance stays at low draw, e.g. for my dryer it needs to be at least 1 minute. Having this too short might trigger premature notifications that the appliance finished. Having it too large will delay the notification.

template:
  - binary_sensor:
      - name: Dryer running
        unique_id: dryer_running
        device_class: "running"
        icon: mdi:tumble-dryer
        delay_off:
          minutes: 5
        state: "{{ states('input_number.dryer_power') | float > 0 }}"

5. Create an automation using this blueprint for 2 phases or this blueprint for 3 phases or this blueprint for 1 phase.

e.g. for my dryer I have:

alias: "Energy Disaggregation: Dryer"
description: ""
use_blueprint:
  path: energy_disaggregation.yaml
  input:
    power_helper: input_number.dryer_power
    timer_helper: timer.dryer_timer
    l1_current_derivative: sensor.micro2wallbox_current_amp_l1_min_derivative
    l2_current_derivative: sensor.micro2wallbox_current_amp_l2_min_derivative
    current_derivative_factor: 1
    l1_voltage: sensor.micro2wallbox_voltage_l1
    l2_voltage: sensor.micro2wallbox_voltage_l2
    l1_lower_bound: 19
    l1_upper_bound: 24
    l2_lower_bound: 17
    l2_upper_bound: 24
    delay:
      hours: 0
      minutes: 0
      seconds: 5

for my microwave I have:

alias: "Energy Disaggregation: Microwave"
description: ""
use_blueprint:
  path: energy_disaggregation.yaml
  input:
    power_helper: input_number.microwave_power
    timer_helper: timer.microwave_timer
    l1_current_derivative: sensor.micro2wallbox_current_amp_l1_10s_derivative
    l2_current_derivative: sensor.micro2wallbox_current_amp_l2_10s_derivative
    current_derivative_factor: 10
    l1_voltage: sensor.micro2wallbox_voltage_l1
    l2_voltage: sensor.micro2wallbox_voltage_l2
    l1_lower_bound: 14
    l1_upper_bound: 18
    l2_lower_bound: -2
    l2_upper_bound: 2
    delay:
      hours: 0
      minutes: 0
      seconds: 3

You will have to pick the lower and upper bounds of increases/decreases by inspecting your history.

For getting notified the dryer finished I use the following automation:

alias: "Notify: Dryer finished"
description: ""
trigger:
  - platform: state
    entity_id:
      - binary_sensor.dryer_running
    from: "on"
    to: "off"
condition: []
action:
  - service: notify.nikos
    data:
      title: Dryer finished
      message: "{{ now().strftime('at %-I:%M %p on %m/%d/%Y') }}"
mode: single
5 Likes

Cool!

I have 3 phases so it will need some modification before I can try it out.

Any suggestion for a local (either wifi or zigbee) sensor with high sensibility and update frequency? I’m currently using a tuya wifi power clamp and I really doubt it is enough

I updated my post with a link to a blueprint for 3 phases.

My power meter is a Carlo Gavazzi EM530 paired with a Wallbox Pulsar Plus and I get the data via MQTT from the Wallbox. I don’t have experience with any other meter. What update frequency do you have? If in your history you see anything similar to the stove and oven signatures in the image of the first post you should be able to use this method.

My device updates every 5 to 10 seconds, I’ll give it a try even if I doubt it can be that effective.

Yeah nice. I played with it in my setup and modified it with both 1 phase and 3. It does not work well with my current sensors. The data have only one decimal and updates only on change. And when my solar panels produces the current sensors are not to any use.

But! I modified it to look at my power sensor. Now I have some appliances that work ok and I have some more testing to do. Really interesting.

My sensors also have only one decimal and update only on change. If you can separately track your solar production you can try creating a template sensor that subtracts it to only get consumption data.

Nice work, tronikos! Unfortunately I have some problems to get the energy sensor summing up. If the current stays constant over a longer time period the derivative drops down to zero. The blueprint takes the derivative and multiplies it with the voltage to calculate the energy, thus with a derivative of zero, no energy is calculated. What is wrong in my understanding!

Just realised my misinterpretation of the automation. The energy sensor is only updated when the automation is triggered, i.e. if the derivates changes within the values entered. Thus the value of the energy sensor, which is used in the integral calculation, remains constant or at least is updated with the new derivates values. Hope I got t wright!

Yes the automation assumes both the current and voltage remain constant throughout the load. That’s why I said in the first post that this works only for resistive or similar loads and that it’s an approximation. Note the automation can correctly detect multiple simultaneous loads as long as they haven’t started at the exact same time.

I have access to the consumed power directly (not current and voltage), and I have a single phase. Could someone help me modify the blueprint ? I suppose I need to create a derivative on power (instead of current) ? Remove all l2 values ?
Thanks!

Hi,
I am really interested in a 1 phase version of the blue print.
What are the modifications required in this case?

So quick question:
Im guessing that you have a 1 phase microwave. If so, are you using the 2-phase blueprint so that you don’t get false alerts with a similar device that is 2-phase?

Correct. I have a 1 phase microwave. I’m using the 2-phase blueprint to avoid false triggering with the oven/stove that is pulling similar load but for both phases. If you notice in my 1st post for the microwave automation I require 14A to 18A increase for the phase the microwave is and no increase (well up to 2A decrease or decrease) for the other phase.

I updated the first post with a link to a blueprint for one phase.

Great. That was what I was guessing. Great blueprint

First of all, thanks a lot, this really great !
One question if someone may have an idea.
The power returned by the number helper is totally out, I’m measuring 140W for my appliance, and the helper returns 2800W.
What could I do wrong ?

How many phases do you have? What time window did you use for the derivative sensors? What current_derivative_factor do you use in the blueprint?

I’ve 3 phases in my house, 6 ct clamps on 6 ads1115 on 2x ESP32.
1 ct clamp per phase, and 1 per room (single phase as well).
I’m focusing for now only on 1 ct clamp, single phase.
Time window is set at 10s.
Current_derivative_factor is at 1. I created a voltage sensor with a factor (divide by) 6.28 and it’s closer to the expected value.