An approach to both detailed and group level energy management

@ccpk1 Wow! This is very impressive! Thanks for sharing!

I’ve got a couple questions for you, I’m very interested in utilizing the energy consumption portion of your efforts on my dashboard across 8 high usage sensors: (Washing Machine, Water Heater, Dryer, AC, etc.).

  1. I haven’t (and probably won’t) convert my Emporia to local ESPHome, so is it possible to plug in the existing entities into the utility meter to track them through the cloud reporting? (I understand this would only get feedback every 60 seconds or so)

  2. If so…I’m still a bit of a hack when it comes to YAML, can I just rip out that section of the code or is it dependent on other variables to function correctly?

  3. Lastly, does the script automatically reset itself every cycle? (Monthy/Day/Today)?

Thanks again! Looking forward to playing with this some more!

@mboarman - You can definitely do all of those things with the cloud integration to Emporia you are using. In fact, I used that integration for at least year before changing to the ESPHome version a few week ago. Agree your wattage data only updates every 60 seconds, but that is granular enough for most things anyway.

As for calculating your energy usage, you can just follow the steps in Simplifying the Energy Calculations section to capture energy usage for anything that is reporting power in watts. I mocked up the example below as if you had sensor.washing_machine_power. (It might have _1min appended if it came from the integration)

sensor:
  ################  Total hourly energy calculation  ###############
  ################  Always shows the past hour of consumption  ###############
  - platform: integration
    name: washing_machine_energy
    source: sensor.washing_machine_power
    unit_time: h
    unit_prefix: k
    round: 2

utility_meter:
  ################  Track daily consumption for each grouping  ###############
 washing_machine_daily_total_energy:
    source: sensor.washing_machine_energy
    cycle: daily
  ################  Track monthly consumption for each grouping  ###############
 washing_machine_monthly_total_energy:
    source: sensor.washing_machine_energy
    cycle: monthly

Lastly, the utility_meter integration automatically takes care of the daily and monthly cycle resets. You can see where they are defined in the example above, but there are several ways to further customize the reset cycles if you want to match them with your utility provider.

Utility Meter - Home Assistant (home-assistant.io)

Good luck

1 Like

@ccpk1 Thanks for clarifying this! I think I have it built out, but I have one additional question: I added the input number yaml to my package:

input_number:
  energy_kwh_cost:
    name: Energy kWh Cost
    icon: mdi:currency-usd
    mode: box
    unit_of_measurement: "USD/kWh"
    min: 0.001
    max: 1

But I don’t understand where I’m supposed to populate the USD/kWh state rate? What am I missing?

Edit: I set my rate: “0.10772” in the developer tab/states. hopefully that will work?

Yes, just editing it in developer tab is fine, that’s what I do. You could also add that input_number to one of your dashboards to edit in the UI, but it rarely changes, so not likely worth it.

Keep in mind, you can edit the native HA energy dashboard settings to reference that entity for your cost information. This way you only have the one source to keep up to date.

1 Like

If you’re using Influx v2 - I would love to see your examples. I have the Emporia V2 but switched over to the Iotawatt as it allowed for easy data retrieval. My Influx/Grafana skills are meager.

Thank you @ccpk1 – this is amazing!

For those that have different rates by time of day, I used the following to setup automatic rate tracking for Toronto Hydro.

Input number is used to define rates for each tariff rate.

input_number:
  energy_tou_rate_onpeak:
    name: Energy kWh Cost On-Peak
    icon: mdi:currency-usd
    mode: box
    unit_of_measurement: "CAD/kWh"
    min: 0.001
    max: 1
    initial: 0.17
  energy_tou_rate_midpeak:
    name: Energy kWh Cost Mid-Peak
    icon: mdi:currency-usd
    mode: box
    unit_of_measurement: "CAD/kWh"
    min: 0.001
    max: 1
    initial: 0.113
  energy_tou_rate_offpeak:
    name: Energy kWh Cost Off-Peak
    icon: mdi:currency-usd
    mode: box
    unit_of_measurement: "CAD/kWh"
    min: 0.001
    max: 1
    initial: 0.082
  energy_tou_rate_average:
    name: Energy kWh Cost Average
    icon: mdi:currency-usd
    mode: box
    unit_of_measurement: "CAD/kWh"
    min: 0.001
    max: 1
    initial: 0.10325

A template is used to define the name of the tariff rate, and another is used to retrieve the current tariff rate. The name is important since it will be used in utility_meter to calculate which rates apply.

The energy_tou_tariff state can be adjusted as needed

template:
  - trigger:
      - platform: time_pattern
        seconds: "/1"
      - platform: homeassistant
        event: start
      - platform: event
        event_type: "call_service"
        event_data:
          domain: "template"
          service: "reload"
    sensor:
      - name: energy_tou_tariff
        unique_id: e09d54cbfd1e4511a66a847f25e246ef
        icon: mdi:chart-bar
        state: >
          {% if now().weekday() >= 0 and now().weekday() < 5 %}
            {% if now().month >= 5 and now().month < 11 %}
              {% if now().hour >= 7 and now().hour < 11 %} onpeak
              {% elif now().hour >= 11 and now().hour < 17 %} midpeak
              {% elif now().hour >= 17 and now().hour < 19 %} onpeak
              {% else %} offpeak
              {% endif %}
            {% else %}
              {% if now().hour >= 7 and now().hour < 11 %} midpeak
              {% elif now().hour >= 11 and now().hour < 17 %} onpeak
              {% elif now().hour >= 17 and now().hour < 19 %} midpeak
              {% else %} offpeak
              {% endif %}
            {% endif %}
          {% else %} offpeak
          {% endif %}
      - name: energy_kwh_cost
        unique_id: 97041eea00d0448caf9e300ee08694a5
        unit_of_measurement: "CAD/kWh"
        icon: mdi:currency-usd
        state: >
          {% if is_state('sensor.energy_tou_tariff', 'onpeak') %} {{ states('input_number.energy_tou_rate_onpeak') | float }}
          {% elif is_state('sensor.energy_tou_tariff', 'midpeak') %} {{ states('input_number.energy_tou_rate_midpeak') | float }}
          {% elif is_state('sensor.energy_tou_tariff', 'offpeak') %} {{ states('input_number.energy_tou_rate_offpeak') | float }}
          {% else %} {{ states('input_number.energy_tou_rate_average') | float }}
          {% endif %}

Utility Meter is setup for daily and monthly cycles, and tariffs are defined. These tariff names must match the template sensor values above for automatic rate select. Once implemented, usage is then split into “buckets”, where the sensor name will now include the tariff name as a suffix.

For example, sensor.electric_home_daily_total_energy would become …

  • sensor.electric_home_daily_total_energy_onpeak for onpeak usage
  • sensor.electric_home_daily_total_energy_midpeak for midpeak usage
  • and sensor.electric_home_daily_total_energy for offpeak usage

Actual tariff names can be changed as needed, just make sure they still match. Note the use of yaml anchor on the first value using <<: &utility_meter_tariff_defaults, and all other utility_meter definitions have <<: *utility_meter_tariff_defaults instead.

utility_meter:
  ################  Track daily consumption for each grouping  ###############
  electric_home_daily_total_energy:
    source: sensor.electric_home_total_energy
    cycle: daily
    <<: &utility_meter_tariff_defaults
      tariffs:
        - offpeak
        - midpeak
        - onpeak
  other_daily_total_energy:
    source: sensor.other_total_energy
    cycle: daily
    <<: *utility_meter_tariff_defaults
# ... repeat as needed ...
  ################  Track monthly consumption for each grouping  ###############
  electric_home_monthly_total_energy:
    source: sensor.electric_home_total_energy
    cycle: monthly
    <<: *utility_meter_tariff_defaults
  other_monthly_total_energy:
    source: sensor.other_total_energy
    cycle: monthly
    <<: *utility_meter_tariff_defaults
# ... repeat as needed ...

Since utility_meter will now output usage into multiple “buckets” by tariff, total usage must then be configured to aggregate each tariff rate.

sensor:
  - platform: template
    sensors:
    ################  Calculate daily energy usage for each grouping  ###############
      electric_home_daily_total_energy_usage:
        friendly_name: "Electric Home Daily Total Energy Usage"
        value_template: "{{ states('sensor.electric_home_daily_total_energy_onpeak') | float(2) + states('sensor.electric_home_daily_total_energy_midpeak') | float(2) + states('sensor.electric_home_daily_total_energy_offpeak') | float(2) }}"
        <<: &energy_usage_defaults
          unit_of_measurement: "kWh"
          icon_template: mdi:gauge
      electric_other_daily_total_energy_usage:
        friendly_name: "Electric Other Daily Total Energy Usage"
        value_template: "{{ states('sensor.other_daily_total_energy_onpeak') | float(2)  + states('sensor.other_daily_total_energy_midpeak') | float(2) + states('sensor.other_daily_total_energy_offpeak') | float(2) }}"
        <<: *energy_usage_defaults
# ... repeat as needed ...
    ################  Calculate monthly energy usage for each grouping  ###############
      electric_home_monthly_total_energy_usage:
        friendly_name: "Electric Home Monthly Total Energy Usage"
        value_template: "{{ states('sensor.electric_home_monthly_total_energy_onpeak') | float(2) + states('sensor.electric_home_monthly_total_energy_midpeak') | float(2) + states('sensor.electric_home_monthly_total_energy_offpeak') | float(2) }}"
        <<: *energy_usage_defaults
      electric_other_monthly_total_energy_usage:
        friendly_name: "Electric Other Monthly Total Energy Usage"
        value_template: "{{ states('sensor.other_monthly_total_energy_onpeak') | float(2)  + states('sensor.other_monthly_total_energy_midpeak') | float(2) + states('sensor.other_monthly_total_energy_offpeak') | float(2) * states('input_number.energy_tou_rate_offpeak') | float(2) }}"
        <<: *energy_usage_defaults
# ... repeat as needed ...

In addition, cost must be calculated and aggregated for each tariff as well.

sensor:
  - platform: template
    sensors:
    ################  Calculate daily energy cost for each grouping  ###############
      electric_home_daily_total_energy_cost:
        friendly_name: "Electric Home Daily Total Energy Cost"
        value_template: >-
          {{ max((states('sensor.electric_home_daily_total_energy_onpeak') | float(2) * states('input_number.energy_tou_rate_onpeak') | float(2))
            + (states('sensor.electric_home_daily_total_energy_midpeak') | float(2) * states('input_number.energy_tou_rate_midpeak') | float(2))
            + (states('sensor.electric_home_daily_total_energy_offpeak') | float(2) * states('input_number.energy_tou_rate_offpeak') | float(2)) | round(2), 0.00) }}
        <<: &energy_cost_defaults
          unit_of_measurement: "$"
          icon_template: mdi:currency-usd
      electric_other_daily_total_energy_cost:
        friendly_name: "Electric Other Daily Total Energy Cost"
        value_template: >-
          {{ max((states('sensor.other_daily_total_energy_onpeak') | float(2) * states('input_number.energy_tou_rate_onpeak') | float(2))
            + (states('sensor.other_daily_total_energy_midpeak') | float(2) * states('input_number.energy_tou_rate_midpeak') | float(2))
            + (states('sensor.other_daily_total_energy_offpeak') | float(2) * states('input_number.energy_tou_rate_offpeak') | float(2)) | round(2), 0.00) }}
        <<: *energy_cost_defaults
# ... repeat as needed ...
    ################  Calculate monthly energy cost for each grouping  ###############
      electric_home_monthly_total_energy_cost:
        friendly_name: "Electric Home Monthly Total Energy Cost"
        value_template: >-
          {{ max((states('sensor.electric_home_monthly_total_energy_onpeak') | float(2) * states('input_number.energy_tou_rate_onpeak') | float(2))
            + (states('sensor.electric_home_monthly_total_energy_midpeak') | float(2) * states('input_number.energy_tou_rate_midpeak') | float(2))
            + (states('sensor.electric_home_monthly_total_energy_offpeak') | float(2) * states('input_number.energy_tou_rate_offpeak') | float(2)) | round(2), 0.00) }}
        <<: *energy_cost_defaults
      electric_other_monthly_total_energy_cost:
        friendly_name: "Electric Other Monthly Total Energy Cost"
        value_template: >-
          {{ max((states('sensor.other_monthly_total_energy_onpeak') | float(2) * states('input_number.energy_tou_rate_onpeak') | float(2))
            + (states('sensor.other_monthly_total_energy_midpeak') | float(2) * states('input_number.energy_tou_rate_midpeak') | float(2))
            + (states('sensor.other_monthly_total_energy_offpeak') | float(2) * states('input_number.energy_tou_rate_offpeak') | float(2)) | round(2), 0.00) }}
        <<: *energy_cost_defaults
# ... repeat as needed ...

Finally, an automation is configured to set the tariff rate for each utility_meter. We use an automated group to simplify rate automation.

automation:
  ################  Create and update power groupings for circuits and devices  ###############
  - alias: "Update Utility Groups"
    trigger:
      - platform: homeassistant
        event: start
      - platform: event
        event_type: "call_service"
        event_data:
          domain: "group"
          service: "reload"
    action:
      - service: group.set
        data_template:
          object_id: utility_meters
          entities: >
            {% set ns = namespace(entities=[]) %}
            {% for s in states.utility_meter if s.object_id.endswith('_daily_total_energy') or s.object_id.endswith('_monthly_total_energy') %}
              {% set ns.entities = ns.entities + [ s.entity_id ] %}
            {% endfor %}
            {{ ns.entities }}
  ################  Update power tariffs for all circuits  ###############
  - alias: "Update Power Tariffs"
    trigger:
      - platform: homeassistant
        event: start
      - platform: event
        event_type: "call_service"
        event_data:
          domain: "group"
          service: "reload"
      - platform: state
        entity_id: sensor.energy_tou_tariff
      - platform: time_pattern
        seconds: "/5"
    action:
      - service: utility_meter.select_tariff
        data:
          entity_id: group.utility_meters
          tariff: '{{ states("sensor.energy_tou_tariff") }}'

Edited: simplified yaml using anchors and automatic group creation

3 Likes

@guardmedia - Thanks for the detailed addition, I’m sure it will help several people out. I updated my original post with a link to your work so it can easily be found.

I’ll try to post some details about my Influx and Grafana in the next few weeks when I get a little free time.

1 Like

Thanks @ccpk1 for sharing this project. I would like to create a light version of that.
I would like to calculate my total energycost for each month.

In general I want an overview for each month

  • used electricity from the grid in peak period x electricity cost per kWh that month
  • used electricity from the grid in low period x electricity cost per kWh that month
  • used amount of natural gas x cost that month per m3 that month
  • injected electricity in peak periode x electricity cost per kWh that month
  • injected electricity in low periode x electricity cost per kWh that month
    Calculate the YTD total cost in order to pay a correct advance to my energy supplier.

One problem I not that familiar with yaml programming as you do. Nor with the creation of custom sensor :smiley:
So I you feel comfortable to help me a bit, thanks in advanced!!

@ccpk1 I’ve been using a variation of your work in my dashboard and absolutely love the way it’s formatted and delivered! I only find myself wishing for one small additional…and to be honest, I don’t even know if it’s possible!

I would like to add a fourth column that shows the previous months Kwh consumption for each sensor so I could see how I’m tracking against it. Is there a way to capture the last day of the month data from the month sensor and store it to display in an additional column?! Thoughts?

Thanks for this! This has really encouraged me to do something similar. I just purchased and flashed an Emporia and working on getting everything set up.

Are you able to share how you track if your washing machine is on and how many times it has ran?

Thanks!

Edit:
Nevermind. Figured out a way! Thanks again!

@ccpk1

Thank you for the information.

How do you reset the data each month? Is there any automation?

This looks awesome, thanks for sharing.
I’m new to the energy consumption side of homeassistant. Hoping to make sense of the out of the box functionality before I delve in to yaml.
Could anyone point me to an energy dashboard idiot’s guide? Beyond adding individual devices (I’ve only got a couple) I can’t figure out how to get any of the data in to a nice fancy dashboard.
Thanks,
matt

@ccpk1 would you mind sharing your esphome yaml config files for the VUE2 devices themselves?

I have 3 vue devices, but they are all under 1 main power supply. 2 vue devices needed in the main panel due to the quantity of circuits, and another vue in a subpanel that is feed by the main panel.

So I’m trying to decide on a naming structure for the system that can be adapted to your naming methods.

@cliffkujala Here is a link to my esphome yaml for my vue in panel 1. https://gist.github.com/flaviut/93a1212c7b165c7674693a45ad52c512?permalink_comment_id=4058452#gistcomment-4058452
Panel 2 is exactly the same except the circuit names / configs where I use “p2” in the naming instead of “p1”. My setup is obviously a little different than yours, but if I were you, here is how I would handle it:

  • Main Panel Vue1 - Configure to monitor total power coming into your home as well as 16 circuits. In your esphome configuration, be sure calculate the “Balance Power” which is essentially the power used by all circuits not individually monitored with this device. This is important because in home assistant you will use this “Balance Power” figure to subtract out the circuits monitored with your Vue2 and Vue3 devices.
  • Main Panel Vue2 - Maybe you’ve already thought of a better way, but it seems to me you would still need to configure this to monitor the same primary feed coming into the panel. In this case you would likely disregard the total power monitored with this device since it will be the about the same as what you already know from Vue1. You would then have the monitoring for 16 more circuits which, as stated above, need to be subtracted from your balance power of Vue1.
  • Sub Panel Vue3 - Even though this is fed from the main panel, I’m assuming you only monitor the total power of this sub panel from the Vue3. (i.e. no CT’s on this feed in your main panel) In this case, you will have the total power monitored in this sub panel, as well as 16 circuits. There are a few options here, but the easiest is just to subtract the Vue3 total panel from your balance power of Vue1. (alternatively, you could subtract out all of your individual Vue3 circuits and the Vue3 balance from that Vue1 balance, but I don’t think it buys you anything)

The method I was describing above should ensure that the power figures of your individual circuits are all tying out to the total power delivered to your home. The specific naming you use for your entities just needs to be consistent and unique so your are able to easily build your groups for the calculations. (i.e. putting all of your Vue2 circuits into a group which you will then use to add the value of the state of each member so you can subtract from the Vue1 Balance Power.) You might use something as simple as v1_01 - v1_16, v2_01 - v2_16, v3_01 - v3_16.

Hope that helps

@mboarman I think in the future there will be some way to access the long term statistics that hold these values when you look at the energy dashboard, but from what I can tell there aren’t any good programatic ways to get them now. In the meantime, I would probably write an automation with a trigger in the last few minutes of the month to capture the value into an input_number before the reset. Alternatively, you could probably put an automation in with a trigger on “sensor.electric_home_monthly_total_energy” that would trigger when the reset occurs and capture that prior value which would have been your monthly total.

Amazing work and skill, wow
Im brand new to HA, been using emporia for a year.
maybe you can answer a question for me
I am also using the magico13 method, the HA energy dashboard does not seem to see any of the 1 minute sensors. I know they are avbl as I can use them on my own dashboard.
right now all Im trying to do is make a card like the HA energy device card where it can show all the circuits, but I want to use the 1 minute sensors not the 1 day or month sensors
any idea what the issue is ?

anyone knows why this does not count correct?

- name: power_g_light_total
  icon: mdi:lightbulb-group
  state: >-
    {% set virtualpower = expand('group.power_g_light_total') | rejectattr('state', 'in', ['unavailable', 'unknown']) | map(attribute='state') | map('float') | sum | round(2) %}
    {{ max( virtualpower | round(1), 0.0 ) }}
  attributes:
    tmp_friendly_name: "Light Group Total Power"
    <<: *power_sensor_force_update
  <<: *power_sensor_defaults

- name: power_g_hvac_total
  icon: mdi:lightbulb-group
  state: >-
    {% set virtualpower = expand('group.power_g_hvac_total') | rejectattr('state', 'in', ['unavailable', 'unknown']) | map(attribute='state') | map('float') | sum | round(2) %}
    {{ max( virtualpower | round(1), 0.0 ) }}
  attributes:
    tmp_friendly_name: "Light Group Total Power"
    <<: *power_sensor_force_update
  <<: *power_sensor_defaults

- name: power_g_hvac_total_cost
  attributes:
    tmp_friendly_name: "Light Group Total Cost"
  state: "{{ max( states('sensor.power_g_hvac_total') | float / 1000 * states('sensor.nordpool_kwh_krsand_nok_3_095_025') | float, 0.0) }}"
  <<: &cost_sensor_defaults
    unit_of_measurement: "NOK"
    state_class: measurement

- name: power_g_light_total_cost
  attributes:
    tmp_friendly_name: "Light Group Total Cost"
  state: "{{ max( states('sensor.power_g_light_total') | float / 1000 * states('sensor.nordpool_kwh_krsand_nok_3_095_025') | float, 0.0) }}"
  <<: *cost_sensor_defaults

@ccpk1
I recently installed an Emporia Vue (using ESPHome) and stumbled on this approach; I really like it.

A few questions:

  1. Are you still using this approach? Any changes or improvements?
  2. You use a template to abstract every power entity into a “virtual” sensor, regardless if it comes from Vue, a smart plug, smart switch, etc?
  3. You only store data (via recorder) for the “virtual” entities?
  4. Did you post your Influx/Grafana details somewhere?
  1. Yes still using this approach, it’s been very reliable.
  2. That is correct. It was more work up front, but I wasn’t able to figure out another way to have the control required to manage this cleanly otherwise. i.e. What do you want to happen when you add any new device with a power entity
  3. That is correct. With that many entities and constant value changes, it can have a performance impact otherwise. Also note it’s important to use trigger templates as stated in the post, otherwise you will likely have performance issues.
  4. I don’t think I did. I don’t use it very often, so I’ve somewhat forgot many of the details. If someone knows a simple way to extract the configuration from those and show it, let me know.