Enphase Envoy on 2024.4

It’s been a few years since I wrote my original guide on how to get Enphase Envoy working with Home Assistant Energy Dashbard.
I figured I would try start from scratch to document what the process and experience would be like for a brand new user with Home Assistant version 2024.4.

Let’s start with my solar setup.
I live in Australia and since 3 years ago in April 2021, I have owned an Enphase Envoy that looks like this on the left and has 2 CT clamps as shown on the right for consumption/production monitoring.

image

Details obtained from the device remotely by visiting https://envoy.local/home and logging in to it using my enphase enlighten/enphase cloud credentials.

Part Number
800-00654-r06
Software Version
D7.6.175 (f79c8d)
Software Build Date
22 Jun, 2023 8:43 PM

In all of the following screenshots, I will be removing my serial number and IP address information for privacy.

New Home Assistant installation

I setup a brand new copy of Home Assistant and once it was finished installing, I found that Home Assistant had automatically discovered my Envoy on the wifi network.

image

I pressed configure and was asked for my Enphase cloud credentials.
To be clear, these are the same credentials you use to login to https://enlighten.enphaseenergy.com/

Once those credentials are accepted, it then shows a Success! message

image

Observations:

  • All power sensors are in kW. (1,000 W = 1 kW)

  • All energy sensors are in kWh. (1 kW of power continuous load over 60mins = 1 kWh of energy consumed)

  • All lifetime energy sensors are in MWh. (1,000,000 Wh = 1,000 kWh = 1 MWh)

  • Each sensor is updated every 60 seconds.

Energy Dashboard

It is pointless trying to get these Enphase provided energy sensors to be used by the Energy Dashboard in their default state as the energy data is different to what is expected by Energy Dashboard, but that doesnt mean we can’t improvise.

The Envoy integration provides sensors for:

  • house consumption instead of grid consumption
    Note: If the house consumes more than the solar system is producing, the house will import the excess power difference it needs from the grid.

  • solar production instead of return to grid
    Note: solar production will first be consumed by the house and if production power is higher than what the house consumes, the excess power difference will be returned to the grid.

As it is physically impossible for your house to be importing and exporting power at the exact same time, we can exploit the mathematical difference between solar production and house consumption to calculate if your house is importing power from the grid or exporting power to the grid.

To get Home Assistant to calculate this for us, we have to setup a template sensor in Home Assistant to calculate what Energy Dashboard is expecting.

The guide is focussed on 2 steps.

Step 1 - Setting up template sensors to perform calculations using data from the enphase sensors.
The template sensors we need to create are:
power import in W (grid in)
power export in W (grid out)

Step 2 - Setting up Riemann Sum sensors which convert:
power import in W to energy import in kWh
power export in W to energy export in kWh

Note: Some people may ask “can’t I have everything in kW?”.
The answer is that the Riemann Sum sensors expect the data source to be in W.
Watts are the only way I could get the Riemann Sum sensors to work.

Edit, I just tried it without specifying the metric prefix, and it will work with whatever source metric you are using.
The issue is, once you click on a metric prefix, you cannot seem to unclick it in the GUI.

If you mess this up, you can correct it in Developer tools

Step 1a - Create a Power Export Template Sensor

Go to settings
image

Devices and services
image

Helpers
image

+Create Helper
image

{} Template
image

Template a sensor
image

Template sensor

Name: Power Export

State template:

{{
[ 0,
states('sensor.envoy_SERIALNUMBER_current_power_production')
| float(0) * 1000
-
states('sensor.envoy_SERIALNUMBER_current_power_consumption')
| float(0) * 1000 ]
| max | int(0)
}}

unit_of_measurement: W
device_class: Power
state_class: Measurement

Press Submit

Step 1b - Create a Power Import Template Sensor

Go to settings
image

Devices and services
image

Helpers
image

+Create Helper
image

{} Template
image

Template a sensor
image

Template sensor

Name: Power Import

State template:

{{
[ 0,
states('sensor.envoy_SERIALNUMBER_current_power_consumption')
| float(0) * 1000
-
states('sensor.envoy_SERIALNUMBER_current_power_production')
| float(0) * 1000 ]
| max | int(0)
}}

unit_of_measurement: W
device_class: Power
state_class: Measurement

Press Submit

Step 1 complete

Step 2a - Create Energy Export Sensor

Go to settings
image

Devices and services
image

Helpers
image

+Create Helper
image

Integration - Riemann sum integral sensor
image

Add Riemann sum integral sensor
Name: Energy Export
Input Sensor: Power Export
Integration Method: Right Riemann Sum
Precision: 3 is default
Metric Prefix: k (kilo)
Time Unit: Hours

Press submit.

Step 2b - Create Energy Import Sensor

Go to settings
image

Devices and services
image

Helpers
image

+Create Helper
image

Integration - Riemann sum integral sensor
image

Add Riemann sum integral sensor
Name: Energy Import
Input Sensor: Power Import
Integration Method: Right Riemann Sum
Precision: 3 is default
Metric Prefix: k (kilo)
Time Unit: Hours

Press submit.

You should now have
2 template based Power sensors
and
2 Integration based Energy Sensors

Step 2 complete.

Note: As it is night time right now for me, my Energy Export sensor will not work at all until tomorrow when the sun starts shining again.

If it is night time, the energy sensor will not be formatted correctly yet.
You can tell due to the subtle difference between the image icon and the image icons for the Energy sensors.

For now, I can go to
Energy
image

and then setup Grid Consumption
image

Choose the Energy Import sensor you just created.
image

I then added my Solar production energy using the sensor:
Envoy SERIALNUMBER Energy production today
image

I then left these overnight to see what happened and what I found was that I had a small amount of production energy occuring overnight which is physically impossible when the sun is below the horizon and something I want to fix in in a post after this post.

3 Likes

Thanks for the work, very clear to follow. I just have a quick question. I see a lot of people using Riemann Sum for utility meters but tbh I haven’t done enough researches to understand why, cause I’m using the directly added values, i.e: sensor.envoy_SN_energy_production_today, or sensor.envoy_SN_current_power_production. What is the advantages of setting up utility meters using Riemann Sum? Thanks again

Hi @xzelz,
The Utility Meter - Home Assistant is used for having multiple tariffs where your provider will charge you different rates at different times of the day.

Here is an example of what is possible with utility meter with 3 import tariffs and 1 export tariff.

image

I have written other guides in the past on how to use utility meter and will eventually get to writing a guide to do it via the GUI in this topic in the near future.

For now, here is a quote from one of my most recent previous guides

1 Like

Thanks @del13r . I’ve seen another post of yours where you used Rest sensors. Have you now abandoned that in favour of the integration?

Hi @Sddawson,

This guide above is for the most basic, bare minimum steps to get a brand new user working with energy dashboard.

I still use the rest sensors in my own setup, however these are a bit more complex for new users so I have chosen not to mention them here.

I wrote this guide as I could tell something had changed with the native enphase integration when I started seeing people trying to multiply kW by 1000 to get W and then asking for help. This indicated to me that my 3 year old guide was out of date and hence why I wrote an updated new user guide above.

OK, makes sense, thanks. When I had an Envoy, I used all your knowledge to integrate that into HA very successfully. I’m about to get a solar quote for a different house, and will probably go Envoy again (in Oz), just because there are guides like yours to follow. I’ll probably go the Rest route. Thanks for everything you do.

1 Like

Thank you for the detailed comment. I do have my setup somewhat similar to yours but I mostly created my sensors with Utility Meter UI and calculations in configuration.yaml and I was not sure if it’s the same as using Riemann sum, I think I get it now. If I was able to read your post earlier I would not do all the energy calculations by myself haha, took me quite some time to finish. I also did calculation for daily power usage, daily solar power usage etc. My electrical plan do onpeak/offpeak/midpeak. The offpeak includes weekend, and holidays as well.

Here is a quick look of my setup:

1 Like

Using Riemann sum integral alone will not calculate costs.

An example of what Riemann sum integral is used for is for recording the instantanous power figures in W and converting them to energy figures in kW.

If I turn on an electrical device that steadily consumes 1030 W and leave that device on for an hour, then that device has consumed 1.03 kWh over the course of an hour.

Power in W - real world example

Riemann sum integral then incrementally records how much power the device consumed each time the power sensor updates and graphs those samples over time.

Energy in kWh - real world example generated by Riemann sum integral using only the power sensor above as the data input source.

Energy Dashboard basically samples the Lifetime Energy Value at both the start and end of each hour and records only the difference.

Energy Dashboard Samples Value Metric
Lifetime energy value at 3:00 PM 2182.52 kWh
Lifetime energy value at 2:00 PM 2181.49 kWh
Difference between start and end 1.03 kWh

How Energy Dashboard presents this same data.

I use both the built in energy dashboard and additional template sensors because energy dashboard does not support my daily service charge.

Here is a sample of how I am using utility meter readings and adding the daily service charge.
Note: this sensor is not used by Energy Dashboard and is only for my own benefit unrelated to energy dashboard.

  - sensor:
      - name: Today Energy Total Cost
        device_class: monetary
        state_class: total
        unit_of_measurement: AUD
        state: >
          {{ (states('sensor.daily_energy_Peak') | float(0) * (states('input_number.peak') | float(0) * states('input_number.discount') | float(0) ) +
              states('sensor.daily_energy_Shoulder') | float(0) * (states('input_number.shoulder') | float(0) * states('input_number.discount') | float(0) ) +
              states('sensor.daily_energy_OffPeak') | float(0) * (states('input_number.offpeak') | float(0) * states('input_number.discount') | float(0) ) -
              states('sensor.daily_energy_export_buyback') | float(0) * states('input_number.feed_in') | float(0) +
              states('input_number.dailysupply') | float(0) * states('input_number.discount') | float(0) ) | round(2) }}

Here is an example graph of 24 hours where I start the day with my $0.98 service charge, then the cost rises as I consume electricity before the sun rises, then the cost drops as I sell electricity back to the grid, then rises again once the sun sets finishing the day with a total of $2.20 charge.


r

Thank you @del13r for your amazing guide so far.
I’ve been putting together my dashboard using this guide and also parts of your old guide from 2021 - looking forward to more updates.

I’m a relatively newbie with Home assistant and YAML, but using a combination guides from here, YouTube and ChatGPT, I’ve built a functional dashboard to suit my purposes so far.

I’ve run into a couple of snags and wondering if anyone could shed some light…

  1. I’ve been trying to use your old code to generate a sensor for Corrected Solar Power/Energy, as my Envoy generates ~20-25W overnight whilst the sun is down. Using the old Code you’ve provided in your 2021 guide, it results in my solar production numbers being dramatically different compared to the numbers provided by the Envoy integration directly.

  2. My energy retailer similarly has a daily supply charge in addition to a stepped tariff (i.e. above/below 30kWh of import per day). It also has a TOU tariff for the solar feed in. Whilst my export tariff works well, I’ve been trying to create a sensor/entity to track the daily import cost as a “total” cost. Whilst the sensor seems to work and track my total daily cost (I can output to a graph and it seems to be tracking appropriately), for the life of me I can’t select it as an option in the energy dashboard to represent the “total cost” of energy import.

Any feedback on the YAML is welcome (it has been generated by ChatGPT).

sensor:
  - platform: template
    sensors:
      export_tariff:
        friendly_name: "Export Tariff"
        unit_of_measurement: "AUD/kWh"
        value_template: >-
          {% set current_hour = now().hour %}
          {% set peak_start_hour = 16 %}
          {% set peak_end_hour = 21 %}
          {% set offpeak_start_hour = 10 %}
          {% set offpeak_end_hour = 14 %}
          {% set shoulder_periods = [(21, 24), (0, 10), (14, 16)] %}
          {% set peak_tariff = 0.106 %}
          {% set offpeak_tariff = 0.039 %}
          {% set shoulder_tariff = 0.055 %}
        
          {% if current_hour < 24 %}
            {% if peak_start_hour <= current_hour < peak_end_hour %}
              {{ peak_tariff }}
            {% elif offpeak_start_hour <= current_hour < offpeak_end_hour %}
              {{ offpeak_tariff }}
            {% else %}
              {% for start_hour, end_hour in shoulder_periods %}
                {% if start_hour <= current_hour < end_hour %}
                  {{ shoulder_tariff }}
                  {% set shoulder_tariff = 0.0 %}
                  {% break %}
                {% endif %}
              {% endfor %}
            {% endif %}
          {% endif %}

      energy_cost:
        friendly_name: "Daily Energy Cost"
        unit_of_measurement: "AUD"
        value_template: >-
          {% set daily_import_kWh = states('sensor.daily_energy_import') | float %}
          {% set supply_charge = 0.93 %}
          {% set tariff_below_30_kWh = 0.198 %}
          {% set tariff_above_30_kWh = 0.242 %}

          {% if daily_import_kWh < 30 %}
            {{ (daily_import_kWh * tariff_below_30_kWh) + supply_charge }}
          {% else %}
            {{ (30 * tariff_below_30_kWh) + ((daily_import_kWh - 30) * tariff_above_30_kWh) + supply_charge }}
          {% endif %}

Whilst I’m here, our energy distributor has a max inverter size of 10kW, but an export limit of 5kW - any idea how to code for this?

Energy cannot disappear into thin air, so what happens when export exceeds the limit? does the inverter shut off, or is it only that the energy company won’t pay you for the excess?

In any way, if you limit the return sensor to a maximum, the dashboard will assume the house consumed it. So that is not an option.

Let me start off by saying this is a nice post, it will help for people who lack some of the sensors. But there are some tips, or things to consider, depending on what you have in terms of sensors:

This is only true if you have one cable coming in your home. A lot of homes in the Netherlands for instance have three phases coming in. Solar can either be on one or all three. So this rule applies per cable. If you have more you can export on one and import on another. That actually happens in my home. So if you can measure the actual values (for instance use the values from the meter) use those. It is the preferred way. If you have only one power cable coming into your home, then the calculation is valid.

This isn’t a valid reason. Rieman sum converts (k)W in (k)Wh. If you have (k)Wh that is always more accurate than the result a Riemann Sum Integral, which estimates it. having a Riemann sum integral is not a goal in itself, if you have (k)Wh then you don’t need Riemann sum.

Especially from Enphase you only get (k)W samples very infrequently, and it can vary a lot between samples. And if the samples of production and use aren’t taken at the same time, the difference between them can lead to nonsensical values. So if you have (k)Wh, and measure all of them, it is better to use those.

What the (k)W calculations do help with, is that you can import one minute, and export the other. (k)Wh sensors are not going to help you calculate that. They will cross them out against each other. That is where your calculations comes in handy. But do remember, Enphase only reports data infrequently, so Watts will be an approximation. The infrequent samples will miss things.

So For many reasons: Measure if you can, and use (k)Wh if you can.

Really nice work @del13r, thanks.

Quick question on:

house consumption instead of grid consumption

solar production instead of return to grid

Is your consumption CT installed as Load only CT (total-consumption)?

If you have it installed as Load with Solar (net-consumption) you should have net lifetime energy production and net lifetime energy consumption to use as grid export/import as of HA 2024.3. These come from the actEnergyRcvd and actEnergyDlvd of the consumption CT, if a net-consumption type.

It is pointless trying to get these Enphase provided energy sensors to be used by the Energy Dashboard in their default state as the energy data is different to what is expected by Energy Dashboard.

I think with a Load with Solar / net-consumption CT these net values can be used with the Energy dashboard.

For ‘load only’ / total-consumption CT you are right, not usable. Your explanation can be used with Load with Solar / net-consumption CT as well of course, but just making sure these net entities are available for use in the dashboard.

I then added my Solar production energy using the sensor:
Envoy SERIALNUMBER Energy production today

My experience is that there are some firmware versions out there that have shown some unexpected behavior in the day (and last 7 day) values and the Envoy SERIALNUMBER Energy lifetime production value turned out to be more reliable overall and might be prefered for use in the Energy dashboard.

Again, excellent piece of documentation!

Hello @catsmanac,

My system is “net-consumption = Load with solar production”.

The native enphase integration does provide those energy sensors for me.

I really have to disagree with your statement about using any of the energy sensors, provided by the native enphase integration, for grid import/export.

Here are 2 diagrams to help explain why i disagree.
The text in red is what the envoy provides.
The text in green is what home assistant needs to calculate using template sensors.

Energy:
image

Now lets try the difference logic against the energy sensors:
40.6 kWh house consumption - 21.5 kWh solar production = 19.1 kWh grid import.
This logic DOES NOT work.

Power:
image

Lets try the difference logic against the power sensors:
446 W house consumption - 102 W solar production = 344 W grid import
This logic DOES work.

To further explain, at this moment in time, my house is consuming power from 2 sources at the same time:
102 W from solar production
344 W from the grid
Add these 2 together and 446 W is my house consumption

I hope this explains why I disagree with your above statement regarding trying to use Enphase provided energy sensors for anything to do with grid import / grid export.

For more information about where the native enphase integration gets its data from, see this post:

Hi @del13r,

thanks for your detailed reply and the validation that the sensors are actually there.

My system is “net-consumption = Load with solar production”.

The native enphase integration does provide those energy sensors for me.

As I added these to pyenphase and the core integration I wanted to make sure these were there, be it useful or not.

As for your power logic, that is sound and can be used to calculate as you do. I’ve learned that the Riemann is remarkably close to what Envoy reports.

I really have to disagree with your statement about using any of the energy sensors, provided by the native enphase integration, for grid import/export.

Understand and I fully respect that. As the grid import/export is a fairly recent add I did, I’m in search of understanding issues and perspectives for these. All values are as send by the Envoy and may need interpretation as you advocate.

For background on the grid import/export sensors, these are not sourced from /production.json but from /ivp/meters/readings[1]. These report (amongst others)

 "actEnergyDlvd": 3183793.885,
 "actEnergyRcvd": 8851.704,
 "activePower": 488.925,

For the net consumption CT actEnergyDlvd is lifetime grid import energy delivered to the switchboard and named Lifetime net energy consumption. actEnergyRcvd is the lifetime export available as Lifetime net energy production and activePower the actual Power in or out as Current net power consumption.

This logic DOES NOT work

Am I correct when I think I hear you saying: When I configure the HA Energy dashboard using Lifetime net energy consumption and Lifetime net energy production for grid import/export and Lifetime Energy Production for solar production, then the House consumption is not correct!? Or even ‘values are not correct’!?

If this is like the 10th time you are asked to explain and have had it by now, please say so, no problem, I would understand.

Tnx.

Hey @catsmanac,
I disagree with your source of the grid import/export sensors as the native integration does not provide grid import/export sensors.
I have previously shown that the native Enphase integration leverages pyenphase to get all of the power and energy data from:

URL_PRODUCTION_JSON = “/production.json”

from that endpoint, these 4 x consumption and 4 x production (8 total) sensors are created by the native integration

watt_hours_lifetime=int(round(consumption[“whLifetime”])),
watt_hours_last_7_days=int(round(consumption[“whLastSevenDays”])),
watt_hours_today=int(round(consumption[“whToday”])),
watts_now=int(round(consumption[“wNow”])),

See the native integration code here

I noticed that you have mentioned “/ivp/meters/readings[1]”.
I looked into this data source about a year ago.
Please see my article below where I break it down.

Hi @del13r, thanks for your explanation. Again, no issues with the Power calculations.

My point is towards energy sensors, not to the power sensors. Grid Energy import/export for use with HA Energy dashboard actually are available since HA 2024.3 in pull request 110884, data provided by pyenphase in PR 133 and PR 135.

Pyenphase is getting the energy data from json[1].actEnergyDlvd and json[1].actEnergyRcvd as you describe in Enphase Envoy - D7 firmware with JWT - A Different Approach. Since this addition one can opt in HA Energy dashboard to:

  • use the Envoy Lifetime net energy consumption entity for Grid consumption
  • use the Envoy Lifetime net energy production entity for Return to grid

So again, not disagreeing with how you calculate Energy from all power sensors. Just trying to share that the native integration has Grid Energy import/export these days from the data sources you describe in your article.

1 Like

Hi @catsmanac,
Aah, OK, I finally see the point you are making.
I did not realise who I was responding to.
Apoligies for missing the point.
I am obviously not up to date on current affairs regarding the native integration :slight_smile:.

I have setup a new HAOS install for testing and have indeed now noticed the new ‘net energy’ sensors:

To confirm, I go to https://envoy.local/ivp/meters/readings and see what is reported there
production “actEnergyDlvd”: 35998263.861,
import “actEnergyDlvd”: 16042946.210,
export “actEnergyRcvd”: 14299294.002,
These all match up with what the lifetime sensors are showing.

My initial observation is that the lifetime figures dont fully match what is shown in
https://enlighten.enphaseenergy.com/web/

While the lifetime energy consumption and lifetime energy production sensors are close, the lifetime net energy production (14.299-13.8=0.499 MWh difference) and lifetime net energy consumption (16.043-15.7=0.343 MWh difference) are not a close to what is reported by Enlighten.
To be clear, this is not a complaint, merely an initial observation on the difference between what the Envoy claims and what Enlighten claims for lifetime. My assumption is that perhaps the Envoy is the source of truth and Enlighten may not be as accurate due to the lower data granularity being sent to enlighten in 15 minute reporting intervals which could cause some import/export data to be unaccounted for. This is the most logical explanation for the lifetime figures variance.

Another observation is that calling them net consumption and net production could be confusing and misleading, as the word consumption is usually related to house consumption (grid import + solar consumption) and the word production is usually related to solar panel production only.

Please allow me some time to test and update my guide accordingly.

Hi @del13r, thanks for your testing, appreciate it.

The difference between these numbers and Enlighten is not unexpected, never seen it match 100%. Until we know what it is they calculate and send it’s second guessing.

Fwiw, json[1].activePower is also available as Current net power consumption.

Your remark on the names is to the point. It’s another story, on hindsight grid import and expert would have been clearer. All I can say is that anything with net in the name comes from the net-consumption/Load with solar production ct.

Any idea when that would be? I do like a good clicky clicky :grin: