Enphase Envoy with Energy Dashboard

hungarianhc · November 3, 2023, 8:18pm

Hey all.

Is this information from the original post still current? It’s a couple years old.

Context: I got the Enphase integration working with Home Assistant, and it’s great, BUT I don’t think all the correct information I need is in the integration. I have Enphase batteries + solar.

For example, there is a sensor called

sensor.envoy_12210203XXXX_current_power_consumption

It looks like this isn’t actual power consumption, but consumption of my home PLUS the batteries filling up when solar is producing. Should I just create a helper that is the sum of all my battery inverters and subtract that from the sensor above?

Or… Should I just go to the first post and start doing everything in there? I’m just a bit nervous because it’s a couple years old, well before Enphase made their API changes w/ the token, etc.

Thanks!

del13r · November 5, 2023, 10:00pm

Hi @hungarianhc, I do not have batteries myself but if I was to make an assumption, it would be that due to where the CT clamp is commonly installed, it is possible that when your battery is filling up, it is counted as house consumption due to the battery being no different to any other appliance in your house that consumes electricity, with the obvious exception that batteries can also supply electricity like your solar panels do. I would expect that the battery consuming the excess solar would be counted as part of the load/consumption.

For example, if I had a battery that needed topping up, I would expect the yellow highlighted part of the image below to go to the battery and show as orange/consumption rather than be exported shown as grey/export.

I’m guessing that you want to seperate the battery consumption from the rest of your house consumption?

hungarianhc · November 6, 2023, 6:29pm

@del13r , thank you for that. The only reason I think you may not be correct is that everything shows perfectly in the Enphase app, and that reads from the same CT clamps. I can see home consumption versus battery versus export, etc. It has been accurate and matched what my utility bill shows for a couple years. It’s really just the sensors themselves in Home Assistant that just don’t seem to be exact matches, with the exception of the solar production. That’s an exact match.

For example, at this very moment, it’s a bit cloudy outside, and my solar is only producing 2.9kW, and that’s accurately reflected in the Enphase app as well as in Home Assistant. Then when I look at my consumption in the Enphase app, I see that 1.2kW is going to my house, and 1.7kW is charging my batteries. However, sensor.envoy_1221020XXXXX_current_power_consumption in Home Assistant is showing the sum of house + battery.

I COULD manually create a helper that takes home consumption and subtracts battery inverter numbers, but that will get complicated when I’m running off battery, and not solar, in the evenings.

del13r · November 6, 2023, 7:46pm

I assume that the enphase app might be showing information from multiple enphase devices that upload data to enphase in the cloud.
My understanding is that the enphase envoy only looks after solar and there might be another box that looks after your battery. You have not specifically said exactly which battery you have so I have to assume it might be the Enphase IQ gateway located inside the IQ system controller 3 INT?

Here is what that box does.

As the name of the sensor has the word envoy in it, could it be that it is only showing what the solar focussed envoy sees?

To confirm this, you can login to https://envoy.local and see what information is available there as that is pretty much the same place that the integration gets its information from.

hungarianhc · November 6, 2023, 10:16pm

I have Enphase batteries + Enphase solar. They all go into the same controller. I just logged into envoy.local (thanks for telling me about this!), and it’s showing the same information as I’m seeing on the Enphase app. Screenshot 2023-11-06 at 2.15.00 PM

There’s an exporting number it shows. There’s no “exporting” entity I can see in Home Assistant, so I wonder where it gets that, as an example. Do I need to calculate it manually?

del13r · November 6, 2023, 10:53pm

Hi @hungarianhc,

I inspected the https://envoy.local/home using browser tools and found the json data source is [“consumption”][0][wNow] in the json tree at https://envoy.local/production.json?details=1

I then look at the code for the Native Enphase Integration at
https://github.com/home-assistant/core/blob/9c0bfc1b582e965ce236e2a02274a091f72e0be6/homeassistant/components/enphase_envoy/sensor.py

and it references pyenphase

from pyenphase import (
    EnvoyEncharge,
    EnvoyEnchargeAggregate,
    EnvoyEnchargePower,
    EnvoyEnpower,
    EnvoyInverter,
    EnvoySystemConsumption,
    EnvoySystemProduction,
)

Next I go to
https://github.com/pyenphase/pyenphase/blob/943204d96f268f3c4ec525612daae98aff35109d/src/pyenphase/models/system_consumption.py
and see the following references

# Data Source: URL_PRODUCTION

        """Initialize from the production API."""

      consumption = data["consumption"][0]

watts_now=int(round(consumption["wNow"]

So then I go to
https://github.com/pyenphase/pyenphase/blob/943204d96f268f3c4ec525612daae98aff35109d/src/pyenphase/const.py
which says

URL_PRODUCTION_JSON = "/production.json"
URL_PRODUCTION = "/production"

For me, this confirms that https://envoy.local/production.json is being used as a data source by the native enphase integration for sensors like house consumption and solar production.

I might be completely wrong as I am not that familiar with python programming, but I think I am on the right track at least.

angusc · November 7, 2023, 9:37am

I have a Tesla battery with Enphase micro inverters. This is my energy map. Right now the battery is at 50% and charging to 100%. But the system overall is in balance. No export to the grid as all power from solar is being used to run the house load and also charging the power wall battery.

Durring the night time I had minimal grid import to maintain the battery at 50% SOC and that was based on the Solcast forecast for my location today

Solcast:

del13r · November 7, 2023, 6:37pm

Hi @angusc,
Thanks for confirming that Telsa AC coupled batteries consume excess solar and this contributes to house load.

On another note, i’ve moved away from the
https://github.com/reptilex/tesla-style-solar-power-card
and am now using
https://github.com/flixlix/power-flow-card-plus

gregzila · November 8, 2023, 4:20pm

Yes ! it’s working

del13r · November 13, 2023, 4:34am

Hi @hungarianhc,
The native Enphase integration still only provides 4 x consumption and 4 x production sensors named:
watt_hours_lifetime
watt_hours_last_7_days
watt_hours_today
watts_now

WattHours being Energy over time (like an odometer) and Watts being instantaneous Power (like a multi-meter).

In order to get the equivalent import and export sensors, you will still need to follow the now 2 years old instructions to get home assistant to calculate import and export by calculating the difference between consumption and production, and then also use the Riemann sum integral integration to record and convert Power/Watts to Energy/WattHours.

The only thing that has changed over those 2 years was the enforcement of the default values. For example, there may be times where I might have used int() or float() in my code and now have to use int(0) or float(0). That’s all I remember being different.

Another observation I noticed was that the screenshot that @angusc posted, his dashboard uses https://github.com/reptilex/tesla-style-solar-power-card which appears to subtract the 1.7 kW (battery charging) and 1.4 kW (car charging) from the 3.5 kW (solar production) leaving only 0.4 kW (house consumption), which appears to do exactly what you are asking for.

Meanwhile, his consumption graph above specifically reports consumption at 3.5 kW shown in orange which indicates both the battery, car and house load are all counted as consumption.

As I don’t have batteries, I am not sure if the native integration gives you an entity for the sum of your battery inverters. That may be the only thing you need to do if there is no native entity to represent your batteries.

The point I am trying to make here is, you may not need to make helper and subtract that from the consumption if there are lovelace cards that appear to be able to do that for you.

All you really need to do is have an entity for house consumption and another entity for the battery charging and reference them in of the HACS lovelace card addons below.

@angusc is using
https://github.com/reptilex/tesla-style-solar-power-card

whereas I am using
https://github.com/flixlix/power-flow-card-plus

mvana · December 4, 2023, 1:19am

Please bear with me: I am probably missing something fundamental:
I do not see any configuration options. I did roam through the main HASS .yaml file nd didn’t find anything relevant and do not see there a specific Enphase integration yaml(?) would be located.

Envoy let’s one install the consumption CT’s either on the

“utility side” (measuring the actual difference between the house consumption and solar production -or-
“load side” (measuring only the house consumption while the solar production has an independent path)

The design of the specific panel may - as it is in my case - preclude installation as per b) but Envoy get’s configured, does the math and all is fine.

Until the point when I start using the HASS integration that does the arithmetic in a way that assumes the load consumed both what Envoy reported as consumed by the load and wrongly adds the solar production to it.
In my actual example below:
Consumed = Produced + Imported
8.8 = 6.5 + 2.3 [kWh] (as correctly reported in Enphase app)

What the HASS integration shows is different (wrong):
15 = 8.6 + 6.4 [kWh] (taking the consumed power reported by Envoy as if it was all coming from the grid).

How do I fix the configuration? Thanks!

del13r · December 4, 2023, 2:35am

Hi @mvana, Which specific HASS integration are you referring to?
There are many to choose from:

briancmpbll/home_assistant_custom_envoy
posixx/home_assistant_custom_envoy (archived on Jul 14, 2023)
jrutski/home_assistant_envoy_d7_fw
DanBeard/enphase_envoy
jesserizzo/envoy_reader
vincentwolsink/home_assistant_enphase_envoy_installer

Also, please advise what type of CT clamp setup you have.

Here is mine for example.

Mine is net-consumption = Load with solar production.

There are subtle differences behind the scenes about how this works

net-consumption = Load with solar production example:

Enphase Envoy - D7 firmware with JWT - A Different Approach

Source: /ivp/meters/readings Description W kW

value_json[0].activePower Power Production 687.208 0.69

value_json[1].activePower Power Net 892.938 0.89

template sensor calculation Power Consumption (Calculated) (production+net) 1580.146 1.58

template sensor calculation Power Export (Calculated) (production-consumption) 0.000 0.00

template sensor calculation Power Import (Calculated) (consumption-production) 892.938 0.89

total-consumption = Load only example:

Enphase Envoy - D7 firmware with JWT - A Different Approach

Source: /ivp/meters/readings Description W kW

value_json[0].activePower Power Production 2000 2.0

value_json[1].activePower Power Consumption 1337.395 1.33

template sensor calculation Power Net (Calculated) (consumption-production) -662.605 -0.66

template sensor calculation Power Export (Calculated) (production-consumption) 662.605 0.66

template sensor calculation Power Import (Calculated) (consumption-production) 0 0

Regardless of your CT clamp location and setting, the integrations that I have tried to understand the code of, all appear to get their data from:
https://envoy.local/production.json
which indicates to me that the Envoy does the required calculations every time that production.json file is requested and also provides a universal total-consumption figure regardless of your ct clamp setting.
For that reason, I can see why this data source is attractive for the developers of Enphase Integrations as it is 1 less variable to worry about.

In the above example, my house is consuming 1604 W and exporting 4482 W to the grid which is indicated by the - in front of the net-consumption number.

mvana · December 4, 2023, 4:00am

Hi @del13r Thank you for the questions and pointers.
Regarding “what integration I have”: I didn’t install any specific integration so I assume this is a native HASS instance. The “documentation” link in the ‘Settings’ page sends me to

so that may be the answer(?)

The configuration seems to be almost identical to what you have:

[
    {
        "eid": 704643328,
        "state": "enabled",
        "measurementType": "production",
        "phaseMode": "split",
        "phaseCount": 2,
        "meteringStatus": "normal",
        "statusFlags": []
    },
    {
        "eid": 704643584,
        "state": "enabled",
        "measurementType": "net-consumption",
        "phaseMode": "split",
        "phaseCount": 2,
        "meteringStatus": "normal",
        "statusFlags": []
    }
]

==> net consumption as expected
As far as the production.json is concerned it may not be telling much as it is pitch black in California (after a gloomy day).
But I even had my Python running for about a year collecting data (prior Enphase pushing the Rev.7 that broke things).
Speaking of which: I didn’t figure out how to harvest the token out from HASS. Hass got the token behind the scenes and I never saw it. Now I requested a token manually and not sure if the token got refreshed - meaning the one HASS used got nixed, I git mine and next HASS will pull another one? (Once I will learn where is the configuration kept I can re-use the same token).
The production.json is below - but perhaps I should abandon this integration and jump to one (you are using?) that may be more configurable?

"varhLagToday":2005.339}],
    "consumption":[
        {
            "type":"eim",
            "activeCount":1,
            "measurementType":"total-consumption",
            "readingTime":1701660810,
            "wNow":732.977,
            "whLifetime":6552246.659,
            "varhLeadLifetime":4330031.39,
            "varhLagLifetime":1734763.269,
            "vahLifetime":14992275.247,
            "rmsCurrent":8.679,
            "rmsVoltage":233.239,
            "reactPwr":-458.684,
            "apprntPwr":2024.218,
            "pwrFactor":0.36,
            "whToday":11385.659,
            "whLastSevenDays":85490.659,
            "vahToday":14755.247,
            "varhLeadToday":5863.39,
            "varhLagToday":2015.269
        },{
            "type":"eim",
            "activeCount":1,
            "measurementType":"net-consumption",
            "readingTime":1701660810,
            "wNow":732.977,
            "whLifetime":3474246.404,
            "varhLeadLifetime":4245132.533,
            "varhLagLifetime":57212.93,
            "vahLifetime":14992275.247,
            "rmsCurrent":7.228,
            "rmsVoltage":233.31,
            "reactPwr":-297.737,
            "apprntPwr":838.929,
            "pwrFactor":0.87,
            "whToday":0,
            "whLastSevenDays":0,
            "vahToday":0,
            "varhLeadToday":0,
            "varhLagToday":0
        }
    ],

Thanks!

del13r · December 4, 2023, 8:26am

Hi @mvana, ah yes, you are correct, my apologies for the mix up

I was easily confused by HASS and I wrongly assumed you meant HACS.
HASS = Home Assistant (Native Integrations)
HACS = Home Assistant Community Store (3rd party repositories)

In answer to your question, the code for the native integration is found here.
homeassistant/components/enphase_envoy

I’m quite novice on python code, so unfortunately I can’t help you find where the token is stored.

At the moment, I am using my own solution via the rest sensor, originally because the native integration didn’t support d7 at the time, but now mainly so I can have efficient 10 second updates instead of 60 second updates. The downside is that it is slightly more complicated to setup than the native implementation and I have to manually refresh my token once a year.
Now that the native implementation supports token authentication, I recommend my solution less than the native one for general use.
I still also use the native Enphase integration for recording sensors like “today_s_energy_production” and “today_s_energy_consumption” so I can verify my figures against what the Envoy reports and also check in on the functionality of the native implementation.

del13r · December 4, 2023, 8:35am

One thing that I notice is that your Grid doesnt have an export sensor.
Is this on purpose?

Here is my config for comparison at https://homeassistant.local/config/energy

Mine is a bit more complicated as I have different costs at different times of the day but I recommend having at least an import (grid consumption) and export (return to grid) sensor as well as a production sensor.

del13r · December 4, 2023, 7:24pm

I found where it might be stored in the code here

github.com

home-assistant/core/blob/516966db332a843f257f727453d677efc14f6b27/homeassistant/components/enphase_envoy/coordinator.py#L135


      
              if not isinstance(envoy.auth, EnvoyTokenAuth):
                  return
              # update token in config entry so we can
              # startup without hitting the Cloud API
              # as long as the token is valid
              _LOGGER.debug("%s: Updating token in config entry from auth", self.name)
              self.hass.config_entries.async_update_entry(
                  self.entry,
                  data={
                      **self.entry.data,
                      CONF_TOKEN: envoy.auth.token,
                  },
              )
          
          async def _async_update_data(self) -> dict[str, Any]:
              """Fetch all device and sensor data from api."""
              envoy = self.envoy
              for tries in range(2):
                  try:
                      if not self._setup_complete:
                          await self._async_setup_and_authenticate()

mvana · December 5, 2023, 5:08am

The underlying issue I am fighting with is that I do not see the
https://homeassistant.local/config/energy you have shown.

What I have related in the '“config” is http://homeassistant.local:8123/config/entities
and I can see Envoy in integrations or devices
but I didn’t find any configurable properties.
Either I do not know where to look or it isn’t there(?)
May be the time to delete this (native?) integration and install the custom one.
Thanks!

mvana · December 5, 2023, 5:10am

Thanks for the Python pointer. Will look how to dump the key into the log file.

del13r · December 5, 2023, 8:13pm

Have you tried
http://homeassistant.local:8123/config/energy

Is there anything there that looks like my screenshot?

I recommend you stick with the native integration, because none of the enphase integrations will automatically provide you with import and export sensors because the envoy doesn’t supply import and export readings. This is a gap that home assistant needs to fill using calculations.

The only exception to this statement is
my own solution via the rest sensor which is not an integration, but simply a rest sensor that targets a different source of information than what the enphase integrations use.

For every enphase integration, native or hacs, the following needs to be done.

What you have to do is go to the first post at top of this article and follow the instructions using:

Step1. Template sensors to calculate power import and power export.

del13r:

template:
  - sensor:
        name: Grid Import Power
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state: >
            {{ [0, states('sensor.envoy_SERIALNUMBER_current_power_consumption') | int(0) - states('sensor.envoy_SERIALNUMBER_current_power_production') | int(0) ] | max }}
  - sensor:
        name: Grid Export Power
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state: >
            {{ [0, states('sensor.envoy_SERIALNUMBER_current_power_production') | int(0) - states('sensor.envoy_SERIALNUMBER_current_power_consumption') | int(0) ] | max }}

Step2. The integration sensor to convert instant power to energy over time.

del13r:

sensor:
  - platform: integration
    name: Grid Import Energy
    source: sensor.grid_import_power
    unit_prefix: k
    unit_time: h
    method: left
    
  - platform: integration
    name: Grid Export Energy
    source: sensor.grid_export_power
    unit_prefix: k
    unit_time: h
    method: left

Once you do those 2 steps, you should be able to choose the new energy sensors in the energy dashboard here

http://homeassistant.local:8123/config/energy

Keith3 · December 6, 2023, 9:44pm

Hi @del13r , thank your for this awesome guide. Quick Question, I’m on latest everything. When I check my unit of measurement for _current_power_consumption and production, it is in kW by default.

Do I need to then change the unit_of_measurement in your template sensor to “kW”.?
If so, then do I also need to remove the unit_prefix of “k” in sensor platform?

Please if you can help / update?

Source: /ivp/meters/readings	Description	W	kW
value_json[0].activePower	Power Production	687.208	0.69
value_json[1].activePower	Power Net	892.938	0.89
template sensor calculation	Power Consumption (Calculated) (production+net)	1580.146	1.58
template sensor calculation	Power Export (Calculated) (production-consumption)	0.000	0.00
template sensor calculation	Power Import (Calculated) (consumption-production)	892.938	0.89

Source: /ivp/meters/readings	Description	W	kW
value_json[0].activePower	Power Production	2000	2.0
value_json[1].activePower	Power Consumption	1337.395	1.33
template sensor calculation	Power Net (Calculated) (consumption-production)	-662.605	-0.66
template sensor calculation	Power Export (Calculated) (production-consumption)	662.605	0.66
template sensor calculation	Power Import (Calculated) (consumption-production)	0	0