Enphase Envoy with Energy Dashboard

Ga1axy · June 11, 2023, 4:24pm

New Enphase system was just connected and after having no luck with the built-in HA integration I read through this whole thread. Thank you for documenting all of this!

I installed the posixx integration, checked the box for Enlighten, and bam it seemed to be working! I was particularly interested in this comment:

This was especially interesting to me because my Enphase app is only showing production, the local page shows metering is not enabled, and my installer said I needed a CT clamp when I asked about it. However in Home Assistant I was able to see these entities with values (also all of the individual inverters show up w/ values):

This got me excited, and I was able to follow your guides to get Grid Import, Grid Export, etc sensors working. Values show 0 for the “Lifetime” and “Today” production/consumption sensors, but I figured since I have current_power_production I could just use this to convert to kWh with:

  - platform: integration
    name: Solar Production Energy
    source: sensor.envoy_SERIAL_current_power_production
    unit_prefix: k
    unit_time: h

All of this gives me an energy dashboard that seemed to work at first, but I quickly realized something is very off. The consumption figure seems to roughly follow the production (or vice versa). They aren’t exactly the same, and it flips between import and export, but there’s no way my actual power use is following the solar curve this closely.

Pretty sure the current_power_consumption sensor is showing something other than my actual home consumption, I think the production sensor is semi accurate, but I can’t really tell.

I assume I just have to get the CT clamp installed and get metering enabled. When I do, do I check the “3-Phase” box when I set this up again?

Also are there any theories as to what my “current_power_consumption” sensor is showing? Because it definitely isn’t my homes power use.

Joster1467 · June 12, 2023, 8:56am

When did you install the Posixx integration?

Worked like a charm over here, but since last week it stopped working. I receive a communicating with API error: Config entry ‘Envoy 122245084166’ for enphase_envoy integration not ready yet: Error communicating with API: All connection attempts failed; Retrying in background

After I put on my debug logging I can see the following warnings:

2023-06-10 13:25:45.458 WARNING (SyncWorker_3) [homeassistant.loader] We found a custom integration enphase_envoy which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 
2023-06-10 13:25:45.459 WARNING (SyncWorker_3) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 
2023-06-10 13:26:11.890 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Envoy 122245084166' for enphase_envoy integration not ready yet: Error communicating with API: All connection attempts failed; Retrying in background 
2023-06-10 13:26:47.385 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: upnp 
2023-06-10 13:27:27.843 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Compal Broadband Networks, Inc CH7465LG' for upnp integration not ready yet: Error connecting to device at location: http://[2001:1c03:5700:0:587:1221:c501:ae5]:5000/rootDesc.xml, err: ("UpnpConnectionTimeoutError('TimeoutError()', None)", None); Retrying in background 
2023-06-10 13:30:11.250 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Envoy 122245084166' for enphase_envoy integration not ready yet: Error communicating with API: All connection attempts failed; Retrying in background 
2023-06-10 13:50:32.178 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Envoy 122245084166' for enphase_envoy integration not ready yet: Error communicating with API: All connection attempts failed; Retrying in background 
2023-06-10 13:52:27.690 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Checking Token value: 
2023-06-10 13:52:27.691 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Found empty token: 
2023-06-10 13:52:31.426 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Commissioned Token valid for 364.9999950603589 days

Seems like a token is needed while this first wasn’t? On the other hand, it still works on your install is it?

zSprawl · June 12, 2023, 4:26pm

I kept having this problem and it would require a reboot. I ended up turning off the “automatically add new entities” setting in the config options of integration so it would stop trying to auto-detect new settings like the IP address.

Notice that ipv6 url? Mine seemed to keep changing from ipv4 to ipv6 too. If you look on .storage, the stupid ip changes from the local to this due to zeroconf. Unfortunately I don’t have logs to show you but something to try.

del13r · July 10, 2023, 5:05am

Hi everyone,

I’ve been monitoring the home assistant and enphase forums, and it would seem a lot of people got their firmware automatically upgraded recently, myself included.

I looked at the code on a few integrations and noticed they all pulled data from https://envoy.local/production.json

From my testing of this URL, it can take anywhere from 400ms to 2000ms for the envoy to generate this json file. I suspect this may be due to the envoy having to perform some calculations every time this url is requested.

This caused me to look for alternatives which led me to https://envoy.local/ivp/meters/readings which from my testing takes on average 50ms to 70ms to respond and offers roughly similar data.

This is where I noticed another thread and decided to do some of my own tweaks.

Here is what I did to get my power sensors back on track.

I removed any existing enphase related integrations (native and/or HACS)

Added my 12 month token to /config/secrets.yaml

enphase_api: "Bearer 408CHARACTERJWT"

Then, back in /config/configuration.yaml, I added these 2 x CT clamp sensors

rest:
  - headers:
      Authorization: !secret enphase_api
    verify_ssl: False
    scan_interval: 15
    resource: https://envoy.local/ivp/meters/readings    
    sensor:
      - name: "Power Production"
        value_template: >
            {% set value = value_json[0].activePower | int(0) %}
            {% if value  <= 5 -%}
                0
            {% elif is_state('sun.sun','below_horizon') %}
                0
            {%- else -%}
                {{ value }}
            {%- endif %}
        device_class: power
        unit_of_measurement: W
        state_class: measurement
        icon: mdi:solar-panel
      - name: "Power Net"
        value_template: "{{ value_json[1].activePower | int(0) }}"
        state_class: measurement
        device_class: power
        unit_of_measurement: W
        icon: mdi:transmission-tower

Then once I was happy with those 2 x CT clamp sensors updating every 15 seconds, I was able to use template sensors to do some calculations with them to get the other sensors I needed.

template:
  - sensor:
        name: Power Consumption
        state_class: measurement
        icon: mdi:home-lightning-bolt
        unit_of_measurement: W
        device_class: power
        state : >
          {{ states('sensor.power_production') | int(0) + states('sensor.power_net') | int(0) }}
          
  - sensor:
        name: Power Export
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state : >
          {{ [0, states('sensor.power_production') | int(0) - states('sensor.power_consumption') | int(0) ] | max }}
          
  - sensor:
        name: Power Import
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state : >
          {{ [0, states('sensor.power_consumption') | int(0) - states('sensor.power_production') | int(0) ] | max }}

Now I have 5 sensors.
The only information currently being extracted from the envoy is the 2 ct clamp sensors power_production and power_net.
The other 3 sensors are template sensors where home assistant performs calculations when the 2 x CT clamp sensors get updated every X seconds.

Now I am able to have a dashboard like this that updates every 15 seconds.
Feel free to try smaller number refresh intervals for yourself if you want.

Or graphs like this

vk2him · July 10, 2023, 5:21am

@del13r - nicely done. How did you get 12 month token, is that documented anywhere? Cheers

del13r · July 10, 2023, 5:28am

Here are the commands I manually used:

command:
curl -s -d 'user[email][email protected]&user[password]=MYPASSWORD' \
https://enlighten.enphaseenergy.com/login/login.json \
| jq -r .session_id

Result:
32CHARACTERSESSIONID

then I used my envoy serial number and that session ID in the next command to get my token.

curl -s 'https://enlighten.enphaseenergy.com/entrez-auth-token?serial_num=ENVOYSERIALNUMBER' \
-H 'cookie: _enlighten_4_session=32CHARACTERSESSIONID' \
| jq -r '.token'

Result:
408CHARACTERJWT

however this script is probably easier

Enphase local API with firmware 7.x - my setup

import json
import requests

# REPLACE ITEMS BELOW 
user='[email protected]'
password='password'
envoy_serial='your_envoy_serial_number'
# DO NOT CHANGE ANYTHING BELOW

data = {'user[email]': user, 'user[password]': password}
response = requests.post('https://enlighten.enphaseenergy.com/login/login.json?',data=data)
response_data = json.loads(response.text)
data = {'session_id': response_data['session_id'], 'serial_num': envoy_serial, 'username':user}
response = requests.post('https://entrez.enphaseenergy.com/tokens', json=data)
token_raw = response.text
print(token_raw)

del13r · July 10, 2023, 6:19am

As for anyone unsure of the length of time that the token is valid for, you can paste it into
https://jwt.io/
and it will decode the token and tell you the issue date and expiry date.

Gilles_C · July 10, 2023, 8:57am

I have copy/paste your code but I have an error : Error fetching data: https://envoy.local/ivp/meters/readings failed with All connection attempts failed

The token works with CURL.

del13r · July 10, 2023, 9:13am

A common issue I’ve seen on other peoples issues is not putting “Bearer “ in front of the token in the secrets file.

Also, try using the ipv4 address rather than envoy.local just incase of dns issues

Gilles_C · July 10, 2023, 9:52am

The ipv4 was the solution for me, thanks a lot !

Smartdrive78 · July 13, 2023, 12:39am

I just got my Enphase system installed and came across this thread. When trying to implement it however, the Grid Import Energy always shows Unavailable and the Grid Import Power 0W

This is in my configuration.yaml file

 - sensor:
        name: Grid Import Power
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state: >
            {{ [0, states('sensor.envoy_xxx_current_power_consumption') | int(0) - states('sensor.envoy_xxx_current_power_production') | int(0) ] | max }}
            
  - sensor:
        name: Grid Export Power
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state: >
            {{ [0, states('sensor.envoy_xxx_current_power_production') | int(0) - states('sensor.envoy_xxx_current_power_consumption') | int(0) ] | max }}
            
sensor:
  - platform: integration
    name: Grid Import Energy
    source: sensor.grid_import_power
    unit_prefix: k
    unit_time: h
    method: left
    
  - platform: integration
    name: Grid Export Energy
    source: sensor.grid_export_power
    unit_prefix: k
    unit_time: h
    method: left

When I look in the Developer Tools I can see the following values:

{{ [0, states('sensor.envoy_xxx_current_power_consumption') | int - states('sensor.envoy_xxx_current_power_production') | int ] | max }}
0

{{ [0, states('sensor.envoy_xxx_current_power_production') | int - states('sensor.envoy_xxx_current_power_consumption') | int ] | max }}
2686

{{ states('sensor.envoy_xxx_current_power_consumption') }}
717

{{ states('sensor.envoy_xxx_current_power_production') }}
3411

Any ideas?

I am using Enphase Envoy (DEV) as the integration and the system itself is on D7.0.88 (5580b1)

del13r · July 13, 2023, 12:57am

There are many possible reasons why this might be a problem.

The best start would be to go to /developer-tools/state
and type in this next to the magnifying glass icon under filter entities:
power_

My setup will look different to yours, but regardless of your integration, you should be able to see at least what your production and consumption sensors are reporting at that point in time.

So given the energy sensor uses the power sensor as the source, lets focus on what the power sensor is reporting first.

If it is in the middle of the day, and you are currently exporting power, then the power import sensor will be 0 (as mine also is right now as shown in the screenshot).

How long has this sensor been 0 for?

Using your own data we can see why:

The formula to record grid import breaks down like this:
if
consumption (717 W in your case)
minus
production (3411 W in your case)

=
0 or a negative figure (-2694 W in your case)
then
record 0 W

Smartdrive78 · July 13, 2023, 1:05am

If it is in the middle of the day, and you are currently exporting power, then the power import sensor will be 0 (as mine also is right now as shown in the screenshot)

OMG. I am so stupid - I am currently not importing energy. Of course it is 0. Unbelievable. I was just confused because one of the sensor graphs (Grid Import Energy) says “unavailable” and that meant to me it is not working at all.

Smartdrive78 · July 13, 2023, 2:45am

I still think I must be doing something wrong or not understanding:

This is what the Enphase app tells me:

but this is what Home Assistant shows:

del13r · July 13, 2023, 2:59am

kW = POWER live / right now / instantaneous
kWh = ENERGY over time (usually an hour/day)

If I turned on a device that consistently uses 1kW of power and left it running for an hour, it would use 1kWh of energy.
If I used that same device for only 30 mins, it would use 0.5 kWh of energy over 30 mins.

Smartdrive78 · July 13, 2023, 3:01am

Okay cool. So the HA view is overall time for a specific date range set in the Energy dashboard.

Now I am wondering if there is a solution for HA to show the live state like Enphase does.

del13r · July 13, 2023, 3:03am

Yep, sure can

This is how I do it with the built in

type: horizontal-stack
cards:
  - type: gauge
    entity: sensor.power_consumption
    severity:
      red: 0
  - type: gauge
    entity: sensor.power_production
    severity:
      green: 1

alexeiw123 · July 13, 2023, 3:12am

There’s also a fantastic card available in HACS that is based on the HA energy card, so the power view ties in perfectly with the energy view, with heaps of customisation.

Smartdrive78 · July 13, 2023, 3:14am

Well… thank you so much. Total Google Fu fail on my part.

Let’s see how much work I get done today working from home. Only got the system yesterday. New toy!

del13r · July 13, 2023, 3:23am

For even more POWER visualisation, i can also duplicate the enphase view using
https://github.com/RomRider/apexcharts-card

type: custom:apexcharts-card
graph_span: 24h
stacked: true
header:
  show: true
  title: Total Power
series:
  - entity: sensor.power_production
    type: column
    name: Produced
    color: '#01B4DE'
    group_by:
      func: avg
      duration: 5min
  - entity: sensor.power_consumption
    transform: return x *-1 ;
    type: column
    name: Consumed
    color: '#F37320'
    group_by:
      func: avg
      duration: 5min
  - entity: sensor.power_net
    type: column
    name: Imported/Exported
    color: '#545456'
    group_by:
      func: avg
      duration: 5min

Enphase Envoy with Energy Dashboard

The formula to record grid import breaks down like this: if consumption (717 W in your case) minus production (3411 W in your case)

The formula to record grid import breaks down like this:
if
consumption (717 W in your case)
minus
production (3411 W in your case)