Enphase Envoy - D7 firmware with JWT - A Different Approach

I’ve done a bit of testing but am still getting some issues.

I uninstalled the Enphase integration and removed the HACS repository. Entities were still dropping to 0 due to what I believe to be the rest error I pasted above.

I tried changing the scan interval to 300 thinking that hitting the URL less frequently might help, but was still getting the same results. At the moment I’m sticking to 60.

I also tried pulling data from my inverter API so I could track production from each inverter. I made this scan every 300 seconds, but have a similar issue here with them becoming unavailable sporadically.

Struggling to find any consistency, other than on some scans, the rest request fails, then on others it works. It’s also not my wifi dropping. @Steve61 I think this is different to the issue with the main integration, as it’s not happening at a consistent time.

I’ll keep digging, but let me know if you can think of something else I should try.

Is this in parallel to the rest sensor?
Is this via the native integration/hacs/other?

The reason why I am asking is that we want to eliminate as many possible causes of your drop outs.
Having multiple integrations polling the envoy at the same time is what we are trying to eliminate.
Can you also temporarily try 15 second scan interval?
The reason why is because if other integrations are also contacting the envoy, setting the scan interval to any multiple of 60 seconds (300 / 5 = 60) may cause both polls to occur at the same time at the start of each minute and they both might fail due to the lag that the non rest integrations create by polling production.json
If you set it to 15 seconds, then it’s possible that 3/4 polls per minute might succeed, if my double knockout theory is correct.

You can test this yourself outside of home assistant by using browser tools (f12) in your browser.

Go to https://envoy.local/ivp/meters/readings

It will then show you a screen where you can either:

  • login to enlighten and generate a temporary 12 hour installer token.
  • paste in your existing 12 month owner token that is in secrets.yaml

Once you do either step, you should be able to see the json output of https://envoy.local/ivp/meters/readings.
Then you can press f12 to open dev tools and then press ctrl + R (refresh) to see how long each request takes in milliseconds or seconds or even if it times out using browser tools.

You can also tick preserve log and then it will show how long each refresh request took in a table at the bottom. (203ms average across 11 refreshes)

compare that with https://envoy.local/production.json and you will definitely see a slower average for the older API endpoint. (780ms average across 11 refreshes)

This method is pretty much the same thing that the rest sensor does, instead you are triggering the refresh manually whenever you press ctrl + R to refresh the page much the same as the scan interval setting would.

1 Like

Is this in parallel to the rest sensor?
Is this via the native integration/hacs/other?

@del13r the issue is using the rest sensors. I essentially replicated your setup l, but also for the inverters. I uninstalled the native integration. So shouldn’t be running in parallel.

I’ve changed the scan for the main API to 15 so I’ll see how that goes. But I was actually getting this issue before I had added the inverter requests, so I’m not to sure that’s the reason.

I’ll also have a play around in browser.

1 Like

i would also be interested in your wifi signal strength that the envoy is reporting via https://envoy.local/home.json

As a test, I successfully setup the following sensor which asks for the wifi strength every 5 mins.

rest:
  - headers:
      Authorization: !secret enphase_api
    verify_ssl: False
    scan_interval: 300
    resource: https://envoy.local/home.json
    sensor:
      - name: "envoy wifi strength"
        value_template: "{{value_json.network.interfaces[1].signal_strength }}"

2/5 signal strength = 40% (not very good).

Example of it graphed.

I’m also in this exact same situation, cannot figure out how to properly calculate energy import and export. My template for calculating import and export does not work, it never shows any value for import, even when I’m importing the grid. I’m reaching out to my installer to see if I can get my switched away from Load Only, so we’ll see what they say, but would love to have this working in case it’s not possible.

You need a Riemann integral helper function to integrate the power import/export sensors. https://www.home-assistant.io/integrations/integration/ That is what I do now.

I am working on making hybrid integration, that gets power locally from the envoy using info from this thread, but gets the energy values from the cloud. Might be noobie-proof method. The cloud seems to know everything, but the envoy does not report it locally!

Ok, let me try tackle this issue for you.
I cannot guarantee the accuracy as my setup is not the same as yours, so sorry in advance if my solution is not perfect.

Step 1, How can I see what my system is currently setup as?
Go to https://envoy.local/ivp/meters/
look at the value of [1].measurementType
value: net-consumption = Load with solar production.
value: total-consumption = Load only

image

Step 2, show the difference between load only and net.
This is where things differ.
View the value of [1].activePower at the URL https://envoy.local/ivp/meters/readings

  • When set to total-consumption/(load only), this value will only ever be positive as it represents what the appliances in house are demanding/consuming.
  • When set to net-consumption/(load with solar production), you will either see a negative value to indicate power grid export or a positive value to indicate grid import as this value represents what is occurring at the utility/grid/meter.

Example of a user with total-consumption/(load only)

Step 3, what can we calculate with load only?

Source: /ivp/meters/readings Description W kW
value_json[0].activePower Power Production 2000 2.0
value_json[1].activePower Power Consumption 1337.395 1.33
template sensor calculation Power Net (Calculated) (consumption-production) -662.605 -0.66
template sensor calculation Power Export (Calculated) (production-consumption) 662.605 0.66
template sensor calculation Power Import (Calculated) (consumption-production) 0 0

The problem with load only setting is that the value for [1].actEnergyRcvd is empty at https://envoy.local/ivp/meters/readings.

Example of a user with total-consumption/(load only)

The absense of that value causes the energy calculations to no longer be possible.

Source: /ivp/meters/readings Description Wh MWh
value_json[0].actEnergyDlvd Energy Production 609552.964 0.6
value_json[1].actEnergyDlvd Energy Consumption 447569.605 0.44
template sensor calculation Energy Export (Calculated) not possible
template sensor calculation Energy Net (Calculated) not possible
template sensor calculation Energy Import (Calculated) not possible

Step 4, How do we fill the gap for load only users?
Create the following sensors for Load Only installations.

rest:
  - headers:
      Authorization: !secret enphase_api
    verify_ssl: False
    scan_interval: 15
    resource: https://envoy.local/ivp/meters/readings
    sensor:

      - name: "Power Production"
        value_template: >
            {% set value = value_json[0].activePower | int(0) %}
            {% if value  <= 5 -%}
                0
            {% elif is_state('sun.sun','below_horizon') %}
                0
            {%- else -%}
                {{ value }}
            {%- endif %}
        device_class: power
        unit_of_measurement: W
        state_class: measurement
        icon: mdi:solar-panel

      - name: "Power Consumption"
        value_template: "{{ value_json[1].activePower | int(0) }}"
        state_class: measurement
        device_class: power
        unit_of_measurement: W
        icon: mdi:home-lightning-bolt

      - name: "Energy Production"
        value_template: "{{ (value_json[0].actEnergyDlvd / 1000 | float(0)) | round(2) }}"
        device_class: energy
        unit_of_measurement: kWh
        state_class: total_increasing
        icon: mdi:solar-panel

      - name: "Energy Consumption"
        value_template: "{{ (value_json[1].actEnergyDlvd / 1000 | float(0)) | round(2) }}"
        device_class: energy
        unit_of_measurement: kWh
        state_class: total_increasing
        icon: mdi:home-lightning-bolt

Then i added these template sensors to calculate power export and power import

template:
  - sensor:
        name: Power Export
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state : >
          {{ [0, states('sensor.power_production') | int(0) - states('sensor.power_consumption') | int(0) ] | max }}
          
  - sensor:
        name: Power Import
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state : >
          {{ [0, states('sensor.power_consumption') | int(0) - states('sensor.power_production') | int(0) ] | max }}

  - sensor:
        name: Power Net
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state : >
          {{ states('sensor.power_consumption') | int(0) - states('sensor.power_production') | int(0)  }}

And then finally, add these 2 sensors to convert:
power in W
to
energy in kWh
using https://www.home-assistant.io/integrations/integration/

sensor:
  - platform: integration
    name: Energy Import
    source: sensor.power_import
    unit_prefix: k
    unit_time: h
    method: left
    
  - platform: integration
    name: Energy Export
    source: sensor.power_export
    unit_prefix: k
    unit_time: h
    method: left

I am unable to test this myself, so feel free to give it a go and let me know.

1 Like

@del13r I have a signal strength of 3. It should be fairly solid. I have a unifi setup and the router is on the other side of the same wall of the house.

I’ve also commented out my inverter rest calls. So basically my setup at the moment is the same as yours. I’ve still noticed some dropped calls. But I’ll track it for a while and see how it goes.

1 Like

Process of elimination: Try eliminate DNS issues as a possible cause by using the IPv4 address (192.x.x.x) instead of envoy.local

Ok, so I’ve noticed that even at 15 second intervals. I get about 2 to 3 drops an hour. Not too bad, and definitely an improvement on before. So there’s definitely something to there being an issue if there’s too many concurrent calls. Like when I had the original integration running side by side.

I’ve now swapped over to the IP address and will see how this goes.

1 Like

Also played around in browser and definitely get the quicker responses using the URL you suggested over the production .json.

1 Like

Yep, that’s really great results. Thanks for taking the time to perform these tests.

Process of elimination: Try eliminate DNS issues as a possible cause by using the IPv4 address (192.x.x.x) instead of envoy.local

@del13r ok I think I have some good results to share finally. Since changing to an IP address instead of the envoy.local url, I’ve had no dropped rest calls. So it must have something to do with hitting the envoy.local link.

I’ve been able to turn on the inverter rest calls as well and both calls are working without issues so far over about a 12 hour period.

Thanks so much for helping work it out!

1 Like

Ah that’s awesome news. Go team.

1 Like

Not to add another option to get enphase working, but I took the great work of @del13r and made a custom integration. Been working well for a couple days, but probably still has errors. This integration only works for people setup in “load only” or “total-consumption” mode. I will probably try to make it support people with “Load with Solar Production” in the future.

https://github.com/jdeath/enphase_local

Features: Uses endpoints @del13r found to get power locally from the envoy. It gets energy values from enphase cloud, since my system does not report this locally. Avoids needing Reiman sum integrals or any manual sensor setup. Sensors can directly be used in the energy dashboard. Takes username/password so do not need to figure out your token, but do need your serial and siteid. Feel free to submit PRs to add capability.

2 Likes

I notice your documentation says that it will poll enphase cloud api every 10 mins.

I might be on the wrong track here, but the free developer account only gets 1000 hits per month.

My calculations are about 44 mins per poll to stay under the limit.

Hits
12000.000 365.000 Days/Year
1000.000 30.417 Days/Month
230.769 7.019 Days/Week
32.877 1.000 Day
1.370 1.000 Hour
1.005 44.000 Minutes
0.023 1.000 Minute

This integration does not use the official API… :slight_smile: No developer account needed. I’ll see if it gets limited (but I doubt it), I have not tested it for more than a day!

Ah yes. I just noticed this in your code


self.url = "https://enlighten.enphaseenergy.com/pv/systems/" + self.siteID + "/today"

@del13r
I had an issue using “INT” in the template sensors:

state : >
          {{ [0, states('sensor.power_production') | int(0) - states('sensor.power_consumption') | int(0) ] | max }}

would throw errors in child sensors, but if I changed it to “float” they disappeared. So:

state : >
          {{ [0, states('sensor.power_production') | float(0) - states('sensor.power_consumption') | float(0) ] | max }}

worked without issue.

I don’t know enough about YAML functions and the only thing I could see was in my version of the source - production and consumption sensors, I converted everything to kW by adding “/1000”.
{% set value = (value_json[0].activePower /1000 | int(0))

I’m not sure if this affects accuracy. Thoughts???