Enphase Envoy - Current state of things

Can someone here confirm which HACS repo is more up to date and maintained?

Or is it a case that @jrutski you intend to do a PR to @briancmpbll at a later time?

Thank you both

2 Likes

@AuroreVgn What credentials did you use, and did you use the http://envoy.local? I can login to the local http://envoy.local and need to submit the token.

I was able to get both my Envoy-R and IQ Envoy (combiner 4) working with Brian Campbell’s integration at GitHub - briancmpbll/home_assistant_custom_envoy .

However, the total daily production in HomeAssistant from the IQ Envoy does not add up to the total in Enlighten. It’s sometimes off by as much as 1 kWh a day. I investigated this and saw that the IQ Envoy current production sensor went offline, sometimes for long periods .

For example, this morning, it happened for about 15 minutes.

All the other sensors for that IQ Envoy also go down during this time.

I know the IQ Envoy device itself did not go down and was pingable on my network, due to running Smokeping on that device. Here is the data for the time period .

So, it looks like there are still issues with the integration getting the data from the IQ Envoy.

The same problem does not happen with the data from my Envoy-R (LCD). The production data for that device is an exact match with what’s displayed on its HTML page, which is rounded to 1 decimal, but still not an exact match with Enlighten (rounded to 2 decimals).

I am having the same problem you are having when trying to login to my IQ Envoy via HTTP. I believe I wasn’t having that problem before installing the latest briancmpbll integration .

All,
I have tried both custom integrations ( @jrutski and @briancmpbll versions).
I am using a Envoy-S-Metered-EM with SW version D7.088
They integrate, I see my entities etc. I am not using the consumption parts, nor the battery parts. I ticked the “Use Enlighten” box.

For both integrations I have a lively working sensor.envoy_1222380xxxxx_current_power_production

However, for both integrations, the sensor.envoy_1222380xxxxx_lifetime_energy_production keeps showing a value of 0 (zero) and does not update. This is the sensor I most need to plug into my energy dashboard. Any tweaks I need to make ?
Thanks, and keep up the good work to take this integration to full maturity !
John

EDIT with Solution:
I strongly believe that my unit only generates “Production” figures, for each individual IQ7+ and a total. I believe the “Energy” numbers like total life cycle energy production are calculated in the Enlight-cloud, and will therefore never be retrievable from the unit itself.
As a workaround, I used the power of HA:
sensor.envoy_1222380xxxxx_current_power_production is input to a helper (Riemann Sum Integral - Per hour) which will show lifetime production in Wh. This is then input to another helper (Utility Meter in Wh) which is plugged into the energy dashboard as Solar Production source.

2 Likes

I have downloaded the Enphase Envoy (DEV) integration as found in the HACS store on 11/10/2022 (v0.0.6) from gtdiehl.

After restarting HA v 2022.11.2, I entered the IP address 192.XXX.YY.ZZ for envoy as seen on the connected device info for my Google WiFi network. I used envoy for Username, left the Password field empty, and enterred my envoy serial number.
The configuration fails when I check Use Enlighten box, but it succeeds when I leave it unchecked. This is an Enphase IQ Combiner 3 unit. I don’t know how to check its firmware version.

When the configured integration tries to initialize it fails and continues to retry:

image

I’ve tried downloading other HACS integrations from the two commonly used custom repositories
by briancmpbll and jrutski but I also get stuck at the same stage initialization step.

The WifI connection to envoy is rather weak but stable with the speed test for it giving 36.3 Mbps.
Is low speed the problem?
Should I simply try moving one of the WiFi routers closer to the envoy enclosure?

Same thing for me with GitHub - briancmpbll/home_assistant_custom_envoy . It succeeds to configure when I leave the password empty, set the serial number and leave the checkbox unchecked. I’ve Software Version R4.10.35 (6ed292) and the built-in integration doesn’t work.
It stays in “Retrying setup: 0”

Sorry for the very long delay. I use my Enlighten account credantials.

Unfortunately the maintainer for both the core component and the associated python package envoy_reader are not really active addressing these two issues - 1) core component does not work with firmware 7+, and 2) core component and HACS components (@jrutski and @briancmpbll) do not calculate grid import and export values and expose them as sensors.

I’ve made a code change to bring these sensors to the HACS component by @briancmpbll, hopefully that will make it into his copy or if @jrutski can explain the differences with his I could submit the PR there as well.

2 Likes

@gmcmicken I just saw your comments in this thread having posted a question here

and from your comments, I assume that the issue is that the integration does not do the grid import/export values as you point out.

Is it possible to create some kind of ‘entity/device/sensor’ that will calculate the difference, so can be configured into the Energy dashboard?

I have written my own basic Python some years ago to poll my envoy and send the data to Splunk and am my inverter is s/ware rev D5.0.62 (5e57a9) -

What’s my best option?

since it’s sort of looking like the core component is orphaned and moving toward non-functional (github issue 79382 has been open since Sept with no one even assigned), is there any path to formally replacing it with one of the two HACS versions - basically promoting one or the other into Core? Since 3rd party integrations require HAOS and I’m using a container, I don’t have an easy path to try to use either :frowning:
I’m happy to open an issue against Core to try to move this direction if any of the maintainers are interested in pursuing this path, but could use some direction on this.

Certainly you can make a template sensor to do this, and there’s a thread over at the home assistant boards with examples. You can also use HACS and install my dev repository gmcmicken/home_assistant_custom_envoy in the meantime until this is sorted out.

1 Like

It’s a shame we can’t just add this to core it’s not much code to support the new firmware or anything…

I’m not sure what you mean about home assistant OS, exactly. Are you referring to testing etc to add features to core?

The developers @jrutski @briancmpbll who are most active with this component could maybe agree to a new repo with HACS listing and a few of us as maintainers.

What I mean is that HA’s docs say that 3rd-party add-ons aren’t supported by HA Container or Core, only HAOS or Supervised. I don’t even get the option to install HACS in HA Container.
i.e. Installation - Home Assistant (compare installations) or the note here: Home Assistant Add-ons - Home Assistant

Its a pity the official component is still not working with firmware 7.
I have tried the patched versions of the component, but found that I only got updates every 15 minutes or so, and I would prefer a bit more frequent updates.

So what I’ve done now, is use Home Bridge and the Envoy plugin to send data to Home Assistant over MQTT. Perhaps a bit of a roundabout way, but it seems very stable and I get several updates per minute.

Sharing here in the hope that it helps someone running into issues with their Envoy.

2 Likes

Im trying to get the sample request to work, but I dont know the format for my Envoy Serial number…can any one guide me?

The EnPhase site says:

curl --location -g --request GET 'https://api.enphaseenergy.com/api/v4/systems/{system_id}/summary?key=4a6580c04b9e6282058df02d6454e659' \
--header 'Authorization: Bearer unique_access_token'
</pre>
curl --location -g --request GET 'https://api.enphaseenergy.com/api/v4/systems/121732007xxx/summary?key=ab56932686997d0fca5e67e345789540' \
--header 'Authorization: Bearer eytesting123NiJ9.eyJhcHBfdHlwZSI6InN5c3RlbSIsInVzZXJfbmFtZSI6ImNocmlzdG9waGVyX2FuZ3VzQGhvdG1haWwuY29tIiwiZW5sX2NpZCI6IiIsImVubF9wYXNzd29yZF9sYXN0X2NoYW5nZWQiOiIxNTgzMzEzMDU2IiwiYXV0aG9yaXRpZXMiOlsiUk9MRV9VU0VSIl0sImNsaWVudF9pZCI6IjQ3ZjEyNThmZmRjMjA1NzhmZTI5M2Q2MTQ0MGRmZDE1IiwiYXVkIjpbIm9hdXRoMi1yZXNvdXJjZSJdLCJpc19pbnRlcm5hbF9hcHAiOmZhbHNlLCJzY29wZSI6WyJyZWFkIiwid3JpdGUiXSwiYXRpIjoiYjg3NGM1MTctNjI5OC00MDc5LWFmZTUtMmNlNGMyM2IwN2JlIiwiZXhwIjoxNjc0ODA0NDc3LCJlbmxfdWlkIjoiMTk0MTc3MyIsImp0aSI6IjUzN2M3OTc3LWIyM2QtNDc5OC04YTU3LWFmMmMyN2FmN2UzYSJ9.KUaeZ1jGYxzV1_insvDbZWGvpgju_CO1_PkxKYr6a7YS6p_Efxe4fcIDOurGETcYOtVZgwhi1pUrN0RBlbfDTM41qQmdx7O1LP9KuForFiY_fU8OGznilJ8M3MjARdq85n49gp51re0KHYh1uEp-WAsvZDpmdWMEfxacaxSXaXE'
</pre>

The cURL output error is:

{
    "message": "Not Found",
    "details": "System not found for {:id=>\"121732007xxx\"}",
    "code": 404
}

You are attempting to get data from the internet via enphase cloud which is updated every 15 mins at best which is how often the envoy uploads its data to enphase cloud. This enphase cloud service also has a limit of how many queries you can make for free.

This discussion is mostly focussed on getting data directly from the envoy on the local area network every minute or less.

Hi @del13r

Yes I’m trying to access the EnPhase cloud service. Let me explain… I have one HA instance running at my home and that has the main solar system installed. Everything works well with the standard EnPhase integration. However, I have a second EnPhase system installed at another location together with a battery system. I want to get data from that second system to be able to monitor consumption and Production for some basic battery charging automatons (15 minute data is fine), but as it’s not on the same home network I haven’t been able to connect it in to my HA using its gateway number. So I’m trying to access the second system using the cloud API. (I have both EnPhase systems registered under my single EnPhase account such they are both selectable as independent systems in Enlighten).
I came across this thread in my search for how to access the EnPhase cloud service. I managed to get as far as generating the client ID, API and Access token. But I can’t pass the final quiry to get the system data. The command line cURL requires the system identity number “{system_id}” but the EnPhase number doesn’t pass…
I was going for some guidance on the structure of the system id format to enter in to the cURL.

SOLUTION: OK I figured it out… ‘systemid’ is actually the site ID and not the Envoy id.

@del13r

I know that you are good at these things, are you able to guide me to get the data from my REST:

If I run this cURL on the reqbin.com

curl --location -g --request GET 'https://api.enphaseenergy.com/api/v4/systems/3793806/summary?key=TESTING123997d0fca5e67e412438110' \
--header 'Authorization: Bearer eyTESTING123NiJ9.eyJhcHBfdHlwZSI6InN5c3RlbSIsInVzZXJfbmFtZSI6ImNocmlzdG9waGVyX2FuZ3VzQGhvdG1haWwuY29tIiwiZW5sX2NpZCI6IiIsImVubF9wYXNzd29yZF9sYXN0X2NoYW5nZWQiOiIxNTgzMzEzMDU2IiwiYXV0aG9yaXRpZXMiOlsiUk9MRV9VU0VSIl0sImNsaWVudF9pZCI6IjQ3ZjEyNThmZmRjMjA1NzhmZTI5M2Q2MTQ0MGRmZDE1IiwiYXVkIjpbIm9hdXRoMi1yZXNvdXJjZSJdLCJpc19pbnRlcm5hbF9hcHAiOmZhbHNlLCJzY29wZSI6WyJyZWFkIiwid3JpdGUiXSwiYXRpIjoiYjg3NGM1MTctNjI5OC00MDc5LWFmZTUtMmNlNGMyM2IwN2JlIiwiZXhwIjoxNjc0ODA0NDc3LCJlbmxfdWlkIjoiMTk0MTc3MyIsImp0aSI6IjUzN2M3OTc3LWIyM2QtNDc5OC04YTU3LWFmMmMyN2FmN2UzYSJ9.KUaeZ1jGYxzV1_insvDbZWGvpgju_CO1_PkxKYr6a7YS6p_Efxe4fcIDOurGETcYOtVZgwhi1pUrN0RBlbfDTM41qQmdx7O1LP9KuForFiY_fU8OGznilJ8M3MjARdq85n49gp51re0KHYh1uEp-WAsvZDpmdWMEfxacaxSXaXE'
</pre>

The output from EnPhase is:

{
    "system_id": 3793702,
    "current_power": 0,
    "energy_lifetime": 943,
    "energy_today": 8,
    "last_interval_end_at": 1672206300,
    "last_report_at": 1672207041,
    "modules": 0,
    "operational_at": 1671720300,
    "size_w": 0,
    "source": "meter",
    "status": "normal",
    "summary_date": "2022-12-28"
}

here is my REST and the sensors:

rest:
  - resource: https://api.enphaseenergy.com/api/v4/systems/3793806/summary?key=TESTING123997d0fca5e67e412438110
    method: POST
    scan_interval: 900 #EnPhase only get data every 15 min.
    headers:
      Authorization: Bearer eyTESTING123NiJ9.eyJhcHBfdHlwZSI6InN5c3RlbSIsInVzZXJfbmFtZSI6ImNocmlzdG9waGVyX2FuZ3VzQGhvdG1haWwuY29tIiwiZW5sX2NpZCI6IiIsImVubF9wYXNzd29yZF9sYXN0X2NoYW5nZWQiOiIxNTgzMzEzMDU2IiwiYXV0aG9yaXRpZXMiOlsiUk9MRV9VU0VSIl0sImNsaWVudF9pZCI6IjQ3ZjEyNThmZmRjMjA1NzhmZTI5M2Q2MTQ0MGRmZDE1IiwiYXVkIjpbIm9hdXRoMi1yZXNvdXJjZSJdLCJpc19pbnRlcm5hbF9hcHAiOmZhbHNlLCJzY29wZSI6WyJyZWFkIiwid3JpdGUiXSwiYXRpIjoiYjg3NGM1MTctNjI5OC00MDc5LWFmZTUtMmNlNGMyM2IwN2JlIiwiZXhwIjoxNjc0ODA0NDc3LCJlbmxfdWlkIjoiMTk0MTc3MyIsImp0aSI6IjUzN2M3OTc3LWIyM2QtNDc5OC04YTU3LWFmMmMyN2FmN2UzYSJ9.KUaeZ1jGYxzV1_insvDbZWGvpgju_CO1_PkxKYr6a7YS6p_Efxe4fcIDOurGETcYOtVZgwhi1pUrN0RBlbfDTM41qQmdx7O1LP9KuForFiY_fU8OGznilJ8M3MjARdq85n49gp51re0KHYh1uEp-WAsvZDpmdWMEfxacaxSXaXE'
      Content-Type: application/json

Sensors:

    sensor:
    - name : "Ustou EnPhase system_id"
      value_template: >
        {%- for dict_item in value_json.dataList -%}
            {%- if dict_item.key == "system_id"-%} 
              {{dict_item.value}} 
            {%- endif -%}
        {%- endfor -%}
      
    - name : "Ustou EnPhase Current Consumption Wh"
      value_template: >
        {%- for dict_item in value_json.dataList -%}
          {%- if dict_item.key == "current_power"-%} 
            {{dict_item.value}} 
          {%- endif -%}
        {%- endfor -%}
      unit_of_measurement: 'Wh'
      device_class: energy
      
    - name : "Ustou EnPhase Energy Produced lifetime Wh"
      value_template: >
        {%- for dict_item in value_json.dataList -%}
          {%- if dict_item.key == "energy_lifetime"-%} 
            {{dict_item.value}} 
          {%- endif -%}
        {%- endfor -%}
      unit_of_measurement: 'Wh'
      device_class: energy
    
    - name : "Ustou EnPhase Solar Production Today Wh"
      value_template: >
        {%- for dict_item in value_json.dataList -%}
          {%- if dict_item.key == "energy_today"-%} 
            {{dict_item.value}} 
          {%- endif -%}
        {%- endfor -%}
      unit_of_measurement: 'kWh'
      device_class: energy
      
    - name : "Ustou EnPhase Number of Inverters"
      value_template: >
        {%- for dict_item in value_json.dataList -%}
          {%- if dict_item.key == "modules"-%} 
            {{dict_item.value}} 
          {%- endif -%}
        {%- endfor -%}

    
    - name : "Ustou EnPhase Total Array Size (W)"
      value_template: >
        {%- for dict_item in value_json.dataList -%}
          {%- if dict_item.key == "size_w"-%} 
            {{dict_item.value}} 
          {%- endif -%}
        {%- endfor -%}


    - name : "Ustou EnPhase System Status"
      value_template: >
        {%- for dict_item in value_json.dataList -%}
          {%- if dict_item.key == "status"-%} 
            {{dict_item.value}} 
          {%- endif -%}
        {%- endfor -%}

After a HA Restart I see the new sensors but there is no data:

My EnvoyS with Firmware R4.10.35 using the integration from GitHub - briancmpbll/home_assistant_custom_envoy succeeds installing but ends up in the Installing/Retrying loop and is getting the error below.
I’ve used the envoy IP, username: envoy, blank password and the envoy serial number.


Logger: custom_components.enphase_envoy
Source: custom_components/enphase_envoy/envoy_reader.py:730
Integration: Enphase Envoy (DEV)
First occurred: 1:20:33 PM (16 occurrences)
Last logged: 1:37:37 PM

Unexpected error fetching envoy Envoy [SERIAL] data: 0
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 225, in _async_refresh
    self.data = await self._async_update_data()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 181, in _async_update_data
    return await self.update_method()
  File "/config/custom_components/enphase_envoy/__init__.py", line 62, in async_update_data
    battery_data = await envoy_reader.battery_storage()
  File "/config/custom_components/enphase_envoy/envoy_reader.py", line 730, in battery_storage
    if len(ensemble_json) > 0 and "devices" in ensemble_json[0].keys():
KeyError: 0