Enphase Envoy - Current state of things

[jnathalia (jn)]

Thanks @del13r,

I have followed your clear instructions with the following results:

Then I logged in with the correct Enlighten credentials:

Then going back to integration:

But it is still failing… (although the error message is now simple as far as I can remember from yesterday when tweaking around again…)

Does it have something to do that I’m working with duckdns.org and https, so accessing from outside? However, I tested it by using my https://192.168.1.140:8123 and it doesn’t make a difference. Same error…

[catsmanac (Arie Catsman)]

You can add below to your configuration.yaml in the HA config folder. Then restart HA and try adding the envoy again. Inspect the home-assistant.log file for any indication of issues.

logger:
  default: warning
  logs:
    custom_components.enphase_envoy: debug
    httpx: debug
    httpcore: debug

[jnathalia (jn)]

thanks @catsmanac as well for supporting me.

I found a few lines interesting (“Found empty token:”):

2024-10-01 21:01:35.486 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Checking Token value:  (Only first 10 characters shown)
2024-10-01 21:01:35.486 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Found empty token:
2024-10-01 21:01:35.487 DEBUG (MainThread) [httpx] load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-01 21:01:35.490 DEBUG (MainThread) [httpx] load_verify_locations cafile='/usr/local/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-01 21:01:35.505 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to load_verify_locations with args (<ssl.SSLContext object at 0x7f4cd1be4d50>,) inside the event loop by custom integration 'enphase_envoy' at custom_components/enphase_envoy/envoy_reader.py, line 253: return self._async_client or httpx.AsyncClient(verify=True, (offender: /usr/local/lib/python3.12/site-packages/httpx/_config.py, line 147: context.load_verify_locations(cafile=cafile)), please report it to the author of the 'enphase_envoy' custom integration

a bit further I found this (seems the token is ok)

2024-10-01 21:01:35.546 DEBUG (MainThread) [httpcore.connection] connect_tcp.started host='enlighten.enphaseenergy.com' port=443 local_address=None timeout=30 socket_options=None
2024-10-01 21:01:35.655 DEBUG (MainThread) [httpcore.connection] connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0x7f4cd1f5f890>
2024-10-01 21:01:35.655 DEBUG (MainThread) [httpcore.connection] start_tls.started ssl_context=<ssl.SSLContext object at 0x7f4cd1be4d50> server_hostname='enlighten.enphaseenergy.com' timeout=30
2024-10-01 21:01:35.757 DEBUG (MainThread) [httpcore.connection] start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0x7f4cd26b22d0>
2024-10-01 21:01:35.758 DEBUG (MainThread) [httpcore.http11] send_request_headers.started request=<Request [b'POST']>
2024-10-01 21:01:35.761 DEBUG (MainThread) [httpcore.http11] send_request_headers.complete
2024-10-01 21:01:35.761 DEBUG (MainThread) [httpcore.http11] send_request_body.started request=<Request [b'POST']>
2024-10-01 21:01:35.762 DEBUG (MainThread) [httpcore.http11] send_request_body.complete
2024-10-01 21:01:35.763 DEBUG (MainThread) [httpcore.http11] receive_response_headers.started request=<Request [b'POST']>
2024-10-01 21:01:37.597 DEBUG (MainThread) [httpcore.http11] receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Tue, 01 Oct 2024 19:01:37 GMT'), (b'Content-Type', b'application/json; charset=utf-8'), (b'Transfer-Encoding', b'chunked'), (b'Connection', b'keep-alive'), (b'Cache-Control', b'max-age=0, private, must-revalidate'), (b'Content-Encoding', b'gzip'), (b'Content-Security-Policy', b"font-src 'self' https: data:; img-src 'self' http://www.google-analytics.com https: blob: data:; object-src 'none'; style-src 'self' https: 'unsafe-inline'; frame-ancestors https://*.enphaseenergy.com;"), (b'Etag', b'W/"a1d04fbe80aee798ddc7f3524775eaba"'), (b'Expect-Ct', b'max-age=1728000, enforce'), (b'Set-Cookie', b'locale=en; path=/; expires=Wed, 01 Oct 2025 19:01:35 GMT; secure; HttpOnly; SameSite=Lax'), (b'Set-Cookie', b'enlighten_manager_token_production=eyJhbGciOiJIUzI1NiJ9.eyJkYXRhIjp7InNlc3Npb25faWQiOiIzM2M4NTFkYzdiMzFmNzkwYzBlOTA1NmI3Nzk5ZDM1MCIsImNvbXBhbnlfaWQiOm51bGwsImVtYWlsX2lkIjoiamVyb2VuQG5hdGhhbGlhYmVoZWVyLm5sIiwidXNlcl9pZCI6MzAwMTM5NiwiY2xpZW50X2FwcCI6Iml0azMiLCJmaXJzdF9uYW1lIjoiSmVyb2VuIiwibGFzdF9uYW1lIjoiTmF0aGFsaWEiLCJsb2dpbl91c2VyIjpudWxsLCJpc19kaXN0cmlidXRvciI6ZmFsc2V9LCJleHAiOjE3Mjg0MTQwOTcsInN1YiI6Implcm9lbkBuYXRoYWxpYWJlaGVlci5ubCJ9.KhJChtoRfz14SfInZhLSfIGQJmjEcHSFwaSpxfjrsdE; domain=.enphaseenergy.com; path=/; SameSite=None; secure'), (b'Set-Cookie', b'_enlighten_4_session=33c851dc7b31f790c0e9056b7799d350; path=/; secure; HttpOnly; SameSite=Lax'), (b'Status', b'200 OK'), (b'Strict-Transport-Security', b'max-age=15724800; includeSubDomains'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'X-Powered-By', b'Phusion Passenger(R) 6.0.19'), (b'X-Request-Id', b'005a7ca03287dba60d0627180d9181e4'), (b'X-Runtime', b'1.730849'), (b'X-Xss-Protection', b'1; mode=block')])
2024-10-01 21:01:37.599 INFO (MainThread) [httpx] HTTP Request: POST https://enlighten.enphaseenergy.com/login/login.json "HTTP/1.1 200 OK"
2024-10-01 21:01:37.599 DEBUG (MainThread) [httpcore.http11] receive_response_body.started request=<Request [b'POST']>
2024-10-01 21:01:37.600 DEBUG (MainThread) [httpcore.http11] receive_response_body.complete
2024-10-01 21:01:37.601 DEBUG (MainThread) [httpcore.http11] response_closed.started
2024-10-01 21:01:37.601 DEBUG (MainThread) [httpcore.http11] response_closed.complete
2024-10-01 21:01:37.603 DEBUG (MainThread) [httpcore.connection] connect_tcp.started host='entrez.enphaseenergy.com' port=443 local_address=None timeout=30 socket_options=None
2024-10-01 21:01:37.710 DEBUG (MainThread) [httpcore.connection] connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0x7f4cd8d3e5a0>
2024-10-01 21:01:37.710 DEBUG (MainThread) [httpcore.connection] start_tls.started ssl_context=<ssl.SSLContext object at 0x7f4cd1be4d50> server_hostname='entrez.enphaseenergy.com' timeout=30
2024-10-01 21:01:37.838 DEBUG (MainThread) [httpcore.connection] start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0x7f4cd8d3dbe0>
2024-10-01 21:01:37.839 DEBUG (MainThread) [httpcore.http11] send_request_headers.started request=<Request [b'POST']>
2024-10-01 21:01:37.841 DEBUG (MainThread) [httpcore.http11] send_request_headers.complete
2024-10-01 21:01:37.841 DEBUG (MainThread) [httpcore.http11] send_request_body.started request=<Request [b'POST']>
2024-10-01 21:01:37.843 DEBUG (MainThread) [httpcore.http11] send_request_body.complete
2024-10-01 21:01:37.843 DEBUG (MainThread) [httpcore.http11] receive_response_headers.started request=<Request [b'POST']>
2024-10-01 21:01:38.018 DEBUG (MainThread) [httpcore.http11] receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Tue, 01 Oct 2024 19:01:37 GMT'), (b'Content-Type', b'text/plain;charset=UTF-8'), (b'Content-Length', b'419'), (b'Connection', b'keep-alive'), (b'Strict-Transport-Security', b'max-age=15724800; includeSubDomains'), (b'server', b'hide'), (b'X-Content-Type-Options', b'nosniff')])
2024-10-01 21:01:38.018 INFO (MainThread) [httpx] HTTP Request: POST https://entrez.enphaseenergy.com/tokens "HTTP/1.1 200 OK"
2024-10-01 21:01:38.019 DEBUG (MainThread) [httpcore.http11] receive_response_body.started request=<Request [b'POST']>
2024-10-01 21:01:38.019 DEBUG (MainThread) [httpcore.http11] receive_response_body.complete
2024-10-01 21:01:38.020 DEBUG (MainThread) [httpcore.http11] response_closed.started
2024-10-01 21:01:38.020 DEBUG (MainThread) [httpcore.http11] response_closed.complete
2024-10-01 21:01:38.021 DEBUG (MainThread) [httpcore.connection] close.started
2024-10-01 21:01:38.022 DEBUG (MainThread) [httpcore.connection] close.complete
2024-10-01 21:01:38.023 DEBUG (MainThread) [httpcore.connection] close.started
2024-10-01 21:01:38.024 DEBUG (MainThread) [httpcore.connection] close.complete
2024-10-01 21:01:38.024 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Obtained Token
2024-10-01 21:01:38.025 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Token expires at: 2025-10-01 21:01:37
2024-10-01 21:01:38.032 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP GET Attempt #1 of 2: https://192.168.1.192/auth/check_jwt: use token: True: Header: <Token hidden>  Timeout: 60 Holdoff: 0
2024-10-01 21:01:38.032 DEBUG (MainThread) [httpx] load_ssl_context verify=False cert=None trust_env=True http2=False
2024-10-01 21:01:38.036 DEBUG (MainThread) [httpcore.connection] connect_tcp.started host='192.168.1.192' port=443 local_address=None timeout=60 socket_options=None

Further, it looks like it’s trying to connect again:

2024-10-01 21:02:38.038 WARNING (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP Timeout in fetch_with_retry, waiting 0 sec: 
2024-10-01 21:02:38.039 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP GET Attempt #2 of 2: https://192.168.1.192/auth/check_jwt: use token: True: Header: <Token hidden>  Timeout: 60 Holdoff: 0
2024-10-01 21:02:38.039 DEBUG (MainThread) [httpx] load_ssl_context verify=False cert=None trust_env=True http2=False
2024-10-01 21:02:38.043 DEBUG (MainThread) [httpcore.connection] connect_tcp.started host='192.168.1.192' port=443 local_address=None timeout=60 socket_options=None

Further on:

2024-10-01 21:03:38.046 WARNING (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP Timeout in fetch_with_retry, raising: 
2024-10-01 21:03:38.061 WARNING (MainThread) [custom_components.enphase_envoy.config_flow] Validate input, getdata returned HTTPError: 

So, maybe there is a problem with the token…

When I call the URL https://entrez.enphaseenergy.com/login I can type in the name of my Enphase instance and then the gateway appears. I can create the token.

When I select uncommissioned I can also create a token:

Should I define this token somewhere in my yaml?

[catsmanac (Arie Catsman)]

It is actually retrieving a token successfully. The empty token at the start is as expected as you are adding the envoy. Then it reaches out to enphase and obtains a token which is valid till October 1, 2025.

2024-10-01 21:01:38.024 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Obtained Token
2024-10-01 21:01:38.025 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Token expires at: 2025-10-01 21:01:37

As a next step it is reaching out to your envoy to authenticate with the envoy, but no reply is returned, a timeout occurs. And it tries again to connect to the envoy

2024-10-01 21:01:38.032 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP GET Attempt #1 of 2: https://192.168.1.192/auth/check_jwt: use token: True: Header: <Token hidden>  Timeout: 60 Holdoff: 0

2024-10-01 21:02:38.038 WARNING (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP Timeout in fetch_with_retry, waiting 0 sec: 

2024-10-01 21:02:38.039 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP GET Attempt #2 of 2: https://192.168.1.192/auth/check_jwt: use token: True: Header: <Token hidden>  Timeout: 60 Holdoff: 0

And times-out for the second time

2024-10-01 21:03:38.046 WARNING (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP Timeout in fetch_with_retry, raising: 
2024-10-01 21:03:38.061 WARNING (MainThread) [custom_components.enphase_envoy.config_flow] Validate input, getdata returned HTTPError:

So the issue is that the Envoy is not replying to authentication requests with the token.

I assume you picked some parts from the log. There may be more detailed lines between the first connection attempt to the Envoy and the last failure, These may contain some more information.

What type of Enphase account do you have, installer/diy or home owner?

[jnathalia (jn)]

I’m a homeowner when looking into account details. System maintainer is someone else (the installer) unfortunately.

[jnathalia (jn)]

I have found this article and I’m testing it: Enphase Envoy - D7 firmware with JWT - A Different Approach

Although the sun is not shining at all, the sensors have been created and measuring 0 Watt at the moment… I hope the sun comes to make me more happy with some figures and overcome the autumn

EDIT: this is actually close to this thread… what a mistake! When I go to top, I see the final implementation. Great work…! Thanks @del13r

[jnathalia (jn)]

I think things are working, however I receive 0 kWh produced.

When looking into more detail, I have noticed https://envoy.local shows 0 Watt production:

This must be an Enphase thing instead of a HA integration issue. It might go beyond this topic, but should I make some settings at the Enphase instance? I’m homeowner or do I need to kick out my system maintainer?

Also, I don’t think my installer has installed a CT clamp. But how is it possible I can see the production at https://enlighten.enphaseenergy.com? Is this because of the Envoy and its cloud connection? Does this integration only work if you have a physical CT clamp?

I do have a P1 reader, so my consumption and/or export back into net is already visable in HA.

[catsmanac (Arie Catsman)]

You can check if the Enphase Website or the Enphase App are showing production data?

[jnathalia (jn)]

I was just editing my own reply above…

Yes, when using https://enlighten.enphaseenergy.com I can see the production data…

[catsmanac (Arie Catsman)]

Looks like you have CT clamps as the envoy page is showing a ''metering" heading as well as production and consumption. Mine, which is non metered only shows micro inverter production.

I don’t know what data Enphase all gets from the Envoy and maybe it uses inverter data to report production.

The 0 production number shown on the Envoy page at 1pm seems very low. Enphase signals 54 watt around 2 pm. Unless it’s pitch dark, even under rainy conditions panels typically produce a little bit. The 54 watt reported by Enlighten over 12 panels is 4.5 watt for each panel. Almost as if the production CT is not reporting properly. I’d expect some small values and also that the lifetime value would increase over time.

If HA is reporting same values as what you see on that Envoy webpage it’s indeed something with the Envoy and not HA.

[jnathalia (jn)]

I have physically checked the Envoy. There are only the phase and neutral wire connected, no wires for a clamp (1 phase).

I think it is because of the model: Envoy-S-Metered-EU. As far as I know, the connection of clamps is optional…

I will look for settings to turn off the metering… or maybe someone else has experience?

[catsmanac (Arie Catsman)]

Envoy metered without installed/configured CT behaves different. In the HA Enphase Envoy Core integration we have some checks for this and use data from different source data if this is the case:

envoy metered without configured CT has whLifetime and wNow in inverters
whLastSevenDays and whToday are incorrect for both so either can be used

So you may need to change your rest setup to use wattHoursLifetime and wattsNow from /api/v1/production.

{
  "wattHoursToday": 36462,
  "wattHoursSevenDays": 189712,
  "wattHoursLifetime": 6139406,
  "wattsNow": 5740
}

[del13r]

I found this diagram for the 230V 50hz (Europe and Australia might be the same) version of the Envoy-S Metered.

Envoy-S_50Hz_MAN_EN_ROW.pdf

[jnathalia (jn)]

When reading, my Envoy S Metered should have a CT clamp for production (which is not installed). Optional is a CT clamp for consumption.

I will contact my installer to ask why he didn’t install.

[jnathalia (jn)]

Do you think I can still read the production without a physical CT clamp?

The integration of @del13r works with https://envoy.local/ which won’t work for me. The portal https://enlighten.enphaseenergy.com gets its information from the micro-inverters I believe, where I presume all data is stored in the cloud of Enphase and therefore still visible for me…

@catsmanac do you think I’m able to fetch the data into HA? When using https://envoy.local/api/v1/production I get:

[catsmanac (Arie Catsman)]

What is /production showing for data?

[jnathalia (jn)]

@jnathalia

https://envoy.local

https://envoy.local/production.json gives:

{"production":[{"type":"inverters","activeCount":12,"readingTime":1727961428,"wNow":567,"whLifetime":2463751},
{"type":"eim","activeCount":1,"measurementType":"production","readingTime":1727961444,"wNow":0.0,"whLifetime":0.049,"varhLeadLifetime":1.963,"varhLagLifetime":1395.445,"vahLifetime":10187.493,"rmsCurrent":-0.0,"rmsVoltage":243.464,"reactPwr":-0.0,"apprntPwr":0.0,"pwrFactor":0.0,"whToday":0.049,"whLastSevenDays":0.049,"vahToday":23.493,"varhLeadToday":0.963,"varhLagToday":3.445}],"consumption":
[{"type":"eim","activeCount":1,"measurementType":"total-consumption","readingTime":1727961444,"wNow":-0.714,"whLifetime":-7186.364,"varhLeadLifetime":-0.29,"varhLagLifetime":31.743,"vahLifetime":257393.467,"rmsCurrent":-0.165,"rmsVoltage":243.403,"reactPwr":0.423,"apprntPwr":-40.238,"pwrFactor":-1.0,"whToday":0.0,"whLastSevenDays":0.0,"vahToday":575.467,"varhLeadToday":0.0,"varhLagToday":0.743},
{"type":"eim","activeCount":1,"measurementType":"net-consumption","readingTime":1727961444,"wNow":-0.714,"whLifetime":-7993.991,"varhLeadLifetime":1.673,"varhLagLifetime":1427.187,"vahLifetime":257393.467,"rmsCurrent":-0.165,"rmsVoltage":243.403,"reactPwr":0.423,"apprntPwr":-40.238,"pwrFactor":-1.0,"whToday":0,"whLastSevenDays":0,"vahToday":0,"varhLeadToday":0,"varhLagToday":0}],"storage":
[{"type":"acb","activeCount":0,"readingTime":0,"wNow":0,"whNow":0,"state":"idle"}]}

“wNow”:567 seems to give the current production in Watt…
“whLifetime”:2463751 seems to give an even better cumulative figure as it can be deducted from the prior read out to get the last production in kWh

[catsmanac (Arie Catsman)]

Yes, these are used the the integration you used before and the HA Core integration.

[dschrijnders (Dennis Schrijnders)]

I have a similar problem as @jnathalia.

I also have the Envoy S-metered-EU, without any CT clamps installed. In the core HA integration the sensors are all at 0. However, when I look at device info is does say “Envoy, phases: 1, phase mode: three, production CT”. See also the attached screenshot.

Like @jnathalia, the data in production.json → inverters is accurate when compared to the Enphase Enlighten website.

How do I get the integration to use the values in the inverters section of the production.json file? Is the integration (wrongly?) thinking I have a production CT (based on the info under device info)?

[jnathalia (jn)]

We have a small difference @dschrijnders as you have been able to install the HA integration and I haven’t. I have another site and there I’m capable with the standard HA integration but this one is at firmware D5.x.

I discussed with my PV installer and he admitted that a clamp should be installed and proposed to send me 2. However, I have a 1-phase installation so 1 is sufficient (for Production), for Consumption I need 3 clamps as I have 3 phases. So, in total 4 are then required. But I do have a P1 reader from HomeWizard so the additional 3 are not necessary.

I have a positive feeling that the RESTfull integration of @del13r will work…

I’m wondering how you succeeded to install the HA integration…

By the way, although you succeed to install the HA integration (maybe DEV from briancmpbll), you still have the same issue as the production meter says 0. Also you, I believe, should install a production CT clamp…

I will report as soon as I have installed into mine.