Trouble with Enphase Envoy

Configuration
21

Hi,

I found that if I take out the 7-day reading it does work ??

22

I dropped it down to just current and lifetime but still no good.
Is the storage values new?
I have SMA for my batteries, they are painful to get info from, wish I had gone Victron instead.

23

Mine looks like this;

# Enphase production readout
- platform: enphase_envoy
  name: Enphase Envoy
  ip_address: x.x.x.x
  monitored_conditions:
    - production
    - daily_production
    - lifetime_production
24

Got mine working again.
I moved ip from secret back to configuration.yaml

25

Hello all, thanks for the neato home-assistant tools, and thanks @DavidDeSloovere, @exxamalte and probably others for doing the envoy setup.

I’m new so my configuration may be the issue, or it may be I have an older Envoy, production only, oval white with small LCD. ( Current s/w R3.7.26 (7888b3), s/w Build Date Mon Aug 25, 2014 01:56 PM PDT )
I have hassbian on a pi, and 0.93.1 home assistant. It seems to work but my only device / sensor is a DTE energy bridge which is now working through mosquitto. This is all great fun, gives me something to do in between replacing the m190s on the roof! :grimacing:

sensor:
  # Solar production
  - platform: enphase_envoy
    ip_address: 192.168.15.18
    monitored_conditions:
      - production

With the envoy, I get an error coming from envoy_reader.py where it’s determining the envoy model.

enphase_envoy: Error on device update!
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity_platform.py", line 261, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 377, in async_device_update
    await self.hass.async_add_executor_job(self.update)
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/enphase_envoy/sensor.py", line 83, in update
    self._state = getattr(EnvoyReader(self._ip_address), self._type)()
  File "/srv/homeassistant/lib/python3.7/site-packages/envoy_reader/envoy_reader.py", line 53, in production
    raw_json = EnvoyReader.call_api(self)
  File "/srv/homeassistant/lib/python3.7/site-packages/envoy_reader/envoy_reader.py", line 39, in call_api
    EnvoyReader.detect_model(self)
  File "/srv/homeassistant/lib/python3.7/site-packages/envoy_reader/envoy_reader.py", line 34, in detect_model
    "Check that the device is up at 'http://" + self.host + "'.")
RuntimeError: Could not connect or determine Envoy model. Check that the device is up at 'http://192.168.15.18'.

For the newer envoy-S with consumption monitoring, the code expects http://{}/production.json to exist, which also does not exist from my envoy, as expected. (it gives an emu picture and ‘page not found’)

The code then expects the page http://{}/api/v1/production to exist for a production only model envoy, but that page also doesn’t work from my envoy. It says:

  {
      "status": 404,
      "error": "",
      "info": "Resource /api/v1/production not found",
      "moreInfo": ""
    }

My envoy does have this page: http://{}/production it returns text in boxes (Currently = dark outside…).

 System has been live since
Thu May 12, 2011 04:44 PM EDT
Currently	0 W
Today	19.5 kWh
Past Week	128 kWh
Since Installation	45.0 MWh

I played with the /srv/homeassistant/lib/python3.7/site-packages/envoy_reader.py file. I changed to look for the /production page mine has, and it gets past the original error detecting the envoy model. It populates the sensor value for production, but the state shows “Unable to connect to Envoy. Check that the device is up at ‘http://192.168.x.x’.”

I changed the envoy_reader.py code to look for ‘Currently’ instead of ‘wattsNow’.

def production(self):
    try:
        raw_json = EnvoyReader.call_api(self)
        if self.endpoint_type == "PC":
            production = raw_json["production"][1]["wNow"]
        else:
            #production = raw_json["wattsNow"]
            #production = raw_json["Currently"]                  tried this
            production = raw_json["Currently"][1]["W"]         then this,  still no go...
        return int(production)

It still gives the same error for the state, “Unable to connect to Envoy. Check that the device is up at ‘http://192.168.x.x’.”

So…, could I have a configuration issue, or is my envoy s/w different and I need to adjust the .py code? Is it SimpleJSON vs. Python JSON, I haven’t installed any python special beyond whatever is in hassbian. In any case, I’m hardly dangerous with python, but I’d love to fix & test it.
Thanks!!

26

Since you’re already looking into the envoy_reader library code, you would have noticed that the first thing it does is trying to detect the API end-point in the detect_model method. And this will still fail because as you pointed out yourself, your Envoy does neither have the /production.json nor /api/v1/production end-point available.
Also, the response you posted is not valid JSON so the parsing code won’t work.

If you have a bit of time at your hands and are willing to do some Python programming, can I suggest you implement a proper extension for the envoy_reader library, i.e. add another check for your Envoy’s text end-point into detect_model, then amend all the *production methods to take care of parsing the text (not JSON) your Envoy returns, possibly converting the values into a standard W/Wh unit, etc., and then ask the maintainer of the library to include your extension.
Maybe there are some more people with that Envoy model who would benefit from this extension.

1 Like
27

Thanks @exxamalte that’s what I would like to do.

Where could I find an example of scraping data from an http web page on a device?

I’m not familiar with json, am I sure yet that my device does not have a page like that? I’m poking at it from a browser, maybe I need to do from python to be sure what the EnvoyReader.call_api() result is.

28

You can have a look at the code of one of my custom components where I am retrieving an HTML page from a cable modem and then use a regular expression to find the data I am looking for. To find regular expressions that match the values you are getting from your Envoy device might be a bit tricky, so I’d recommend using a helper like pythex.

29

I think you’re right, that’ll be the first tricky part. I played with pythex, plugging in the relevant html returned by my Envoy:
Does this look like it would work? It gets the desired zero, but, Again, it’s nighttime, I’ll test with data from when it’s producing, the extra digits might matter depending on how it formats it I suppose.
<td>Currently</td> <td>(......)W</td>

Here is the html the envoy returns from the production page:

    <!-- START MAIN PAGE CONTENT -->
      <h1>System Energy Production</h1>
      <div style="margin-right: auto; margin-left: auto;"><table>
      <tr><td colspan="3">System has been live since
                             <div class=good>Thu May 12, 2011 04:44 PM EDT</div></td></tr>
      <tr><td>Currently</td>    <td>    0 W</td></tr><tr><td>Today</td>     <td> 30.0 kWh</td></tr><tr><td>Past Week</td>    <td>  132 kWh</td></tr><tr><td>Since Installation</td>    <td> 45.0 MWh</td></tr>
      </table><br></div>
    <!-- END MAIN PAGE CONTENT -->
30

I’m afraid that this regex would not be generic enough. Your six dots only match if there exactly six characters including whitespace. As soon as you produce 10W it wouldn’t match anymore.
You probably need to look at using character classes and quantifiers. The following for example matches any number, with or without digits.
<td>Currently</td> <td>\s*(\d+|\d+\.\d+)\s*W</td>

And as mentioned before you would also need to take care of parsing the units as I assume that they change. The following matches “W” or “kW” in a second group:
<td>Currently</td> <td>\s*(\d+|\d+\.\d+)\s*(W|kW)</td>

31

I have the envoy_reader working now for my older Envoy C which has only html pages, if anyone else comes along with an Enphase system circa 2010/2011.

This still has to get rolled in after checking on another issue, but this worked for me on 0.93.1 ha for about a month, and on 0.94.4 ha today after putting in github and mostly integrating with the latest envoy_reader version. https://github.com/dalklein/envoy_reader

It should still work for the newer Envoys with .json data, including the recently added data by inverter serial number. But test it, please let me know if not

Thanks for the help and pointers! (@exxamalte )

1 Like
32

@daklein I’m able to use the latest envoy_reader but with two minor changes. It seems after the update to envoy_reader about a week ago, it broke the gathering of pages to Envoy-C because an async call would fail, as well the Envoy-C does not have an info.xml page and that call would fail as well.

I have opened a PR to make two small minor changes so that it will pull only the HTML pages. In addition to the changes made on the envoy_reader side, HA was updated but not yet pushed to the released build as far as I can see from this PR. I’m running my private build of the envoy_reader.py along with the sensor.py from the previously mentioned HA PR with HA 0.94.3 against an Envoy running R3.11.30, without any problems.

Take a look and see if changing the envoy_reader.py to my private build along with the updated sensor.py in the PR, fixes your problem.

Greg

1 Like
33

thanks Greg @gregtd I think I have done that, taken your fixes to envoy_reader.py and the async related changes that will be in enphase_envoy/sensor.py component. I tried again by cloning jesserizzo/ha and using the sensoy.py from there.

Now I get this error, one for each of the 4 items it’s reading. It may be due to my changes. I can try your envoy_reader also, but it won’t do the older envoy non .json.

   Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity_platform.py", line 261, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 375, in async_device_update
    await self.async_update()
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/enphase_envoy/sensor.py", line 89, in async_update
    _state = await getattr(EnvoyReader(self._ip_address), self._type)()
  File "/srv/homeassistant/lib/python3.7/site-packages/envoy_reader/envoy_reader.py", line 248, in seven_days_production
    return int(seven_days_production)
UnboundLocalError: local variable 'seven_days_production' referenced before assignment
34

@daklein From the trace it appears you are running your envoy_reader.py from https://github.com/dalklein/envoy_reader/blob/ef2633b0887ccaa09fb219255afbe09a3e4d3195/envoy_reader/envoy_reader.py?

I’m still looking through the code, but from what I can see all of the 4 production methods have the same if else logic.

Your Envoy does not respond to http://IP_ADDR/api/v1/production, but responds to http://IP_ADDR/production, correct? So the code should be setting the model (endpoint_type) to P0?

Thanks
Greg

1 Like
35

Thanks for looking !

yes, the P0 section is what I added. somehow now it is failing in this set of if / else logic I think? not getting a value assigned. I tried initializing it to zero, which prevents the use before assign error, but then it stays stuck at zero.

I’m not sure how it’s failing now. maybe it’s the re.search looking for text, but text still awaiting?

36

@daklein Your envoy_reader.py also fails on my Envoy running R3.11.30. I found 4 places where an await needs to be added. Check out Lines 109, 162, 219, and 274. After I added the await to those lines I was able to retrieve production data from my Envoy.

Line 109 await EnvoyReader.detect_model(self)
Line 162 await self.detect_model()
Line 219 await self.detect_model()
Line 274 await self.detect_model()

Greg

1 Like
37

Thanks Greg @gregtd, much appreciated.

Tested and works now for me. With Envoy C s/w R3.7, ha 0.94.4 with requests_async installed to the ha venv, enphase_envoy component updated from this HA PR (thanks jesserizzo), and my changes to envoy_reader for the older Envoy C with only html pages.

Should show up in a future ha release. But in any case, it’s fun to have it working now for my setup.

38

Hi all,

My daily production always starts at approx 30Kw and counts up from there.
The number isn’t the same each day.
does anyone else have this issue or a fix?
with thanks :slight_smile:

39

Working fine for me.
Have you checked on the gateway directly? You can browse to some pages on the gateway.
Find the IP of the gateway and try some of the URLs that can be found here: https://github.com/jesserizzo/envoy_reader

40

#This works for me with RPI on python3.9
import time
import serial
import os
import glob
import time
import datetime
import urllib.request,json
from time import gmtime, strftime
from datetime import timedelta
import mysql.connector
#import Adafruit_BMP.BMP085 as BMP085
from time import gmtime, strftime
import sys

#HAAL NIEUWE CELLEN WEST) OP***
url = “http://192.168.178.185/api/v1/production”
#frequency = 180
response = urllib.request.urlopen(url)
data = json.loads(response.read())
E_tot = data[‘wattHoursLifetime’]
E_today = data[‘wattHoursToday’]
P_nu = data[‘wattsNow’]
#print(data)
print(“Totaal opgewekt : %6d Whr” %E_tot)
print(“Vandaag opgewekt : %6d Whr”%E_today)
print(“Momenteel vermogen: %6d W”%P_nu)
#HAAL NIEUWE CELLEN WEST) OP***