Rainforest EAGLE-200 Energy Gateway - Does you have one?

slaveconstruct · August 14, 2017, 9:57am

Got it to work, but with a bit of modification. Biggest change was using rainforestcloud.com instead of polling the device locally, addition of auth headers, and adjusted formatting for the sensor value transform from hex to decimal.

- platform: rest
  name: instant demand
  resource: https://rainforestcloud.com:9445/cgi-bin/post_manager?Content-Type=application/x-www-form-urlencoded     HTTP/1.1
  method: POST
  payload: '{ <Command><Name>get_instantaneous_demand</Name><MacId>0xXXXXXXXXXXXXXXXX</MacId><Format>JSON</Format></Command> }'
  headers:
    Content-Type: application/json
    Cloud-ID: !secret api_rainforest_cloudid
    User: !secret usr_rainforest
    Password: !secret pwd_rainforest
  value_template: '{{ value_json["InstantaneousDemand"]["Demand"]}}'

- platform: template
  sensors:
    energy_meter:
      value_template: '{{ states("sensor.instant_demand") | int(states("sensor.instant_demand"), 16) }}'
      friendly_name: Energy Meter
      unit_of_measurement: 'kW'

Cheers.

shire · February 2, 2018, 5:35am

Are any of you having an issue where your eagle stops responding after being polled for a few hours? For me Home Assistant pulls data initially but after a while the cloud light on the eagle starts flashing and Home Assistant no longer gets a valid response. I have to reboot the device for it to start working again.

I have the original EAGLE-100 and I have tried polling locally and thru the cloud interface and I get the same results either way. I’m not sure if I have a defective device or if the original model just can’t handle the consistant polling.

@slaveconstruct why did you choose to poll thru the cloud interface instead of locally?

slaveconstruct · February 3, 2018, 7:23am

@shire - I’m running into the same issue with an EAGLE-100. Started about a couple of weeks ago. The web interface also ends up going belly up and is inaccessible until it’s rebooted.

I went the cloud way since all the docs from RainForest suggested this was best practice. Did end up polling it locally since I ran into polling issues via RainForest Cloud, only to discover the issue lies with the device itself.

A few reviewers on Amazon mention they are experiencing the same issue also:

https://www.amazon.com/gp/customer-reviews/R27ZNRUGBBRNN9/ref=cm_cr_getr_d_rvw_ttl?ie=UTF8&ASIN=B00AII248U

TD22057 · February 5, 2018, 2:31am

I had an eagle hooked up at my old house. The new house (which is actually a much older house) electric company won’t activate it so I can’t use it anymore. However, I don’t think you want to poll the device or poll the cloud - you can configure the eagle to directly post to a local web server instead of a remote server. I wrote a small python (python 2) web server which decodes the xml and sends out mqtt messages for everything that the eagle posts. You can find the code in this repo: https://github.com/TD22057/T-Home. Basically you run it and open a port on your local machine (server, pi, etc) and tell the eagle to use that as the “cloud” url to post to. I haven’t run this code in last year or two since we moved but I’d be happy to help someone get it back up and working. I still have hopes my electric company will activate my device at some point in the future.

TD22057 · February 9, 2018, 1:18am

ps: Let me know if you want to use my mqtt converter. I don’t think it would take a lot of work to make it a stand alone package and update it to use python 3.

kozoke · June 6, 2018, 12:44pm

I have the same issue with polling the eagle locally. From what I read, the device becomes overwhelmed by the api requests and locks up; hence, no data and the web interface no longer work. Knowing I also want pricing tier and tariff rate from the eagle (so I can price my daily energy usage - because I can ) I needed to change my approach.

I now only poll for the overall energy received once every 6 minutes using my config above for instant demand and then once a day for tier/tariff.

Based on some great insights/help from @metbril in this link: Turn on/off input boolean based on state change (integer increasing or equal)?

I am considering using a

statistics sensor paired with a
binary sensor to ID change in energy delivery,
linked to a history_stat sensor.

I think I could coax out some sort of instant demand every minute, to replicate/replace the instant_demand reading that is overwhelming the eagle.

Hopefully that helps.

dethpickle · July 17, 2018, 5:04am

I was having the same type of problem - I ended up by “solving it” by having the eagle on a z-wave switch I could clobber from the comfort of my desk and the iOS app. Lame, but handy.

However, I recently shifted quite a lot of my stuff over to node-red. When doing that I took advantage of the fact that Rainforest allows you to setup different push endpoints aside from just them. I have it pushing to my hassio with a local address and have the following flow suck it into HA.

[{"id":"73259baf.26b934","type":"comment","z":"35a3515e.e0e226","name":"Get Power Data","info":"","x":100,"y":80,"wires":[]},{"id":"b9a3d8cc.852f8","type":"http in","z":"35a3515e.e0e226","name":"RainforestEagle","url":"/rest/metering/upload","method":"post","upload":false,"swaggerDoc":"","x":100,"y":120,"wires":[["c45399e.ae29fe8","67380bd.3d47674"]]},{"id":"c45399e.ae29fe8","type":"http response","z":"35a3515e.e0e226","name":"","statusCode":"200","headers":{},"x":390,"y":200,"wires":[]},{"id":"67380bd.3d47674","type":"change","z":"35a3515e.e0e226","name":"Filter","rules":[{"t":"set","p":"payload","pt":"msg","to":"$round($number(payload.body[0].data.demand), 3)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":570,"y":120,"wires":[["514562f5.a44664","5a317d88.75af14","9cb0c6d3.e1317","251101ef.a64d46"]]},{"id":"514562f5.a44664","type":"mqtt out","z":"35a3515e.e0e226","name":"mqtt","topic":"sensor/Rainforest/InstantaneousDemand","qos":"2","retain":"true","broker":"8119e086.37a008","x":950,"y":40,"wires":[]},{"id":"5a317d88.75af14","type":"ui_gauge","z":"35a3515e.e0e226","name":"Power Guage","group":"ff52d772.df3c2","order":1,"width":0,"height":0,"gtype":"gage","title":"","label":"kilowatts","format":"{{value}}","min":0,"max":"3","colors":["#00B500","#E6E600","#CA3838"],"seg1":"1","seg2":"2","x":980,"y":100,"wires":[]},{"id":"9cb0c6d3.e1317","type":"ui_chart","z":"35a3515e.e0e226","name":"Power Graph","group":"ff52d772.df3c2","order":2,"width":0,"height":0,"label":"Instantaneous Power","chartType":"line","legend":"false","xformat":"hh:mm a","interpolate":"linear","nodata":"Nothing Yet","dot":false,"ymin":"","ymax":"","removeOlder":"2","removeOlderPoints":"1000","removeOlderUnit":"3600","cutout":0,"useOneColor":false,"colors":["#1F77B4","#AEC7E8","#FF7F0E","#2CA02C","#98DF8A","#D62728","#FF9896","#9467BD","#C5B0D5"],"useOldStyle":false,"x":970,"y":140,"wires":[[],[]]},{"id":"251101ef.a64d46","type":"debug","z":"35a3515e.e0e226","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":770,"y":200,"wires":[]},{"id":"8119e086.37a008","type":"mqtt-broker","z":"","name":"Local MQTT","broker":"http://localhost/","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","closeTopic":"","closeQos":"0","closeRetain":"false","closePayload":"","willTopic":"","willQos":"0","willRetain":"false","willPayload":""},{"id":"ff52d772.df3c2","type":"ui_group","z":"","name":"Power","tab":"4e73ab2e.417e0c","order":1,"disp":true,"width":"6","collapse":false},{"id":"4e73ab2e.417e0c","type":"ui_tab","z":"","name":"Meters","icon":"dashboard","order":1}]

I haven’t had a crash since - I also happen to not be polling it at all anymore, since I don’t need to.

There’s probably a native HA way to do this. But I was trying to learn Node-red and it’s really easy there and all…

Dave

poldim · July 17, 2018, 11:07am

Are you still pulling from the RF cloud or directly from the device over the LAN? I’m not even sure if the local HAN device itself runs the API framework to serve the information or if it’s only a client that serves it up.

dethpickle · July 17, 2018, 2:32pm

Right. It’s all local, though you have to configure the Eagle from their cloud service. If you go to the cloud section of the Rainforest controls and Add a new Upload Destination then you can type in anything you want.

The url is in addition to a field there for hostname. I’ve put in my local address for the hassio as the hostname (where I’m running node-red) and the url is just the endpoint where node-red is listening. It works just fine. The biggest upshot is that I’m never going to poll the same info twice. The Eagle seems to be spitting it out every 10s or so. Every once in awhile it seems faster or slower, but it’s basically as often as it would update the cloud service.

poldim · July 28, 2018, 11:15pm

Ok, so I’ve gotten a bit further. I went into RF Settings for cloud, hit the plus to add a custom cloud service, gave it a name and then gave it the LAN IP for my NR instance with the addition of the URL /rest/metering/upload. After it was added, I went and selected it as the cloud service. It went green to indicate it’s connected and I started to get some items to come up on the debug screen.

However, the debug node that you had in place only states msg.payload : undefined
I added another one directly off of the HTTP IN node. This one does provide some basic information, but it’s lacking the data field so the change node returns nothing. There is no payload.body[0].data.demand.

<?xml version="1.0"?><rainforest macId="*" version="undefined" timestamp="1532808872s">
<InstantaneousDemand>
  <DeviceMacId>*</DeviceMacId>
  <MeterMacId>*</MeterMacId>
  <TimeStamp>0x22ef8b28</TimeStamp>
  <Demand>0x0001b5</Demand>
  <Multiplier>0x00000001</Multiplier>
  <Divisor>0x000003e8</Divisor>
  <DigitsRight>0x03</DigitsRight>
  <DigitsLeft>0x0f</DigitsLeft>
  <SuppressLeadingZero>Y</SuppressLeadingZero>
  <Port>/dev/ttySP0</Port>
</InstantaneousDemand>

</rainforest>

Any ideas on what I can do to get the demand info out?

dethpickle · July 29, 2018, 2:45am

Ehh… that was awhile ago, but… here it is again:

[{"id":"73259baf.26b934","type":"comment","z":"35a3515e.e0e226","name":"Get Power Data","info":"","x":100,"y":80,"wires":[]},{"id":"b9a3d8cc.852f8","type":"http in","z":"35a3515e.e0e226","name":"RainforestEagle","url":"/rest/metering/upload","method":"post","upload":false,"swaggerDoc":"","x":100,"y":120,"wires":[["c45399e.ae29fe8","67380bd.3d47674"]]},{"id":"c45399e.ae29fe8","type":"http response","z":"35a3515e.e0e226","name":"","statusCode":"200","headers":{},"x":390,"y":200,"wires":[]},{"id":"67380bd.3d47674","type":"change","z":"35a3515e.e0e226","name":"Filter","rules":[{"t":"set","p":"payload","pt":"msg","to":"$round($number(payload.body[0].data.demand), 3)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":570,"y":120,"wires":[["514562f5.a44664","5a317d88.75af14","9cb0c6d3.e1317","251101ef.a64d46"]]},{"id":"251101ef.a64d46","type":"debug","z":"35a3515e.e0e226","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":770,"y":200,"wires":[]}]

I’ve chopped out everything but the endpoint, 200 response, a filter and a debug node to see what’s coming in.

The point of the endpoint is that it must match whatever you made in the rainforest cloud for your local endpoint - mine is rest/metering/upload and it looks like that might be yours too. I’m assuming the 192.168.1.100 is your hassio/node-red box. If so, when you deploy that, you should be getting your rainforest to push in every update to your node-red. What you do with it from there is up to you. I had some charts and feed it into my radiotherm display and what-not, also kicking it over the hassio via mqtt.

But, assuming your rainforest and hassio are on the same network and the 192.x.x.100 was the hassio address, you’re doing what I’m doing. Direct off the endpoint you should be getting a whole mess of stuff. The filter chops it down to just the instant demand for me.

You may have selected the XML format, instead of json, in the cloud setup. My raw dump out of the endpoint looks like this:
{"timestamp":"1532832222000","deviceGuid":"xxxxx","body":[{"timestamp":"1532832217000","subdeviceGuid":"xxxxx","componentId":"all","dataType":"InstantaneousDemand","data":{"demand":1.048,"units":"kW"}}]}

poldim · July 29, 2018, 4:49pm

I didn’t see this option in my setup. Just checked again. Any chance you can take a screenshot of your view where it gives you that option?

Doing a bit more research, do you have the four or five light one? https://rainforestautomation.com/cloud-login/

dethpickle · July 29, 2018, 7:06pm

Without channeling Orwell or Star Trek, I see five lights. The newer Eagle200.

The pisser about the Rainforest Cloud is that they won’t show me how it’s configured / you can’t edit an existing setup. Just delete it and make a new one. Now, I’m at: https://portal.rainforestcloud.com/user/settings and it looks like this. You can see the existing one and I’ll fill out a new one to the best of my recollection:

shire · July 30, 2018, 2:43am

@TD22057 - I finally got tired of eagle crashing and I downloaded your code. I know enough Python that I was able to get it working on Python3.5. I commented out all the non-eagle/non-essential modules, commented out a couple of error checking lines (raise statements) I didn’t know how to rewrite and updated all the StringIO references to io since it is the 3.x replacement. I think there was a couple of other lines I had to update but overall it wasn’t too hard.

I have had a steady stream of energy data for more than a week now. Thanks for sharing your code!

raaahbin · August 1, 2018, 4:20am

@shire this sounds like exactly what I need to do! However, I’ve not played much with python, and although I don’t think I need to do any updating for 3.5 (the system I’m running on has a 2.7.x version), I would like to replicate what you’ve done (i.e. just run the eagle component). I have mosquitto running on my local server now, and it’s working perfectly, so this is the last piece of the puzzle. Can you tell me how you commented out all the modules, and how you actually start it running? I’ve tried running the bin/tHome-eagle.py file, and it just fails with: ImportError: No module named tHome (previously it was telling me that it was missing “bottle”, but I used pip to install that).

I’ve edited the conf/eagle.py file to reflect what I’d like it to tell the broker, so I think if I can just start it, it should work… but that’s the point where I’ve got stumped. Some help from someone who’s just done it would be most appreciated!

raaahbin · August 3, 2018, 12:04pm

I got it working… almost! So close, and yet so far. It appears the server is running, but there is a failure to communicate - specifically, the Eagle is saying something the server doesn’t want to hear. I’ve attempted setting the Eagle to send XML (raw), XML (processed) and JSON, and while JSON definitely gets a worse problem, neither of the XML options appear to work. Here’s the error that appears in the tHome/eagle.log file:

2018-08-03 21:56:44,227 : ERROR: Error parsing Eagle posted data
Traceback (most recent call last):
 File "/Users/robin/thome/bin/tHome-eagle.py", line 66, in root_post
   obj = T.eagle.parse( data )
 File "/Users/robin/thome/python/tHome/eagle/parse.py", line 16, in parse
   assert( root.tag == "rainForest" )
AssertionError

What I get from that is, it’s receiving XML from the Eagle, but not what the script is expecting. Any suggestions as to how I might find out what the Eagle is sending, and to persuade the tHome script to parse it?

shire · August 6, 2018, 1:23pm

@raaahbin I’m not where I can get to my install at the moment but I think the issue with that line was that the root tag should be all lowercase (root.tag == “rainforest”).

raaahbin · August 6, 2018, 1:47pm

Thanks - it was! I actually figured it out soon after I posted my previous, then was so busy dealing with the influx of tasty, tasty data that I forgot to come back and post that I’d solved it.

evanrich · August 27, 2018, 4:27am

@TD22057 would love to see this work with python3. I’ve got your code running right now in a freebsd jail with an Eagle (first gen), but I run all my stuff in Kubernetes(docker) at the moment. Working on trying to port your code into a docker container based on alpine, having difficulty though (never wrote a docker container before). I’ll post here if I get it working, thanks for posting your code!

evanrich · August 28, 2018, 4:36am

Here’s a start to the Dockerfile, if anyone is interested in taking a look/contributing. It works as is, you need to change configs for MQTT broker IP and port, of course, but I’ll add in variables for those shortly. Once it’s fully ready I’ll push to dockerhub.