Goodwe rest sensor for multiple json elements

Sprk-nl · June 23, 2018, 5:09pm

After zillion tries, I can’t seem to get the template configuration correct.
I hope someone has more experience?

{
	"code": 1,
	"result": [{
			"inventersn": "xxxxxxxx",
			"desc": "",
			"vpv1": "353.40",
			"vpv2": "261.40",
			"ipv1": "0.90",
			"ipv2": "0.90"
		}
	]
}

sensor.yaml file:

# Goodwe Solar data
- platform: rest
  name: goodweinverter
  json_attributes: true
  value_template: '{{ value_json.code}}'
  resource: https://eu.goodwe-power.com/Mobile/GetMyDeviceListById_V3?stationId=xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx
  scan_interval: 300

- platform: template
  sensors:
    goodweinverter_test:
      value_template: '{{ states.sensor.goodweinverter.attributes}}'
      entity_id: sensor.goodweinverter
    goodweinverter_inventersn:
      value_template: '{{ states.sensor.goodweinverter.attributes.result[0]["inventersn"]}}'
      entity_id: sensor.goodweinverter
    goodweinverter_vpv1:
      value_template: '{{ states.sensor.goodweinverter.attributes["vpv1"]}}'
      unit_of_measurement: "V"
      entity_id: sensor.goodweinverter

2018-06-23 19:02:01 ERROR (MainThread) [homeassistant.components.sensor.template] Could not render template goodweinverter_inventersn: UndefinedError: ‘mappingproxy object’ has no attribute ‘result’

The first entry within the rest platform, showing “1” is ok.
I can’t get the other value’s out in correct sensors.

Sprk-nl · July 2, 2018, 4:09pm

I guess for the rest sensor the relation of the value template is directly for the template sensor.
Looking at the JSON data: “code” is on the same level, so I changed the sensor to “result[0]”

# Goodwe Solar data
- platform: rest
  name: goodweinverter
  value_template: '{{ value_json.result[0]}}'
  resource: https://eu.goodwe-power.com/Mobile/GetMyDeviceListById_V3?stationId=xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx
  scan_interval: 300
  json_attributes:
    - inventersn
    - vpv1

- platform: template
  sensors:
    inventersn:
      friendly_name: 'inventersn'
      value_template: '{{ states.sensor.goodweinverter.attributes["inventersn"] }}'
    vpv1:
      friendly_name: 'vpv1'
      value_template: '{{ states.sensor.goodweinverter.attributes["vpv1"] }}'
      unit_of_measurement: "V"

Before adding json_attributes in the rest sensor, it complained:

homeassistant.exceptions.InvalidStateError: Invalid state encountered for entity id: sensor.goodweinverter. State max length is 255 characters.

With this feedback, I think the data-set capture is correct.
I’ve added two attributes to the rest sensor: “inventersn” and “vpv1”, and the error is gone.
next the template sensor need to be configured correctly.

Sprk-nl · October 11, 2018, 3:27pm

I stopped using the API’s as it doesn’t give me all the details I would like.
Now I scrape the Goodwe Sems portal with the following code:

Stringyb92 · November 25, 2018, 9:47pm

Does this still work ? I am looking to use it within my setup but unsure how to go about it? A simple guide would be brilliant! Good work so far!

Sprk-nl · November 26, 2018, 8:14am

Does it work? Yes. Can it be done better. Definitely
Check this Guide on how to run the code

Paulhoek · December 20, 2018, 5:24pm

@Sprk-nl

Can you post a copy of your sensor configuration in the configuration.yaml?

TimmyBankers · December 28, 2018, 4:17pm

I got it to work!

Clone the repo (github.com/Sprk-nl/goodwe_sems_portal_scraper) somewhere.
Edit the file to have your user and password for the SEMS portal.
I also added some print statements for debugging.
Also change:
mqtt_host = "localhost"
and add after line client =mqtt.Client(mqtt_client_name):
client.username_pw_set('homeassistant', password='hello')

In automation.yaml:

# needed by gsps_beta
mqtt:
  password: hello # this starts an embedded MQTT server
  discovery: true
  discovery_prefix: homeassistant

Now you can run the python file:
python gsps_beta.py

It should add a gazillion sensors to home assistant. I hope they can somehow be grouped.

I now run the python file using crontab every 5 mins:
crontab -e
in my current (to be fixed) case: */5 * * * * /opt/homeassistant/bin/python /home/homeassistant/.homeassistant/custom_components/gsps_beta.py >> /var/log/gsps_beta.py.log 2>&1

Stringyb92 · December 31, 2018, 12:25pm

Did you install this on Hassio or just Home Assistant? I am running Hassio so i am unable to use the script as it doesnt allow for using imports.

TimmyBankers · December 31, 2018, 1:18pm

I’m using home assistant.
Unfortunately the script itself runs outside of home assistant and I just call it using Python.
Home assistant just runs the MQTT bus that the script connects to.

Stringyb92 · December 31, 2018, 1:58pm

I’ve just ran the script within VSC and it connected with HA, and i can see on MQTT.FX that it was sending all the info through. However i cannot see the new sensors which should have appeared?

TimmyBankers · December 31, 2018, 2:09pm

gist.github.com

https://gist.github.com/TimSoethout/64c26c89409d9d591de15f67407426a5

gsps_beta.py

#!/usr/bin/env python3
import sys
import datetime
import time as t
import os
import subprocess
import json
import re
import base64
import ssl

This file has been truncated. show original

I added username and password for MQTT.
And print statements for some output.

For the new sensors, you need home assistant to listen to the MQTT and have autodiscovery on.
The config:

mqtt:
  password: hello # this starts an embedded MQTT server
  discovery: true
  discovery_prefix: homeassistant

enables this with an MQTT server embedded in home assistant.

Stringyb92 · December 31, 2018, 4:26pm

Managed to get it working now on my Mac running it on visual studio. Need to try and work my Nuc so i can run this as a cron task within the server instead of my manually deploying

TimmyBankers · December 31, 2018, 5:52pm

I’m now trying to convert @sprk-nl’s code into a custom home assistant component. It will not use the MQTT stuff anymore then.
Any comments or ideas I should take into account?

Stringyb92 · December 31, 2018, 7:13pm

I noticed that it pulls through the information, but without cross checking the info and the web page its hard to tell where the information has come from! If i can help let me know

TimmyBankers · December 31, 2018, 7:17pm

gist.github.com

https://gist.github.com/TimSoethout/6a9ff051eb8d334f00923c95c8797755

configuration.yaml

sensor:
  - platform: sems
    username: 'XXXX'
    password: 'XXXX'

sems.py

#!/usr/bin/env python3
import voluptuous as vol

import json
import logging
# import urllib.request
import requests
from bs4 import BeautifulSoup

from homeassistant.const import TEMP_CELSIUS

This file has been truncated. show original

Done for now. Seems to work. Nothing polished yet.

TimmyBankers · December 31, 2018, 7:26pm

TimmyBankers · December 31, 2018, 7:27pm

Stringyb92 · January 1, 2019, 6:55pm

Just tried it, and all working this end.

How often does this sensor update, and is it possible to change the refresh time?

TimmyBankers · January 1, 2019, 9:07pm

Nice to see it is working.

I don’t have a clue how often it updates. If I look in my grafana dashboard which shows the data that home assistant exports to influxdb, i have a data point around every 2/3 minutes for this day. If you have a clue how to control it, please let me know.

Sprk-nl · January 1, 2019, 9:42pm

Hi there,
Nice to see this progress on a Goodwe integration.
I have also received info about native access instead of web scraping for which I’ll update the code… Soon ish