#Challenge:
Hey all. A few months ago, we had a few solar panels installed. I was doubting between the brands of the inverter, but went with the SMA. I knew that they didn’t have an API for getting the data, but I knew I could probably retrieve it one way or the other. So my challenge was to get the values from the inverter into HA.
#Setup:
I want to run everything on a Raspberry Pi, even though the description below was only tested on a laptop running Linux. So still need to test it on the Rasp. Also, I was doing this on the train, with bad wi-fi, so I did not actually test the complete code yet.
In regards to the inverter: there are two ways that I can pursue to solve this challenge. I can either collect the data from the inverter directly, or through the Sunny Portal. In both cases I webscrape. I believe there are some SMA inverters and extensions that allow direct polling for values. If so, you can change your project accordingly.
#How I tackled it:
Part 1: Webscraping
My first step was to tackle to get the values like generated power. Since there is no API, I decided to go with webscraping. For that, the easiest way that I could find was CasperJS, as that had an intuitive way of logging in and getting the HTML of the page. You can find more about CasperJS here. Below you can find the code, but the steps that I follow on a high level are:
- Go to the Sunny Portal (or, connect to the wi-fi signal in your inverter)
- Sign in
- Get the value and print it
var casper = require(‘casper’).create({
pageSettings: {
loadImages: false,// Don’t load images, as it makes the script quicker
loadPlugins: false,// Don’t load plugins, as it makes the script quicker
userAgent: ‘Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36’,
},
waitTimeout: 10000,// A longer wait timeout
});
// Start the script
casper.start();
// Open Sunny Portal
casper.then(function(){
this.open('https://www.sunnyportal.com/', {
method: 'get'
});
});
// Fill in the credentials
casper.waitForSelector("#txtUserName", function() {
this.evaluate(function(username, password) {
document.querySelector('#txtUserName').value = username;
document.querySelector('#txtPassword').value = password;
}, USERNAME, PASSWORD); // Replace USERNAME and PASSWORD with your Sunny Portal credentials
});
// Click button to login
casper.then(function(){
this.evaluate(function() {
document.querySelector('#ctl00_ContentPlaceHolder1_Logincontrol1_LoginBtn').click();
});
});
// Get the current value
casper.waitForSelector(".mainValueAmount", function(){
var value = this.evaluate(function(){
return document.querySelector(".mainValueAmount").innerHTML;
});
// Get the value and parse it to float
value = parseFloat(value);
console.log(value);
});
// Run the script
casper.run(function(){
this.exit();
});
The above script only outputs the current power that is being generated, but of course you can output other values too.
Part 2: Adding value to HA
This part was interesting, as I was considering different options:
- Create a REST API using NodeJS, which performs a shell command and have HA poll the API in intervals
- Use NodeJS and the shell command option to create an MQTT client and have it send the information to the MQTT broker embedded in the HA (or another broker of course)
- Use NodeJS and the shell command option to send information to the REST API from HA, in combination with a template sensor
- Use the command line sensor option from HA
I started with implementing the first option and the third option. Both worked, but I thought it was a bit weird that I would need to implement NodeJS and have that running continuously to being able to have the value. This then also applied for the second option. So I found the possibility for the fourth option, and decided to go for that. And actually succeeding.
The only thing I needed to do was to add the command line sensor, and make sure it runs the casperjs shell command. That’s actually just it. It is important to note that the path has to be absolute. If not, it won’t find it and it won’t work.
// sensors.yaml
- platform: command_line
command: "casperjs ~/.homeassistant/sensors/scrape_current_data.js"
name: current solar power
unit_of_measurement: w
Add the sensor to anywhere you want, and it will update at regular intervals with the necessary information.
Things to still do and possible additions:
I do have some adjustments that I want to make:
- I want to test this in a Raspberry Pi in combination with a good internet connection. I have now tested it with fake data, but I am not sure if HA polling will wait for the value to be retrieved (as CasperJS takes some time to collect the information)
- I want to change the scanning interval, as Sunny Portal is updated about every 5-7 seconds. I have tried to change it, but it doesn’t seem to change. I’ll do some researching though
- I am considering to also send this information to InfluxDB and Grafana, tackling two things at once. To do this, it should be quite easy to create a bash script in which you perform the CasperJS script as well as sending the information to InfluxDB
#In conclusion
It is not the most elegant way of collecting data, and it is subject to change of the Sunny Portal website or the inverter website, but it is the easiest way I found to actually do this. If somebody has a better/cleaner/nice way of doing this, please let me know! Also, if somebody has an idea why one of the other options of getting the information into HA (with NodeJS and/or MQTT), I am curious about your thoughts.
).
