Solar Analytics API integration into HA Energy Management

Third party integrations
42

Hi all,

I’ve added a proposed update to GitHub as follows:
Use Solar Analytics sourced data to report Consumed, Generated and Import-Export Power (W)

Working Lovelace cards:-

Summary house hold power import/export, consumption and generation:

image
Current (last 5 minute average) power import/export - showing minimal export as our EV is currently charging:

image

image
image1083×765 39.9 KB

43

More updates on github - as per the link provided above.

44

Hi Peter

Nice work with the doco!

I’ve finally pushed my HA config to github and there are a few things in there you might be interested in. See https://github.com/123go/home-assistant-config

Firstly, rather than put my SA config directly in configuration.yaml, I’ve put mine in a package that is included in the main yaml file (see solar_analytics.yaml in the packages folder). This helps to modularise things and keep the main file a lot cleaner.
Secondly, (and maybe of more interest to you) I’ve used the site_id that is retrieved from the site_list service call to build the URL for the site_data service call. This means your config doesn’t need to hard code the site_id.
Lastly, I’ve created two “calculation sensors” (this is my terminology, not an official term… as far as I know) that hold all the logic for extracting values from the REST services and calculating any totals, most recent values, etc. The benefit of this is that it separates the complicated code and puts it all in one place and greatly simplifies the config for the sensors that are actually usable in HA.

Anyway, hope you can find something useful in there. Let me know if there’s anything that needs more explanation.

Regards
Glen

2 Likes
45

Thanks again to both of you for all of this. It’s been running nicely for me. And yesterday I updated it with the latest version on github.

I’ve been background reading and poring over the yaml to try to better understand it… then went down lots of dead ends before arriving at some extra sensors. I can now read the latest 5 minute data for any of the SA clamps. I got there by hacking together some of your earlier yaml and things I picked up elswhere. But it turns out it’s a lot like your sa_data_detail (which for me returns data for my stove and oven) - except that it’s the most recent readings. I managed this by using an end date that should be always one day in the future (kudos to my daughter for that bit of the yaml). It’s easily adaptable to return data for any or all 6 of the clamps.

Will copy my code below… welcoming any feedback on how it could be better… there are plenty of bits I don’t truly understand. Maybe it will be helpful for someone else with the same basic skills as me.

Now I can get on with some automations using the SA clamps as triggers. Will also be doing some work to better understand time syntax too… just that I noticed that SA report in Unix time syntax but as if it were in the local time zone. So my time sensor is not right… maybe that’s why your sa_detail_time sensor is reporting the end of the 24 hour period as if it were midnight UTC rather than local time.

Substitute site ID for the #s in the resource template. And note that I am only looking at data for 3 of the attributes… but could easily get the others if needed.

  - platform: rest
    name: sa_my_clamp
    resource_template: 'https://portal.solaranalytics.com.au/api/v2/site_data/######?gran=minute&raw=true&tstart={{ now().strftime("%Y%m%d") }}&tend={{ ((as_timestamp(now()) | int) + 24*60*60) | timestamp_custom("%Y%m%d") }}'
    username: !secret sa_username
    password: !secret sa_password
    authentication: basic
    value_template: "{{  now() }}"
    json_attributes_path: "$.data.[2].[0]"
    json_attributes:
      # - "apparentPower"
      - "current"
      - "energy"
      # - "energyNeg"
      # - "energyPos"
      - "power"
      # - "powerFactor"
      # - "reactiveEnergy"
      # - "reactivePower"
      - "time"
      - "voltage"
    scan_interval: 120
    force_update: true

  - platform: template
    sensors:
      sa_my_clamp_power:
        friendly_name: "My Clamp Power"
        value_template: "{{ state_attr('sensor.sa_my_clamp', 'power') | round(1) }}"
        unit_of_measurement: "W"
        device_class: power
      sa_my_clamp_time:
        friendly_name: "My Clamp Time Read"
        value_template: "{{ state_attr('sensor.sa_my_clamp', 'time') | int | timestamp_custom ('%d-%m-%Y %H:%M')  }}"
46

Thanks for your work on this, I’ve just added the config to my HA (paitently waiting for data to be populated, nothing yet).

Do the secrets have to be in inverted commas? “username” “password”?

Thanks.

47

Yes they do

48

Everything is up and running, no drama’s.
FYI, the secrets in the secrets.yaml do not have to be in inverted commas.

49

Thanks… will have a better look at my setup and apologies if I wasted any of your time.

1 Like
50

Thanks for putting this up. I’m pretty new to all this but I can copy and paste.
I’m not sure if what I did is correct of not but based on the instructions I changed the site_id 36304 in the 2 URLs as mentioned.

However there were 2 URLs and 2 sections (Dashboard status and 5min update) that didn’t seem to get data until I changed the 36303 (note the different number) to my site_id
I do get data doing this, the Dashboard status section has gone from unknown to uncertain and the 5min update section now has data.

Is this what I should have done?

Thanks, Josh

Note: mine shows GET360 as the portal as Green Energy Tech installed this if that makes a difference.

51

Is it working now? I’m not quite sure what your asking… sounds like you’ve done the right thing!

You should replace 36304 or 36303 with your site ID in the 2 relevant places in the yaml

52

Sorry, it is a little confusing, I’m pretty new to all this. It’s all working fine now. The original instructions on Github are as below:

  1. Substitute your site_id for the placeholder site_id (i.e. 36303) in the 2x resource URLs below.

Changing just those two URLs with 36303 didn’t work. I had to change another 2 URLs with 36304 to get it to work. The instructions mention 36303 and the config yaml mentions 36304

These snippets are in the original Github configuration.yaml

# Note: site_id value is required in URLs used in gets below - e.g. 36304

# Solar Analytics - get the site status for the specified site_id

    resource: https://portal.solaranalytics.com.au/api/v3/site_status/36303

# Solar Analytics - get the site raw data for the specified site_id

    resource: https://portal.solaranalytics.com.au/api/v2/site_data/36304?gran=minute&raw=true  

# Solar Analytics - get the daily data

    resource: https://portal.solaranalytics.com.au/api/v2/site_data/36304

# Solar Analytics - get detailed 5 minute data 

    resource_template: 'https://portal.solaranalytics.com.au/api/v2/site_data/36303?gran=minute&tstart={{ now().strftime("%Y%m%d") }}&tend={{ now().strftime("%Y%m%d") }}'

I just wanted to check what I did was right and I’m also guessing since mine is working the instructions should be:

  1. Substitute your site_id for the placeholder site_id (i.e. 36303 and 36304) in the 4x resource URLs below.

Hopefully that makes a little more sense.
Thanks, Josh

53

Does make sense and good you got it working!

54

Many thanks to all the contributors on this, you’re legends! Have updated my config and some early numbers have just started coming into the HA energy dashboard so looking like its working.

Best Regards
James

55

Hi All

I hadn’t been able to play with this for a few months but I took some time recently to revamp and simplify my Solar Analytics sensors.

There were quite a few things I didn’t like about previous iterations of this config:

  • It was mixed in with everything else in the configuration.yaml file. The latest version is in a standalone package file that is included in the main configuration.yaml file.
  • It was complicated. This was partly due to the complexity of the data in the “site_data” service but also just my inexperience with rest sensors, template sensors and jinja syntax. The latest version is much simpler.
  • The rest sensor that called the site_data service either relied on a hardcoded site_id (which would be exposed publicly if published to a git repository) or relied on called the site_list service first (which seemed like an unnecessary service call for something that never changes!). The latest version uses an input_text helper to store the site_id and it retrieves the value from the secrets file.
  • The site_data service with “raw=false” returns decent data for solar generation and total consumption but due to the fact that the data comes in 5 minute “buckets” it is not very accurate for calculating the energy that is imported from or exported to the grid. Fortunately, the “raw=true” data already contains these grid import/export values so we can use them directly. But as @PeterH24x7 pointed out here the timestamps in the raw data are a mess… but they’re not irredeemable. Thanks to @ddwdiot’s post here, I realised that the timestamps in the raw service were (kind of) valid, they were just using the wrong timezone - they were using the timezone of the solar site instead of using UTC. This meant that when the timestamps were converted using standard unix epoch timestamp conversion rules, it looked like the time was about 10 or 11 hours ago (I’m in NSW, Australia) whereas they were really for 5 minutes ago. Once I had that worked out, it was relatively simple to work around the timestamp screwiness and now my data, using the raw=true service, is more precise.
  • My sensors used the older syntax with the “sensor” integration + “rest” and “template” platforms. The latest version now uses the rest and template integrations. (Is the newer syntax any better? It’s debatable… but I think it is slightly simpler)

Anyway… all that to say that I have a new version of my Solar Analytics config that can be found here:
https://github.com/123go/home-assistant-config

If you want to install it the package manually, just upload the package file to your config folder (or subfolder) and then ensure you have a packages item in configuration.yaml that includes the package file itself (or the whole subfolder. I use the subfolder option).

Enjoy!
Glen

56

Hey Glen, so is there any difference in the data delivered between these two configs?

57

In my experience, yes.
The “raw data” sensor more closely matches the data from Solar Analytics web page/app. From memory, the difference between the sensors is more obvious when your consumption exceeds your solar production within any 5 minute period. And when those differences are added up over a day then it can become significant.
YMMV

58

No problem, I’m in the process of updating my config from Peters to yours. Will let you know if I see any major differences.

59

So, getting an error trying to implement.

All I did was remove Peter’s and insert yours, update my secret with the site id. My yaml file was already called solar_analytics.yaml

I also can’t see this direct code in the config.

expected a dictionary for dictionary value @ data[‘packages’][‘sensor’]

60

In which folder is your solar_analytics.yaml file located and what does the “packages” item in your configuration.yaml look like.

61

Solar is sitting in config/packages
configuration.yaml is: config/configuration.yaml
image

Noting that this is the original config. When I utilised your config, I used the hashed line.