Solar Analytics API integration into HA Energy Management

I just created my sensors and incorporated them into the Energy Management config.
Whilst the sensor entities are showing data, they aren’t showing any data in the energy dashboard yet so I’m not sure if there’s still a problem with my sensors or if I just have to wait for Energy Management to “catch up”

    sa_energy_grid_import:
      unique_id: "sa_energy_grid_import"
      friendly_name: Energy Imported From Grid
      unit_of_measurement: "Wh"
      value_template: >
        {% set energy = namespace(imported=0) %}
        {% for sensor_data in states.sensor.sa_minute.attributes.data | rejectattr('energy_consumed', 'equalto', None) if sensor_data.energy_consumed > sensor_data.energy_generated %}
        {% set energy.imported = energy.imported + sensor_data.energy_consumed - sensor_data.energy_generated %}
        {% endfor %}
        {{ energy.imported }}
      icon_template: mdi:transmission-tower-export
      device_class: energy
      attribute_templates:
        state_class: total_increasing
    sa_energy_grid_export:
      unique_id: "sa_energy_grid_export"
      friendly_name: Energy Exported To Grid
      unit_of_measurement: "Wh"
      value_template: >
        {% set energy = namespace(exported=0) %}
        {% for sensor_data in states.sensor.sa_hour.attributes.data | rejectattr('energy_generated', 'equalto', None) if sensor_data.energy_generated > sensor_data.energy_consumed %}
        {% set energy.exported = energy.exported + sensor_data.energy_generated - sensor_data.energy_consumed %}
        {% endfor %}
        {{ energy.exported }}
      icon_template: mdi:transmission-tower-import
      device_class: energy
      attribute_templates:
        state_class: total_increasing
1 Like

Here’s some of the data to date. It looks a little strange as 99% of what would have been exported energy is going into our EV. A normal full day’s data will look better - especially for the bar charts (not shown).

Looks like I need to get myself an EV! :wink:

1 Like

@TheOtherGlen - I note your export template is still using sa_hour, rather than sa_minute.

Legend. Thanks. Changed now.

It still doesn’t explain why my export/import values aren’t showing up in the dashboard but I’m hoping it’s just this:

This is what I see in the Sources section of the dashboard:
image

I would say that you’re both legends!

One of the chief drivers of my fumbling efforts with HA has been the recent arrival of an EV. I’m about to get a sparkie to put one of my SA clamps on the relevant power outlet and was thinking I could use my solar more intelligently than I have been. Also have Fronius solar hot water diversion (but that’s off topic)

But I’ve been distracted by how well I can control my lights and various other devices/sensors… So I haven’t even looked at the HA Tesla integration - and now I find that the Charge HQ app, currently in Beta is very promising for EV charging using PV.

Had given up on SA integration until you two came along. But now I’m hoping I can use it as a means of automating my aircon more effectively.

Hi @ddwdiot - you’re again welcome. I hope this whole thread has made sense to you. I’m just watching my HA Energy module data over the next 24 hours to see that it’s all working as expected. I’ll then publish the final configuration.html code that will incorporate Glen’s optimisations as the likely final solution for this thread.

We have a Tesla M3 too and are using the Zappi v1 charger. It’s import/export aware and can maximise the charging from your own solar PV (see SA image below - you will note Melbourne had two heavy downpours this afternoon). If you only have single phase power (we have single phase max 60Amps) and you get close to your max current draw sometimes, the Zappi will also limit how much power it draws up to the max supply current so you don’t trigger the mains fuse protection. We sold our Tesla Wall Charger that came with our car for $500 and spent extra to get the Zappi. The only downside is that the Zappi doesn’t have a button on the charge handle to open/close the charge-port on the Tesla. I’ve separately mentioned to Glen, my focus with HA is on the energy mangement and a key part of that is my ability to enable/disable charging on our EV in response to variable 1/2 hour wholesale pricing offered by Amber. Sometimes Amber’s energy prices are negative (e.g. in the middle of the day) and I hope to get paid to charge my car (this also involves HA auto turning down the power output on my PV inverter so I don’t get charged to export).

1 Like

So we’re following similar paths (Model 3 and getting started with HA with initial interest in energy management). But I’m a bit behind in implementation and technical nous.

I’ve chosen not to get a Zappi - or Tesla equivalent… or wait on the Fronius Wattpilot. Was aware that it takes a long time to pay off and thinking I already have solar monitoring - the Tesla is smart and maybe software could help me with the rest. I needed another project to save me from watching rubbish TV at night anyway.

No Amber in Northern NSW at the moment… but I do have a Lifx bulb tracking the NEM price using nemy and IFTTT. Just for interest.

Looking forward to giving the latest yaml a good workout on the weekend.

1 Like

Everything seems to be working as expected and the numbers match what is in the Solar Analytics web-app. Here’s a screen shot of the HA Energy Management output. The low PV output reflects the presence of heavy clouds in Melbourne this morning.

@TheOtherGlen - I trust yours is working too?

I will shortly post the final code (so @ddwdiot can get it going) as the solution, and will provide some instructions for those that find this thread in the future. As a version 1 I think this is a good minimum viable product. I have some further ideas for a next round of development, but I think it would be best to start a second discussion for version 2. I’ll also publish it on github for wider public access.

1 Like

This is the Solution

Steps:

  1. Cut/past the YAML code below into the “sensor:” section of your HA configuration.yaml file.
  2. Edit your secrets.yaml file to include your Solar Analytics account username and password as sa_username and sa_password. For example:
sa_username: "[email protected]"
sa_password: "secret1"
  1. Validate the configuration.yaml, and then restart HA.
  2. On first run there won’t be any data, look in the sa_site_list attributes for your site_id.
  3. Substitute your site_id for the placeholder site_id (i.e. 36303) in the 2x resource URLs below.
  4. Again validate the configuration.yaml, and then restart HA and the data gets should now be working.
  5. In the HA Configuration > Energy page enter the details for
  • Grid consumption - as sa_todays_energy_imported
  • Return to grid - as sa_todays_energy_exported
  • Solar production - as sa_todays_energy_generated_total
  1. If all working correctly, the Energy dashboard should start populating data within a couple of hours.

A big thank you to @TheOtherGlen for his templating expertise and optimisations.
:grinning:

sensor:

#
# - SOLAR ANALYTICS SETUP ------------------------------------------------------
#
# Solar Analytics - get the site details associated with the given user account login
# Note: site_id value is required in URLs used in the gets below - e.g. 36303
# Updated every 24 hours
  - platform: rest
    name: sa_site_list
    resource: https://portal.solaranalytics.com.au/api/v3/site_list?hardware=true&capacity=true&subscription=true
    username: !secret sa_username
    password: !secret sa_password
    authentication: basic
    value_template: "{{ now() }}"
# note: assumes only a single device per account, there could be a second device (e.g. replaced device due to fault/upgrade) at $.data.[1]
    json_attributes_path: "$.data.[0]"
    json_attributes:
      - "e_status"
      - "fault_class"
      - "fault_id"
      - "has_pv"
      - "mer_status"
      - "overall_status"
      - "retailer_user"
      - "s_cli_site_name"
      - "site_id"
      - "site_inactive"
      - "capacity"
      - "devices"
      - "sub_type"
    scan_interval: 62400

# Solar Analytics - get the site status for the specified site_id
# Updated every hour
  - platform: rest
    name: sa_status
    resource: https://portal.solaranalytics.com.au/api/v3/site_status/36303
    username: !secret sa_username
    password: !secret sa_password
    authentication: basic
    value_template: "{{ value_json['data']['mer_status'] }}"
    json_attributes_path: "$.data"
    json_attributes:
      - "dashboard_status"
      - "event_id"
      - "event_list"
      - "fault_status"
      - "mer_percentage"
      - "mer_status"
      - "mer_text"
    scan_interval: 3600
    force_update: true

  - platform: template
    sensors:
      sa_dashboard_status:
        friendly_name: "Dashboard Status"
        value_template: "{{ state_attr('sensor.sa_status', 'dashboard_status') }}"
        entity_id: sensor.sa_status
      sa_mer_status:
        friendly_name: "System Status"
        value_template: "{{ state_attr('sensor.sa_status', 'mer_status') }}"
        entity_id: sensor.sa_status
      sa_mer_percentage: 
        friendly_name: "PV Performance"
        value_template: "{{ state_attr('sensor.sa_status', 'mer_percentage') }}"
        entity_id: sensor.sa_status
        unit_of_measurement: "%"

# Solar Analytics - get detailed 5 minute data 
# Used to calculate today's total energy for consumed, generated, imported and exported energy for the HA Energy module
# Updated every 5 minutes
  - platform: rest
    name: sa_data_by_5min
    resource_template: 'https://portal.solaranalytics.com.au/api/v2/site_data/36303?gran=minute&tstart={{ now().strftime("%Y%m%d") }}&tend={{ now().strftime("%Y%m%d") }}'   
    username: !secret sa_username
    password: !secret sa_password
    authentication: basic
    value_template: "{{ now() }}"
    json_attributes:
      - "data"
    state_class: measurement
    device_class: energy
    scan_interval: 300
    force_update: true

  - platform: template
    sensors:
      sa_todays_energy_consumed_total:
        friendly_name: Total Energy Consumed
        unit_of_measurement: "Wh"
        value_template: "{{ states.sensor.sa_data_by_5min.attributes.data | rejectattr('energy_consumed', 'equalto', None) | sum(attribute='energy_consumed') }}"
        icon_template: mdi:home-lightning-bolt-outline
        device_class: "energy"
        attribute_templates:
          state_class: total_increasing
      sa_todays_energy_generated_total:
        friendly_name: Total Energy Generated
        unit_of_measurement: "Wh"
        value_template: "{{ states.sensor.sa_data_by_5min.attributes.data | rejectattr('energy_generated', 'equalto', None) | sum(attribute='energy_generated') }}"
        icon_template: mdi:solar-power
        device_class: "energy"
        attribute_templates:
          state_class: total_increasing
      sa_todays_energy_imported:
        friendly_name: Total Energy Imported
        unit_of_measurement: "Wh"
        value_template: >
          {% set energy = namespace(imported=0) %}
          {% for sensor_data in states.sensor.sa_data_by_5min.attributes.data | rejectattr('energy_consumed', 'equalto', None) if sensor_data.energy_consumed > sensor_data.energy_generated %}
            {% set energy.imported = energy.imported + sensor_data.energy_consumed - sensor_data.energy_generated %}
          {% endfor %}
          {{ energy.imported }}
        icon_template: mdi:transmission-tower-export
        device_class: "energy"
        attribute_templates:
          state_class: total_increasing
      sa_todays_energy_exported:
        friendly_name: Total Energy Exported
        unit_of_measurement: "Wh"
        value_template: >
          {% set energy = namespace(exported=0) %}
          {% for sensor_data in states.sensor.sa_data_by_5min.attributes.data | rejectattr('energy_generated', 'equalto', None) if sensor_data.energy_generated > sensor_data.energy_consumed %}
            {% set energy.exported = energy.exported + sensor_data.energy_generated - sensor_data.energy_consumed %}
          {% endfor %}
          {{ energy.exported }}
        icon_template: mdi:transmission-tower-import
        device_class: "energy"
        attribute_templates:
          state_class: total_increasing

# -----------------------------------------------------------------------------

Yep. All good here now. It just took a couple of hours to start pulling the data from the new sensors (as expected).

Here are some sample HA Lovelace cards:

image

image

Nice work @PeterH24x7!

I have a couple of ideas for small improvements/simplifications to your config but maybe I’ll wait until you post your github repo and create some pull requests there instead.

For others that might want to do something similar, these are the links I used to learn more about Home Assistant’s YAML working:

Home Assistant Introduction to Energy Management

Energy Management in Home Assistant

Understanding Home Energy Management

Solar Analytics Developer Documentation

SA Public API Documentation

Solar Analytics API YAML

Template Designer Documentation

Jinja Documentation (3.0.x)

Smart Home Makers - on YouTube

Home Assistant Coding Tutorial APIs - Integrate any Device

Create your OWN Template Sensor with Home Assistant Coding Tutorial

START Leveraging Loops NOW - Home Assistant Coding Tutorial (Templating)

2 Likes

The solution is now on GitHub as follows:

https://github.com/PeterH24x7/-Solar-Analytics-integration-for-Home-Assistant-Energy-monitoring

Can’t say thanks enough for this Peter and Glen

Instructions are very easy to follow - simple enough for someone with little experience in HA or yaml to get going.

I think I get most of how the yaml works… can now fall back on annoying a software engineer relative for some support if needed - will be aiming to get readings for 3 additional clamps I have in my SA setup… one will be for my EV charging outlet charger and another for my aircon which I aim to automate.

Not sure if it’s a result of something else in my HA setup but thought I’d let you know that I have a warning when viewing in the HA Studio Code Server addon “String does not match the pattern of “Deprecated^”” in each of the entity ID lines in the following part of the yaml.

  - platform: template
    sensors:
      sa_dashboard_status:
        friendly_name: "Dashboard Status"
        value_template: "{{ state_attr('sensor.sa_status', 'dashboard_status') }}"
        entity_id: sensor.sa_status
      sa_mer_status:
        friendly_name: "System Status"
        value_template: "{{ state_attr('sensor.sa_status', 'mer_status') }}"
        entity_id: sensor.sa_status
      sa_mer_percentage: 
        friendly_name: "PV Performance"
        value_template: "{{ state_attr('sensor.sa_status', 'mer_percentage') }}"
        entity_id: sensor.sa_status
        unit_of_measurement: "%"

I’ll do a bit of reading on strings not matching pattern of “deprecated” this afternoon or tomorrow. But it doesn’t seem to impact on the function.

:+1:

this might be relevant?? The 'entity_id' option is deprecated, please remove it from your configuration - #44 by virgilm

Hi @ddwdiot

While the warning message is a bit obscure, I think the forum posting you gave is fairly self explanatory… entity_id is deprecated.
Have you tried removing the entity_id from your sensor to see if it avoids the warning?

Thanks @TheOtherGlen… I didn’t want to remove it on the assumption that it was part of the suggested code. Also that everything was working so well. But I thought it might worry others like me in the future.

Have now commented the relevant lines out and restarted and all is still good. Will delete the offending lines when I understand the entity_id deprecation issue better. Have been reading lots about YAML and HA but still at the stage where I have to continually revise.

I’ve commented out all the “entity_id” lines of code - representing legacy efforts to get the Energy function working. Doesn’t seem to have had any impact on the workings of the data and results.

1 Like