@james_hiscott You’re welcome. My main goal is automation, so the actual figures are enough for me. Maybe you will find more help in this mentioned topic.
Hi @climb - thank you so much for doing this work and making it available. It solves a big problem for me because my my time zone has the solar production period cut up by the UTC day change.
It also looks like this could be good for me to use for my split system. I think I can duplicate and edit the code to work, but wanted to check that the items I’d need to change in the duplicated code are:
the solcast_forecast_data (e.g. solecast_forecast_data_2) in its definition and when called in the code
the SOLCAST_RESOURCE_ID
the name of each of the sensors (e.g. solcast_forecast_average_30min becomes solcast_forecast_average_30min_2)
Is that it? Or do I also need to change some of what look like internal working variables like forecast, forecast.pv_estimate, ns.fc_today etc.
For those that would be interested to create a forecast graph together with the actual production, you can use the code below. It requires the custom component apexcharts-card.
The code below will generate a curve (orange) showing your actual production (in my case coming from sensor.inverter_gw5000_eh, replace this with your own solar production sensor) and will plot the forecast data extracted from sensor.solcast_forecast_data (red). I also plot the history of the forecast data (blue) to see how good the forecasts track my actuals. I find having this graph interesting to quickly glance to decide if it is a good idea to start the dishwasher or other appliances that are not yet fully automated.
type: custom:apexcharts-card
graph_span: 36h
span:
start: day
offset: '-6h'
header:
show: true
title: Solar Production vs. forecast
show_states: true
now:
show: true
label: now
apex_config:
legend:
show: false
series:
- entity: sensor.inverter_gw5000_eh
name: Actual
unit: W
fill_raw: last
extend_to_end: false
group_by:
func: avg
duration: 30min
- entity: sensor.solcast_forecast_average_30min
transform: return x * 1000;
name: Forecast
unit: W
fill_raw: last
extend_to_end: false
- entity: sensor.solcast_forecast_data
type: line
extend_to_end: false
unit: W
show:
in_header: false
data_generator: |
return entity.attributes.forecasts.map((entry) => {
return [new Date(entry.period_end), entry.pv_estimate*1000];
});
The Forecast.solar integration is useless if you live below the equator. North America and Europe only.
As Australia has the highest number of rooftop solar installtions per capita, it would be nice to see Solcast available as an integration that works with the Energy Panel.
Beyond my capabilities, though.
But I have modified the above slightly to use the new template format in HA.
I’ve limited the API call to the next 72 hours, added the next 30 min estimate as the state of the REST sensor and added some device_class niceties:
sensor:
- platform: rest
name: "Solcast Forecast Data"
json_attributes:
- forecasts
resource: https://api.solcast.com.au/rooftop_sites/SOLCAST_RESOURCE_ID/forecasts?format=json&api_key=SOLCAST_API_KEY&hours=72
method: GET
value_template: "{{ value_json.forecasts[0].pv_estimate|round(2) }}"
unit_of_measurement: "kW"
device_class: power
scan_interval: 00:30
force_update: true
template:
- sensor:
- name: "Solcast Forecast Energy Today"
unique_id: solcast_forecast_energy_today
unit_of_measurement: "kWh"
device_class: energy
state: >
{% set ns = namespace (fc_today = 0) %}
{% for forecast in state_attr('sensor.solcast_forecast_data', 'forecasts')|default('variable is not defined') %}
{% set daydiff = as_local(strptime(forecast.period_end, '%Y-%m-%dT%H:%M:%S.%f0Z').replace(tzinfo=utcnow().tzinfo)).date() - as_local(utcnow()).date() %}
{% if daydiff.days == 0 %}
{% set ns.fc_today = ns.fc_today + (forecast.pv_estimate/2)|float %}
{%- endif %}
{%- endfor %}
{{ ns.fc_today|round(2) }}
- name: "Solcast Forecast Energy Tomorrow"
unique_id: solcast_forecast_energy_tomorrow
unit_of_measurement: "kWh"
device_class: energy
state: >
{% set ns = namespace (fc_tommorrow = 0) %}
{% for forecast in state_attr('sensor.solcast_forecast_data', 'forecasts')|default('variable is not defined') %}
{% set daydiff = as_local(strptime(forecast.period_end, '%Y-%m-%dT%H:%M:%S.%f0Z').replace(tzinfo=utcnow().tzinfo)).date() - as_local(utcnow()).date() %}
{% if daydiff.days == 1 %}
{% set ns.fc_tommorrow = ns.fc_tommorrow + (forecast.pv_estimate/2)|float %}
{%- endif %}
{%- endfor %}
{{ ns.fc_tommorrow|round(2) }}
- name: "Solcast Forecast Peak Power Today"
unique_id: solcast_forecast_peak_power_today
unit_of_measurement: "kW"
device_class: power
state_class: measurement
state: >
{% set ns = namespace (fc_today_max = 0) %}
{% for forecast in state_attr('sensor.solcast_forecast_data', 'forecasts')|default('variable is not defined') %}
{% set daydiff = as_local(strptime(forecast.period_end, '%Y-%m-%dT%H:%M:%S.%f0Z').replace(tzinfo=utcnow().tzinfo)).date() - as_local(utcnow()).date() %}
{% if daydiff.days == 0 %}
{% if ns.fc_today_max < forecast.pv_estimate|float %}
{% set ns.fc_today_max = forecast.pv_estimate|float %}
{%- endif %}
{%- endif %}
{%- endfor %}
{{ ns.fc_today_max|round(2) }}
- name: "Solcast Forecast Peak Power Tomorrow"
unique_id: solcast_forecast_peak_power_tomorrow
unit_of_measurement: "kW"
device_class: power
state_class: measurement
state: >
{% set ns = namespace (fc_tomorrow_max = 0) %}
{% for forecast in state_attr('sensor.solcast_forecast_data', 'forecasts')|default('variable is not defined') %}
{% set daydiff = as_local(strptime(forecast.period_end, '%Y-%m-%dT%H:%M:%S.%f0Z').replace(tzinfo=utcnow().tzinfo)).date() - as_local(utcnow()).date() %}
{% if daydiff.days == 1 %}
{% if ns.fc_tomorrow_max < forecast.pv_estimate|float %}
{% set ns.fc_tomorrow_max = forecast.pv_estimate|float %}
{%- endif %}
{%- endif %}
{%- endfor %}
{{ ns.fc_tomorrow_max|round(2) }}
- name: "Solcast Forecast Peak Time Today"
unique_id: solcast_forecast_peak_time_today
state: >
{% set ns = namespace (fc_today_max = 0, fc_today_max_time = 0) %}
{% for forecast in state_attr('sensor.solcast_forecast_data', 'forecasts')|default('variable is not defined') %}
{% set daydiff = as_local(strptime(forecast.period_end, '%Y-%m-%dT%H:%M:%S.%f0Z').replace(tzinfo=utcnow().tzinfo)).date() - as_local(utcnow()).date() %}
{% if daydiff.days == 0 %}
{% if ns.fc_today_max < forecast.pv_estimate|float %}
{% set ns.fc_today_max = forecast.pv_estimate|float %}
{% set ns.fc_today_max_time = as_local(strptime(forecast.period_end, '%Y-%m-%dT%H:%M:%S.%f0Z').replace(tzinfo=utcnow().tzinfo)).time() %}
{%- endif %}
{%- endif %}
{%- endfor %}
{{ ns.fc_today_max_time }}
- name: "Solcast Forecast Peak Time Tomorrow"
unique_id: solcast_forecast_peak_time_tomorrow
state: >
{% set ns = namespace (fc_tomorrow_max = 0, fc_tomorrow_max_time = 0) %}
{% for forecast in state_attr('sensor.solcast_forecast_data', 'forecasts')|default('variable is not defined') %}
{% set daydiff = as_local(strptime(forecast.period_end, '%Y-%m-%dT%H:%M:%S.%f0Z').replace(tzinfo=utcnow().tzinfo)).date() - as_local(utcnow()).date() %}
{% if daydiff.days == 1 %}
{% if ns.fc_tomorrow_max < forecast.pv_estimate|float %}
{% set ns.fc_tomorrow_max = forecast.pv_estimate|float %}
{% set ns.fc_tomorrow_max_time = as_local(strptime(forecast.period_end, '%Y-%m-%dT%H:%M:%S.%f0Z').replace(tzinfo=utcnow().tzinfo)).time() %}
{%- endif %}
{%- endif %}
{%- endfor %}
{{ ns.fc_tomorrow_max_time }}
- name: "Solcast Forecast Power Next Hour"
unique_id: solcast_forecast_power_next_hour
state: >
{{ ((state_attr('sensor.solcast_forecast_data', 'forecasts')[0].pv_estimate|default('variable is not defined') + state_attr('sensor.solcast_forecast_data', 'forecasts')[1].pv_estimate|default('variable is not defined'))/2)|round(2) }}
unit_of_measurement: 'kW'
device_class: power
Beyond anyone’s at the moment. The PVGIS data set that Forecast Solar uses does not cover the area. I contacted PVGIS and got this response:
we are working on updating the various datasets of PVGIS to include data up to 2020 and also to extend the spatial coverage.
It would be great to be able to provide sastellite based solar radiation data for Australia and New Zealand, but for the time being, that area is not among our priorities. However, we intend to provide data from the reanalysis product ERA5 that would cover that part of the world.
We would like to have this update ready by the end of this year.
So there is the possibility of hourly forecasts soon.
2021.9 Adds support for alternative Solar Forecasts, (commit: 54576) its now just a matter of integrating the Solcast API to use the documented energy forecast panel integration.
Using the apex chart config you have kindly provided, every morning when the new day’s data comes in my forecast data graph goes a bit wrong. There’s no way I’m going to generate power at midnight, and the data from the day before certainly did not predict that.
Do you have any idea why this happens?
I was thinking it might be a timezone thing as the forecast data is in UTC (https://hatebin.com/aogimlkrfj) but that cant be it as the actual and forecast data for the rest of the day lines up.
type: custom:apexcharts-card
graph_span: 36h
span:
start: day
offset: '-6h'
header:
show: true
title: Solar Forecast
show_states: true
now:
show: true
label: now
apex_config:
legend:
show: true
series:
- entity: sensor.sma_inverter_power
name: Actual
unit: W
fill_raw: last
color: '#0da035'
stroke_width: 2
extend_to_end: false
group_by:
func: avg
duration: 30min
show:
legend_value: false
- entity: sensor.solcast_forecast_average_30min
transform: return x * 1000;
name: Past Forecast
unit: W
fill_raw: last
color: '#e0b400'
stroke_width: 2
extend_to_end: false
show:
legend_value: false
- entity: sensor.solcast_forecast_data
type: line
name: Future Forecast
extend_to_end: false
stroke_width: 2
unit: W
show:
in_header: false
legend_value: false
data_generator: |
return entity.attributes.forecasts.map((entry) => {
return [new Date(entry.period_end), entry.pv_estimate*1000];
});
Edit: I think I’ve worked it out. This sensor (for the past data) does not update while the value is a t 0 all night:
- name: solcast_forecast_average_30min
unit_of_measurement: "kW"
device_class: power
state: >
{{ state_attr('sensor.solcast_forecast_data', 'forecasts')[0].pv_estimate|default('variable is not defined')|round(2) }}
So the graph draws a curve from the last data point. Setting the graph to step line shows this:
So I think I need to force the template sensor for the 30min average to update, maybe by including now() like this:
- name: solcast_forecast_average_30min
unit_of_measurement: "kW"
device_class: power
state: >
{% set force_update = now() %}
{{ state_attr('sensor.solcast_forecast_data', 'forecasts')[0].pv_estimate|default('variable is not defined')|round(2) }}
If that does not work I’ll try adding a random number in the 0.1 to 0.2W range every 1 minute.
I have the same behavior. The apex chart card does some kind of low pass filtering on the data to generate smoother plots. If you compare the apex-chart with your actual history of your production, you will see a lot more peaks in the raw data. Whatever smoothing apex-chart card does together with the fact that the sensor data values are not stored when the forecasts stays at 0 during the night, results in that graph you are seeing. I haven’t looked into that, but indeed your solution to either adding a random number could work or we could check with the developer of apex-chart if there are other smoothing functions we can try?
The default for the apexcharts curve option is smooth which draws a curve between the points. Not ideal when your last point was 12 hours ago. The other options are straight (direct line between points) or stepline (flat line until next point then straight up or down).
Only the last option will not introduce the issue but it’s not the best looking.
I think if the template sensor is forced to update every minute like I have done with now() in the template above it should fix the issue. As there will be a point every 60 seconds even when the value is stuck at 0 for the whole night. I’ll let you know if it works in 12 hours or so.
I will try the adding of a small random number if 0.
state: >
{% set force_update = now() %}
{% if state_attr('sensor.solcast_forecast_data', 'forecasts')[0].pv_estimate|default('variable is not defined')|round(2) == 0 %}
{{ range(-100,100)|random / 100000 }}
{% else %}
{{ state_attr('sensor.solcast_forecast_data', 'forecasts')[0].pv_estimate|default('variable is not defined')|round(2) }}
{% endif %}
The other issue is that the simple integrations for for the today and tomorrow energy forecast sensors are often very inaccurate. Like +/-200% inaccurate - even when my actual power follows the forecast power graph closely.
I can see what the template is trying to do, it’s a half hour power prediction, so divide the power by two and add it to the total energy. But when I do this manually using the forecast power data visible in the graph I get a much more accurate answer (±10%). As the power data seems correct I’m assuming it is the date range selection that is at fault. Not sure why you need to replace the TZ info and the predictions appear to be in UTC anyway ( period_end: '2021-09-09T23:30:00.0000000Z' ), and you seem to be comparing it to the local date.
state: >
{% set ns = namespace (fc_tommorrow = 0) %}
{% for forecast in state_attr('sensor.solcast_forecast_data', 'forecasts')|default('variable is not defined') %}
{% set daydiff = as_local(strptime(forecast.period_end, '%Y-%m-%dT%H:%M:%S.%f0Z').replace(tzinfo=utcnow().tzinfo)).date() - as_local(utcnow()).date() %}
{% if daydiff.days == 1 %}
{% set ns.fc_tommorrow = ns.fc_tommorrow + (forecast.pv_estimate/2)|float %}
{%- endif %}
{%- endfor %}
{{ ns.fc_tommorrow|round(2) }}
