Sounds simple… Yes it would be nice to know, as the sun sinks below the horizon, just how accurate that forecast really was. I am working on this with a long-term database, but here is one solution. Hope this helps.
The forecast.solar integration in HA pulls in 48 hours of data covering the full period for today and tomorrow, ‘randomly’ each hour. The sensor energy_production today/tomorrow is the sum of each 24 hours. The base forecast data is updated every 15 mins, the HA integration entity every hour, and the ‘total today/tomorrow’ can change sometimes hourly (but not always) as a result.
See above my history for the forecast integration. My update is happening at 58 minutes past each hour (but it could happen at any point in the hour). Up to midnight, tomorrow is tomorrow, but sometime between 00:00 and 01:00 local time, tomorrow becomes today, thus to get “the forecast” for today requires obtaining ‘tomorrow’ from yesterday, and/or ‘today’ from today. Whilst ‘tomorrow’ changes during the day, it is always a forecast. ‘today’ also changes during the day, and therefore gradually becomes more of an historic record, so by the end of the day is it a “hindcast”.
Quite therefore what you pick out of that to be the ‘forecast’ is a bit of a guess. At the moment (over the past summer) I am finding for myself that the average of the first five hourly values for ‘today’ is the best resulting match to my actual value (at the end of the solar day). In the graph above, the orange line is the ‘tomorrow’, and the blue line is ‘today’, so if we look at the 3rd October, the blue ‘today’ averages around ‘3’ at the start of the day before it starts to change over the active solar day, and the orange ‘yesterday’ (over the 2nd October) averages about the same.
HA entity values are stored in history, and with a bit of effort past state values can be retrieved using the available HA API call. There is an SQL integration now, but even when asking for an array of values, this only returns on the most recent value.
https://developers.home-assistant.io/docs/api/rest/#get-apihistory
I use Node-RED for these things as I find it a) quicker b) easier and c) more powerful.
I have managed to put together up the following flow, which triggers after 22:00 each day, pulls in the last 48 hour records of the necessary entities, and processes the data. I have two planes (forecast.solar integrations) so it does this for both, and it does this for ‘today’ and for ‘tomorrow’ (four sensor entities in total).
I use JSONata to pick out today from today, and tomorrow from yesterday. I have opted here to pick only the first five records from today (pre sunrise) and for all from tomorrow-yesterday, averaged across all planes. This gives me the average forecast for today - from part of today - and the average forecast for today - from all of yesterday.
I then pick up the solar PV actual value (as pulled in from my inverter by Modbus read), and finally calculate a ratio, based on solar PV / forecast-today. The result can be passed back to HA in a sensor node (both state and attributes) and then HA will record the entity state/attribute values in history so it is easy to plot these values on an apex graph.
This is my current data. In the summer months the ratio (actual / forecast) was better, but this does suggest that my forecast is roughly a consistent 3 kWh too low. I am storing this data in a long term database, so once I have a full solstice to solstice dataset perhaps I can work out how to ‘adjust’ the forecast to better match the resulting actual.
Node-RED flow:
[{"id":"fe9e80e0ad137e77","type":"group","z":"95dab86a0bf42ced","name":"Calculate Solar Forecast to Actual Ratio","style":{"label":true},"nodes":["7a1df2413cd1c7ab","35296f1b09346067","cbf4985441778907","34585541521986ca","786207faf91b25ae","38e37a8043b02cc4"],"x":54,"y":959,"w":972,"h":122},{"id":"7a1df2413cd1c7ab","type":"api-get-history","z":"95dab86a0bf42ced","g":"fe9e80e0ad137e77","name":"forecast history","server":"","version":0,"startdate":"","enddate":"","entityid":"sensor.energy_production_today,sensor.energy_production_tomorrow,sensor.energy_production_today_2,sensor.energy_production_tomorrow_2","entityidtype":"is","useRelativeTime":true,"relativeTime":"48 hours","flatten":true,"output_type":"array","output_location_type":"msg","output_location":"payload","x":180,"y":1040,"wires":[["38e37a8043b02cc4"]]},{"id":"35296f1b09346067","type":"inject","z":"95dab86a0bf42ced","g":"fe9e80e0ad137e77","name":"At 22:15","props":[],"repeat":"","crontab":"15 22 * * *","once":false,"onceDelay":0.1,"topic":"","x":160,"y":1000,"wires":[["7a1df2413cd1c7ab"]]},{"id":"cbf4985441778907","type":"api-current-state","z":"95dab86a0bf42ced","g":"fe9e80e0ad137e77","name":"Solar Today","server":"","version":3,"outputs":1,"halt_if":"","halt_if_type":"str","halt_if_compare":"is","entity_id":"sensor.mb_solar_e_today","state_type":"num","blockInputOverrides":false,"outputProperties":[{"property":"payload.solar","propertyType":"msg","value":"","valueType":"entityState"}],"for":"0","forType":"num","forUnits":"minutes","override_topic":false,"state_location":"payload","override_payload":"msg","entity_location":"data","override_data":"msg","x":530,"y":1040,"wires":[["34585541521986ca"]]},{"id":"34585541521986ca","type":"change","z":"95dab86a0bf42ced","g":"fe9e80e0ad137e77","name":"Add ratio and save","rules":[{"t":"set","p":"payload.ratio","pt":"msg","to":"$round(payload.solar/payload.fcToday,2)","tot":"jsonata"},{"t":"set","p":"solar","pt":"flow","to":"payload","tot":"msg","dc":true}],"action":"","property":"","from":"","to":"","reg":false,"x":710,"y":1040,"wires":[["786207faf91b25ae"]]},{"id":"786207faf91b25ae","type":"ha-sensor","z":"95dab86a0bf42ced","g":"fe9e80e0ad137e77","name":"Solar v Forecast","entityConfig":"3ab8194663a50d4b","version":0,"state":"payload.ratio","stateType":"msg","attributes":[{"property":"date","value":"payload.date","valueType":"msg"},{"property":"solar","value":"payload.solar","valueType":"msg"},{"property":"fctoday","value":"payload.fcToday","valueType":"msg"},{"property":"fcyesterday","value":"payload.fcYesterday","valueType":"msg"}],"inputOverride":"allow","outputProperties":[],"x":920,"y":1040,"wires":[[]],"server":""},{"id":"38e37a8043b02cc4","type":"change","z":"95dab86a0bf42ced","g":"fe9e80e0ad137e77","name":"Get FC figures","rules":[{"t":"set","p":"payload","pt":"msg","to":"(\t\t/* FUNCTION: extract individual plane and day from returned 48 hour history, uses entity_id */\t/* to look for given plane [_\"1\" or _\"2\" etc] and day [_\"today\" or \"tomorrow\"] */\t $pday:= function($plane, $day){(\t $x:=$array[$contains(entity_id, $plane) and $contains(entity_id, $day)];\t $x.{\"time\": last_updated, \"uts\": $toMillis(last_updated), \"value\": state}^(>time)\t )};\t\t/* have to deal with possible multiple integrations, so assume there are two or more */\t/* look at entity 'sensor.energy_production_today' and where there is no number suffix, append \"_1\" */\t/* for each plane we have, and for both 'today' and 'tomorrow', built table to search */\t $array:=payload~>|$|{\"entity_id\": $count($split(entity_id,\"_\"))<4 ? entity_id & \"_1\"}|;\t $table:=$distinct($array.entity_id).$substringAfter(\"production_\");\t $results:=$table.(\t $x:=$split($,\"_\");\t {\"plane\": $x[1], \"day\": $x[0], \"forecast\": $pday($x[1], $x[0])}\t );\t\t/* need number of planes for later */\t $planes:=$count($distinct($results.plane));\t\t/* get last & previous midnight UTC as millisec and use to pick today (from today) and tomorrow (from yesterday) */\t $timestamp:=$results[0].forecast[0].time;\t $midnight:=$toMillis($substringBefore($timestamp,\"T\") & \"T\" & \"00:00:00\");\t $daybefore:=$midnight-24*60*60*1000;\t\t/* forecast for today: get both 'today' today and 'tomorrow' yesterday */\t $today:=($results[day=\"today\"].forecast)^(uts);\t $tomorrow:=$results[day=\"tomorrow\"].forecast^(uts);\t $today_today:=$today[uts>$midnight]^(time);\t $tomorrow_yesterday:=$tomorrow[uts>$daybefore and uts<$midnight]^(time);\t\t/* decide what we want to get a figure for 'forecast' across today-today and tomorrow-yesterday */\t/* for myself, pick first 5 from today (only), average and summ (across all planes) */\t/* and also take whole of yesterday (tomorrow) average and sum for all planes */\t\t $pick:=5;\t $range:=[0..$pick*$planes-1];\t\t $fctd:=$today_today[$range].$number(value);\t $fcyd:=$tomorrow_yesterday.$number(value);\t $fcToday:=$round($sum($fctd)*$planes/$count($fctd),1);\t $fcYsday:=$round($sum($fcyd)*$planes/$count($fcyd),1);\t {\"date\": $substringBefore($timestamp,\"T\") ,\"fcToday\": $fcToday, \"fcYesterday\": $fcYsday};\t\t\t)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":360,"y":1040,"wires":[["cbf4985441778907"]]},{"id":"3ab8194663a50d4b","type":"ha-entity-config","server":"","deviceConfig":"","name":"SC HA2 Solar v Forecast","version":"6","entityType":"sensor","haConfig":[{"property":"name","value":"Solar v Forecast"},{"property":"icon","value":""},{"property":"entity_category","value":""},{"property":"entity_picture","value":""},{"property":"device_class","value":""},{"property":"unit_of_measurement","value":""},{"property":"state_class","value":""}],"resend":true,"debugEnabled":false}]
Usual rules apply. Has been scrubbed (so add in your own HA server), set up for two integrations (so remove second if you don’t need), change entity name for solar PV actual to match your own. Code should work out for itself how many integrations you have but this is only tested for my setup.
Should manage to work over DST changes but as yet untested. Deals with the current (at the time of writing) websocket API History node ‘include’ error by listing the entities as a string - modify this as required.
Apex Chart config used
type: custom:apexcharts-card
graph_span: 7d
span:
end: day
offset: '-1d'
header:
show: true
title: Solar Actual Vs Forecast
show_states: true
colorize_states: true
yaxis:
- id: ratio
min: 0.5
max: 4
decimals: 1
apex_config:
tickAmount: 7
- id: energy
min: 0
max: 25
decimals: 0
apex_config:
tickAmount: 5
series:
- entity: sensor.solar_v_forecast
yaxis_id: ratio
type: line
stroke_width: 3
name: Ratio
- entity: sensor.solar_v_forecast
yaxis_id: energy
type: line
stroke_width: 3
attribute: solar
name: Solar PV
- entity: sensor.solar_v_forecast
yaxis_id: energy
type: line
stroke_width: 3
attribute: fctoday
name: Forecast
Errors and omissions excepted.
this should fit your needs better than forecast.solar: