Thanks a lot! This is very helpful. I only have one remaining question. I didn’t quite get how you setup “sensor.solar_energy_hr30”. I have also setup a sensor which is a Riemann Sum on my inverter power and I could setup a utility sensor with hourly reset. But how can I make sure that it actually shows the energy over the last hour? If it resets mid-hour than depending on when the flow actually runs, the utility sensor only displays the energy generated since the last reset, doesn’t it? It would be great if you could share some more insights on how you calculate the sensor. Thank you!
Yup I just knew you were going to ask that one sooner or later.
So, the solar forecast is power, measured at the hour, and I plot this on my graph - power level, each hour.
To get energy from the forecast power, I can get the average power, measured across one hour, which then becomes watt-hours. Eg 9:00 power 800 W, 10:00 power 1200 W, simple average is 1000 W, so average 1000 Watts for one hour = 1000Wh. Now, this value should be plotted at 9:30 as it represents the period 9-10, not plotted at the hour 9:00 or hour 10:00. But I am plotting (power) at the hour and I don’t want to plot half way between. So, I decided to take the power at 9:30, the power at 10:30, and average them to get the energy between 9:30 -10:30, and plot this energy value at 10:00. The power at 9:30 is the average power between 9 and 10, and for 10:30 between 10 and 11. Thus the power 9:30 to 10:30 is ( power(9:00) + 2*power(10:00) + power(11:00) ) / 4
This gives the forecast average power, for the half hour to half hour period, which can be plotted at the hour.
Now I also want the ‘actual’ as power in watt-hours, for the period 9:30 to 10:30, to plot at 10:00 in the same way.
I have:
Inverter power sensor → inverter energy sensor using Riemann Sum.
I added:
Utility meter, based on the inverter energy, which has to be set up in the configuration file, as hourly with an offset of 30 minutes. The helper Utility Meter cannot cope with offsets.
This utility meter now records the energy between, for example, 9:30 and 10:30, and resets to zero at 10:30.
If, at around 11:10 I read this meter, it returns in the entity state the current energy between 10:30 and 11:10, but in the payload.attributes it has the value ‘last_period’ which happens to be the total energy for the period ending at the last reset, or for 9:30 to 10:30. I grab this, and push it to the array for the last_reset hour, being 10:00 (from 10:30 reset).
Yes, it only works correctly for an hourly update between hour:01 and hour:29.
Correction: works always. If, at around 11:50 I read this meter, it returns energy 10:30 to 11:30 in the last_period, and last_reset is 11:30 so hour is 11:00. Posts 10:30 to 11:30 energy at 11:00.
I use a different type of node to trigger, and set this to run at hour:04. However, being community minded I am aware that if everyone wanted their flow to update at hour:04 the ‘free’ API call would not cope. In the HA ForecastSolar integration, the update is ‘random’, thus preserving an even spread for demand on the API. I was just mindful when publishing this not to encourage bunching, but clearly that is up to the end users.
This all seems to work - I get the power forecast, which changes during the day, a history record of the power forecast now in the past, an energy forecast, and the actual energy. I have a really good match (on full sun days) with all four plots for the afternoon, but a difference in the morning, from my East facing panels. I am still working on why - some of this is my panel orientation and placement (being North of East) but I still think the forecast is out.
And yes, I know the colours on my Node-RED dashboard graph don’t match those on my HA Apex Charts graph. I have been working on this stuff for months!
2 Likes
Thanks again! I did get it to work. I now just have to extend it to 4 planes. 
Hopefully, 4 additional API calls per hour together with the forecast.solar integration will not exceed the limit of 12 calls per hour. Do you know by any chance what the number of API calls is that the integrated forecast.solar integration triggers per hour?
The integration does one call per integration per hour. This is at the ‘random’ time of the update. Current allowance is 12 per hour per IP address. I don’t know if it is a rolling hour or fixed hour. With two planes, two integrations, and two of my own API calls, I consume 4/12 per hour so room for the odd testing call as well.
The full return from the API call sends back location as well as how many API calls you have left. During testing I have checked this number and it seems reasonably accurate, so it could be used to manage or track useage.
1 Like
Everything is working perfectly right now. Incl. persistant storage in HA and the 4 planes. I only have one small question left: How can I access the values stored in sensor.fc_table in HA? I see that the values are displayed in the attributes of the sensor and are separated by commas, but is there any way to access them directly, let’s say to use the values in an apexcharts card?
Edit: Answered this myself. It’s an array so that I can easily access the values in the apexcharts data generator. Perfect!
I bought HA to use it to monitor and control my Solar PV system. The feature that ‘sold’ it to me was the integrated Solar Forecast - however I have found it to be not entirely accurate and of little practical use as we can’t get at the data.
The big question for Solar Forecasting is not ‘how to get it’ but ‘what to do with it once you have it’. I was inspired by
https://community.home-assistant.io/t/forecast-solar-predict-when-output-power-will-be-above-predefined-level/424348
to start working on Solar Forecast, with a view to answering this very question - to decide when output power will be above a predefined level.
Getting the API calls to work was the easy bit. Next was a graph, and that is what sensor.fc_table provides. As you have answered - this holds all the values in arrays in the sensor attributes and therefore was designed to run almost directly into apex charts. I had not used apex charts before, but it was not that difficult once I got my head around the data_generator bit. Still don’t know quite how it works, but it does.
type: custom:apexcharts-card
graph_span: 3d
span:
end: day
offset: +1d
header:
show: true
title: 'Solar Forecast: yesterday - today - tomorrow'
now:
show: true
label: now
show:
last_updated: true
series:
- entity: sensor.fc_table
data_generator: |
return entity.attributes.fchours.map((fchr, index) => {
return [new Date(fchr).getTime(), entity.attributes.fcwatts[index]];
});
curve: smooth
name: Forecast
show:
in_header: false
legend_value: false
stroke_width: 2
color: orange
- entity: sensor.fc_table
data_generator: |
return entity.attributes.fchours.map((fchr, index) => {
return [new Date(fchr).getTime(), entity.attributes.fcold[index]];
});
curve: smooth
name: History
show:
in_header: false
legend_value: false
stroke_width: 2
color: magenta
- entity: sensor.fc_table
data_generator: |
return entity.attributes.fchours.map((fchr, index) => {
return [new Date(fchr).getTime(), entity.attributes.fcactual[index]];
});
curve: smooth
name: Actual
show:
in_header: false
legend_value: false
stroke_width: 2
color: blue
- entity: sensor.fc_table
data_generator: |
return entity.attributes.fchours.map((fchr, index) => {
return [new Date(fchr).getTime(), entity.attributes.fcwh[index]];
});
curve: smooth
name: Energy
show:
in_header: false
legend_value: false
stroke_width: 2
color: green
My next objective is to get the time to start an appliance given a required power level. This is where FC energy estimate sensor comes in. The state value is my estimate of the total daily energy, and the attributes hold the power table as well as the hour of start, stop, and maximum. If there are humps in the day there will also be one or more minimums.
I have got as far as being able to provide in HA an array of events for a given power level, and even sorting this for the largest duration if there are more than one - this returns ‘start’ and ‘stop’ as hh:mm, and duration in minutes. I am trying to extend this to working out, for a given power, earliest start time, and minimum duration, a return that says when the period starts, when to switch on the device, and how long you have left. Doing the unpicking of the power array in HA has defeated me, so I am now looking at using Node-RED to do the work, based on an API call from HA back to Node-RED with the query parameters - power, start, duration, with a return object containing switching states and times. Still work in progress, and I am looking to first fine-tune my forecast to better match my actual.
Hope you find the code of use!
2 Likes
All your input is very helpful! Many thanks for this!
Hi, sorry to drop in on a seemingly solved matter.
But can’t we grab the already existing data from somewhere in the database?
Anyone know?
I don’t know the database structure that well. But this dashed curve is “plotted” from info that must be stored locally somewhere.
Just a thought.
My goal is to simply get the forecast energy (kWh) for the remainder of the day. (from now until sunset)
The marked solution is an alternate way to solve this by making additionnal calls to the Forecast.Solar API, so it may not be optimal.
What you are saying is exactly the original issue on this thread.
There is no known solution to directly “grab” that dashed ploted data.
If you find it please share for others…
I’ll grab the database from a backup file, and dissect it to try to find it there.
…
[EDIT] Ok, after some digging I don’t think this is stored in the HA database.
Maybe this is stored in RAM afterall.
The Energy settings file also doesn’t help (.storage/energy)
"data": {
"energy_sources": [
(...)
{
"type": "solar",
"stat_energy_from": "sensor.[YOUR_ENERGY_SENSOR]",
"config_entry_solar_forecast": [
"[long identifier found in the core.config_entry file]"
]
},
(...)
I have posted in WTH: WTH can’t we access all data from solar forecast as in energy dashboard. You can allways vote for it to be solved.
Mean while I have a workaround where you actually access the data from HA so there is no need for extra calls to the Forecast.Solar API. The data is only accessible through the web-socket API.
(I rushed this together this afternoon so I can’t guarantee it will work for everyone, and it most certainly contains some bugs.)
It requires Node Red and node-red-contrib-home-assistant-websocket. Here is a flow you can import:
[{"id":"022a95a855f03d7e","type":"ha-api","z":"4d1a86a4c406178b","name":"","server":"993dd4ef.f733c8","version":1,"debugenabled":false,"protocol":"websocket","method":"get","path":"","data":"{\"type\":\"energy/solar_forecast\"}","dataType":"json","responseType":"json","outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"results"}],"x":350,"y":180,"wires":[["3855bddf.df6552"]]},{"id":"3855bddf.df6552","type":"function","z":"4d1a86a4c406178b","name":"Calculate","func":"// Fix values from HA forecast integration\nlet id = Object.keys(msg.payload)[0];\nconst whHours = msg.payload[id].wh_hours;\nvar newTable = {};\n\nObject.keys(whHours).forEach(key => {\n var t = new Date(key);\n if (whHours[key] == 0) {\n t.setMinutes(0,0,0);\n newTable[t.getTime()] = whHours[key];\n } else if (t.getMinutes() > 0) {\n t.setHours(t.getHours()+1,0,0,0);\n newTable[t.getTime()] = whHours[key];\n } else {\n newTable[t.getTime()] = whHours[key];\n }\n});\n\n// Time zone fixes\nlet firstKey = Object.keys(whHours)[0];\nconst tzOffset = firstKey.slice(-6);\nvar tzSeconds = parseInt(tzOffset.slice(1,3)*60*60 + tzOffset.slice(-2)*60)\nif (tzOffset.slice(0,1) === \"-\") {\n tzSeconds = -tzSeconds\n}\n\n// Create 48 hours array\nconst today = new Date();\nconst tomorrow = new Date();\ntomorrow.setDate(tomorrow.getDate() + 1);\n\nvar todayHours = [];\nvar tomorrowHours = [];\nfor (var i = 0; i < 24; i++) {\n today.setHours(i,0,0,0);\n todayHours.push(today.getTime());\n tomorrow.setHours(i,0,0,0);\n tomorrowHours.push(tomorrow.getTime());\n}\nallHours = todayHours.concat(tomorrowHours)\n\n// Create forecast table\nvar solarForecast = [];\nconst now = new Date(msg.timestamp + tzSeconds*1000);\nvar this_hour = null;\nvar next_hour = null;\n\nallHours.forEach(hour => {\n startDate = new Date(hour + tzSeconds*1000);\n stopDate = new Date(hour + tzSeconds*1000);\n stopDate.setHours(stopDate.getHours() + 1);\n if (msg.timestamp > hour) {\n this_hour = newTable[hour];\n next_hour = newTable[hour+3600000];\n }\n startStr = startDate.toISOString().slice(0,19) + tzOffset;\n stopStr = stopDate.toISOString().slice(0,19) + tzOffset;\n if (hour in newTable) {\n solarForecast.push({\n \"start\": startStr,\n \"end\": stopStr,\n \"value\": newTable[hour]\n });\n } else {\n solarForecast.push({\n \"start\": startStr,\n \"end\": stopStr,\n \"value\": 0\n });\n }\n});\n\nmsg.solar_forecast = solarForecast;\nmsg.this_hour = this_hour\nmsg.next_hour = next_hour\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":480,"y":180,"wires":[["6c9647caa4b673a9"]]},{"id":"6c9647caa4b673a9","type":"ha-entity","z":"4d1a86a4c406178b","name":"Solar PV Forecast","server":"993dd4ef.f733c8","version":2,"debugenabled":false,"outputs":1,"entityType":"sensor","config":[{"property":"name","value":"Solar PV Forecast"},{"property":"device_class","value":"energy"},{"property":"icon","value":"mdi:solar-power-variant"},{"property":"unit_of_measurement","value":"Wh"},{"property":"state_class","value":"measurement"},{"property":"last_reset","value":""}],"state":"this_hour","stateType":"msg","attributes":[{"property":"full_forecast","value":"solar_forecast","valueType":"msg"},{"property":"next_hour","value":"next_hour","valueType":"msg"}],"resend":true,"outputLocation":"payload","outputLocationType":"none","inputOverride":"allow","outputOnStateChange":false,"outputPayload":"","outputPayloadType":"str","x":650,"y":180,"wires":[[]]},{"id":"f3b80fbf412bc248","type":"server-state-changed","z":"4d1a86a4c406178b","name":"Forecast.Solar","server":"993dd4ef.f733c8","version":4,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"sensor.energy_next_hour","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"","halt_if_type":"str","halt_if_compare":"is","outputs":1,"output_only_on_state_change":false,"for":"0","forType":"num","forUnits":"minutes","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"topic","propertyType":"msg","value":"forecast_uppdated","valueType":"str"},{"property":"timestamp","propertyType":"msg","value":"","valueType":"date"}],"x":180,"y":200,"wires":[["022a95a855f03d7e"]]},{"id":"9e794fb98f28d5e8","type":"inject","z":"4d1a86a4c406178b","name":"Every hour","props":[{"p":"topic","vt":"str"},{"p":"timestamp","v":"","vt":"date"}],"repeat":"3600","crontab":"","once":true,"onceDelay":0.1,"topic":"start","x":190,"y":160,"wires":[["022a95a855f03d7e"]]},{"id":"993dd4ef.f733c8","type":"server","name":"Home Assistant","version":4,"addon":false,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":30,"areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m"}]
It will create a new sensor named sensor.solar_pv_forecast. The state of the sensor is this hours predicted production (the same as Estimated Energy Production - This Hour (in kWh) with the Forecast.Solar integration). But it also has an attribute named full_forecast which contains a list of all hours prediction for this day and the next day. So in total 48 hours prediction or forecast.
Here is an exampel how to visualize it useing a apexcharts-card:
type: custom:apexcharts-card
header:
show: false
show_states: true
title: Solar forecast
now:
show: true
label: Now
graph_span: 2d
span:
start: day
series:
- entity: sensor.solar_pv_forecast
data_generator: |
return entity.attributes.full_forecast.map((entry) => {
return [new Date(entry.start), entry.value];
});
offset: '-30min'
name: Forecast
type: column
color: darkgreen
opacity: 0.4
stroke_width: 1
show:
in_header: false
legend_value: false
3 Likes
Hi Biscuit,
may I ask for the bits of the configuration.yaml regarding the sensor.solar_energy_hr30. Mine looks like this:
utility_meter:
energy:
unique_id: solar_energy_hr30
name: solar_energy_hr30
source: sensor.inverter_power #inverter power AC [W]
cycle: hourly
offset: '00:30'
I assume, the inverter energy Sensor using Riemann Sum has been set up via the UI? What input from the inverter did you choose here? I use a Hoymiles inverter and I used the “Power AC” [W] at the moment. Correct so far? My sensor.solar_energy_hr30 is working fine. Sums up the Watts and resets. I checked the plot in the HA history. But nothing is displayed in the Apex chart. Yesterday it showed the actual plot, but today nothing, without having touched anything. API call is set up correctly, sensor.table_fc is filled with forecast data (at fcwatts).
I would really appreciate any tips.
Greetings, Björn
EDIT: messed up the timezone stuff in the “Date/Time formatter” node. Switched to a different style, as I’m in DE. Now I can see the forecast plot. Think this is solved. On my own 
Hi.
I can try and put my brain into gear!
I think your utility yaml is good to go, but I think you are using power and should be using energy as the input.
OK, so the (solar energy) utility meter I wanted resets hourly but at 30 minutes past the hour, which cannot be set up using the helpers, so I have done this by hand in yaml.
In my yaml this is (sensor called ‘solar_energy_hr30’)
utility_meter:
solar_energy_hr30:
source: sensor.solar_energy
cycle: hourly
offset:
minutes: 30
This gives me a sensor that shows the solar energy generated in the last hour, from x:30 to x+1:30. This is based on sensor.solar_energy.
Note - this is a “utility” meter (a counter) which is just something that keeps count of energy used/generated and resets every so often. To make this work, it has to use ‘energy’ and not ‘power’.
A solar PV generates power, which is an instantaneous figure, usually in Watts or kW. You have to turn power into energy, using the Riemann Sum integration. This integrates (sums) the individual power values over a set period of time to get energy, usually in kWh (use kilo and hours).
sensor:
- platform: integration
source: sensor.mb_solar_power
name: solar_energy
unit_prefix: k
unit_time: h
round: 2
method: trapezoidal
My source here is my Modbus reading of Solar Power (Watts) from my inverter. Today I would use the helper RI to set up this sensor as it is easier to do.
So…
Reimann Sum (helper integration) takes Solar Power [Watts] → Energy kWh
Utility meter (yaml) takes total energy → energy per hour (offset by 30 minutes)
Yup, time zones are a real pain. I live in the UK, and we have the unfortunate situation of being on GMT which is the same as UCT, until we go to DST on BST, and then things stop working.
1 Like
This is a great thread.
I am currently struggling with the predictions on my three PV arrays.
I’m trying to modify your code you shared above, but I’m having trouble with the fact that I have two inverters, thus I don’t know how to configure solar energy hr30 properly.
I guess I will have to add the two readings together huh?
Yup. If you have two inverters then you will need to sum them somewhere.
Either
- have a template sensor to add together the two inverter power figures, then into one Reimann sum etc
- have two Reimann sums, then a template sensor to add the two energy figures into one utility meter
- have two energy utility meters, and a template sensor to add them together and pick that up in the code
- or two utility meters and change the Node-RED code to pick up both and add them together there
Great, thanks for the reply.
I have one more question.
I have composed the code according to you, added my api data.
I have three fields. All of them are giving me a prediction, but the save record node still reports an error. I can’t find it.
But is it by adding another field.
I added P3 in the Init/Shift array, but I couldn’t find anything else that needed to be added.
Do you have any ideas?
I wrote this (some time back) to deal with two planes.
The node ‘init/shift array’ does the work of setting up the solar array context variable if it does not exist.
In this code, in the ‘init’ part, it will set up the data object. You should find a line reading something like
solarFC.lastRead={P1: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}, P2: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}};
and you can see that this adds object P1 and P2.
The error you are seeing in the save record node is where this has read the solarFC.lastRead object and is trying to update the time. Since you have three planes, you are trying to write to P3 which does not exist, hence ‘cannot set the properties of undefined’ as P3 is not there!
Answer - find the single code line as above and add in the P3 bit (replace it as follows)
solarFC.lastRead={P1: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}, P2: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}, P3: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}};
I can’t guarantee this is 100% correct, but hopefully you get the idea!
Also, you need to go into the context tab on the debug window, find the SolarForecast variable and delete it. This will force the ‘init/shift’ function node to rebuild the array (since it is not there) and this should add the missing P3 object to the ‘last read’ section, and it should all work.
Fingers crossed.
Edit - there is going to be other places where the code is setup for plane 1 and 2, and will not include 3. Don’t have time to look at it at the moment, but hopefully you can see how far you get yourself!
I found this line and I already edited it, but it didn’t help.
I tried overwriting it with your code, but still nothing.
As you can see in the screen, I have debug 9 connected to the Init/Shift array and this is how P3 will give its data. But if I link debug9 to save record the P3 data will not come.
Your debug output for node debug 9 shows that P1, P2 and P3 records are being produced.
The error comes from ‘save record’ and only when it gets the P3 record. The code as was only deals with seeing P1 and P2.
The code in ‘save record’ reads the context variable and expects P3 to be there, but it is not.
If you look at the context variables in the debug window, look at SolarForecast, and look at LastRead you should see objects P1 and P2. But no P3.
The new code will create P1 and P2 and P3 for a new array, but this only runs when the node can’t find the array.
So, you need to delete the existing variable, then the next time the init function node runs it will not find it and create it from scratch, and with the new code it will now add P3 (which you can check by looking at the context variable in the debug window).
I don’t know where to look for a existing variable to delete … 