All your input is very helpful! Many thanks for this!
Hi, sorry to drop in on a seemingly solved matter.
But can’t we grab the already existing data from somewhere in the database?
Anyone know?
I don’t know the database structure that well. But this dashed curve is “plotted” from info that must be stored locally somewhere.
Just a thought.
My goal is to simply get the forecast energy (kWh) for the remainder of the day. (from now until sunset)
The marked solution is an alternate way to solve this by making additionnal calls to the Forecast.Solar API, so it may not be optimal.
What you are saying is exactly the original issue on this thread.
There is no known solution to directly “grab” that dashed ploted data.
If you find it please share for others…
I’ll grab the database from a backup file, and dissect it to try to find it there.
…
[EDIT] Ok, after some digging I don’t think this is stored in the HA database.
Maybe this is stored in RAM afterall.
The Energy settings file also doesn’t help (.storage/energy)
"data": {
"energy_sources": [
(...)
{
"type": "solar",
"stat_energy_from": "sensor.[YOUR_ENERGY_SENSOR]",
"config_entry_solar_forecast": [
"[long identifier found in the core.config_entry file]"
]
},
(...)
I have posted in WTH: WTH can’t we access all data from solar forecast as in energy dashboard. You can allways vote for it to be solved.
Mean while I have a workaround where you actually access the data from HA so there is no need for extra calls to the Forecast.Solar API. The data is only accessible through the web-socket API.
(I rushed this together this afternoon so I can’t guarantee it will work for everyone, and it most certainly contains some bugs.)
It requires Node Red and node-red-contrib-home-assistant-websocket. Here is a flow you can import:
[{"id":"022a95a855f03d7e","type":"ha-api","z":"4d1a86a4c406178b","name":"","server":"993dd4ef.f733c8","version":1,"debugenabled":false,"protocol":"websocket","method":"get","path":"","data":"{\"type\":\"energy/solar_forecast\"}","dataType":"json","responseType":"json","outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"results"}],"x":350,"y":180,"wires":[["3855bddf.df6552"]]},{"id":"3855bddf.df6552","type":"function","z":"4d1a86a4c406178b","name":"Calculate","func":"// Fix values from HA forecast integration\nlet id = Object.keys(msg.payload)[0];\nconst whHours = msg.payload[id].wh_hours;\nvar newTable = {};\n\nObject.keys(whHours).forEach(key => {\n var t = new Date(key);\n if (whHours[key] == 0) {\n t.setMinutes(0,0,0);\n newTable[t.getTime()] = whHours[key];\n } else if (t.getMinutes() > 0) {\n t.setHours(t.getHours()+1,0,0,0);\n newTable[t.getTime()] = whHours[key];\n } else {\n newTable[t.getTime()] = whHours[key];\n }\n});\n\n// Time zone fixes\nlet firstKey = Object.keys(whHours)[0];\nconst tzOffset = firstKey.slice(-6);\nvar tzSeconds = parseInt(tzOffset.slice(1,3)*60*60 + tzOffset.slice(-2)*60)\nif (tzOffset.slice(0,1) === \"-\") {\n tzSeconds = -tzSeconds\n}\n\n// Create 48 hours array\nconst today = new Date();\nconst tomorrow = new Date();\ntomorrow.setDate(tomorrow.getDate() + 1);\n\nvar todayHours = [];\nvar tomorrowHours = [];\nfor (var i = 0; i < 24; i++) {\n today.setHours(i,0,0,0);\n todayHours.push(today.getTime());\n tomorrow.setHours(i,0,0,0);\n tomorrowHours.push(tomorrow.getTime());\n}\nallHours = todayHours.concat(tomorrowHours)\n\n// Create forecast table\nvar solarForecast = [];\nconst now = new Date(msg.timestamp + tzSeconds*1000);\nvar this_hour = null;\nvar next_hour = null;\n\nallHours.forEach(hour => {\n startDate = new Date(hour + tzSeconds*1000);\n stopDate = new Date(hour + tzSeconds*1000);\n stopDate.setHours(stopDate.getHours() + 1);\n if (msg.timestamp > hour) {\n this_hour = newTable[hour];\n next_hour = newTable[hour+3600000];\n }\n startStr = startDate.toISOString().slice(0,19) + tzOffset;\n stopStr = stopDate.toISOString().slice(0,19) + tzOffset;\n if (hour in newTable) {\n solarForecast.push({\n \"start\": startStr,\n \"end\": stopStr,\n \"value\": newTable[hour]\n });\n } else {\n solarForecast.push({\n \"start\": startStr,\n \"end\": stopStr,\n \"value\": 0\n });\n }\n});\n\nmsg.solar_forecast = solarForecast;\nmsg.this_hour = this_hour\nmsg.next_hour = next_hour\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":480,"y":180,"wires":[["6c9647caa4b673a9"]]},{"id":"6c9647caa4b673a9","type":"ha-entity","z":"4d1a86a4c406178b","name":"Solar PV Forecast","server":"993dd4ef.f733c8","version":2,"debugenabled":false,"outputs":1,"entityType":"sensor","config":[{"property":"name","value":"Solar PV Forecast"},{"property":"device_class","value":"energy"},{"property":"icon","value":"mdi:solar-power-variant"},{"property":"unit_of_measurement","value":"Wh"},{"property":"state_class","value":"measurement"},{"property":"last_reset","value":""}],"state":"this_hour","stateType":"msg","attributes":[{"property":"full_forecast","value":"solar_forecast","valueType":"msg"},{"property":"next_hour","value":"next_hour","valueType":"msg"}],"resend":true,"outputLocation":"payload","outputLocationType":"none","inputOverride":"allow","outputOnStateChange":false,"outputPayload":"","outputPayloadType":"str","x":650,"y":180,"wires":[[]]},{"id":"f3b80fbf412bc248","type":"server-state-changed","z":"4d1a86a4c406178b","name":"Forecast.Solar","server":"993dd4ef.f733c8","version":4,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"sensor.energy_next_hour","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"","halt_if_type":"str","halt_if_compare":"is","outputs":1,"output_only_on_state_change":false,"for":"0","forType":"num","forUnits":"minutes","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"topic","propertyType":"msg","value":"forecast_uppdated","valueType":"str"},{"property":"timestamp","propertyType":"msg","value":"","valueType":"date"}],"x":180,"y":200,"wires":[["022a95a855f03d7e"]]},{"id":"9e794fb98f28d5e8","type":"inject","z":"4d1a86a4c406178b","name":"Every hour","props":[{"p":"topic","vt":"str"},{"p":"timestamp","v":"","vt":"date"}],"repeat":"3600","crontab":"","once":true,"onceDelay":0.1,"topic":"start","x":190,"y":160,"wires":[["022a95a855f03d7e"]]},{"id":"993dd4ef.f733c8","type":"server","name":"Home Assistant","version":4,"addon":false,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":30,"areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m"}]
It will create a new sensor named sensor.solar_pv_forecast. The state of the sensor is this hours predicted production (the same as Estimated Energy Production - This Hour (in kWh) with the Forecast.Solar integration). But it also has an attribute named full_forecast which contains a list of all hours prediction for this day and the next day. So in total 48 hours prediction or forecast.
Here is an exampel how to visualize it useing a apexcharts-card:
type: custom:apexcharts-card
header:
show: false
show_states: true
title: Solar forecast
now:
show: true
label: Now
graph_span: 2d
span:
start: day
series:
- entity: sensor.solar_pv_forecast
data_generator: |
return entity.attributes.full_forecast.map((entry) => {
return [new Date(entry.start), entry.value];
});
offset: '-30min'
name: Forecast
type: column
color: darkgreen
opacity: 0.4
stroke_width: 1
show:
in_header: false
legend_value: false
3 Likes
Hi Biscuit,
may I ask for the bits of the configuration.yaml regarding the sensor.solar_energy_hr30. Mine looks like this:
utility_meter:
energy:
unique_id: solar_energy_hr30
name: solar_energy_hr30
source: sensor.inverter_power #inverter power AC [W]
cycle: hourly
offset: '00:30'
I assume, the inverter energy Sensor using Riemann Sum has been set up via the UI? What input from the inverter did you choose here? I use a Hoymiles inverter and I used the “Power AC” [W] at the moment. Correct so far? My sensor.solar_energy_hr30 is working fine. Sums up the Watts and resets. I checked the plot in the HA history. But nothing is displayed in the Apex chart. Yesterday it showed the actual plot, but today nothing, without having touched anything. API call is set up correctly, sensor.table_fc is filled with forecast data (at fcwatts).
I would really appreciate any tips.
Greetings, Björn
EDIT: messed up the timezone stuff in the “Date/Time formatter” node. Switched to a different style, as I’m in DE. Now I can see the forecast plot. Think this is solved. On my own 
Hi.
I can try and put my brain into gear!
I think your utility yaml is good to go, but I think you are using power and should be using energy as the input.
OK, so the (solar energy) utility meter I wanted resets hourly but at 30 minutes past the hour, which cannot be set up using the helpers, so I have done this by hand in yaml.
In my yaml this is (sensor called ‘solar_energy_hr30’)
utility_meter:
solar_energy_hr30:
source: sensor.solar_energy
cycle: hourly
offset:
minutes: 30
This gives me a sensor that shows the solar energy generated in the last hour, from x:30 to x+1:30. This is based on sensor.solar_energy.
Note - this is a “utility” meter (a counter) which is just something that keeps count of energy used/generated and resets every so often. To make this work, it has to use ‘energy’ and not ‘power’.
A solar PV generates power, which is an instantaneous figure, usually in Watts or kW. You have to turn power into energy, using the Riemann Sum integration. This integrates (sums) the individual power values over a set period of time to get energy, usually in kWh (use kilo and hours).
sensor:
- platform: integration
source: sensor.mb_solar_power
name: solar_energy
unit_prefix: k
unit_time: h
round: 2
method: trapezoidal
My source here is my Modbus reading of Solar Power (Watts) from my inverter. Today I would use the helper RI to set up this sensor as it is easier to do.
So…
Reimann Sum (helper integration) takes Solar Power [Watts] → Energy kWh
Utility meter (yaml) takes total energy → energy per hour (offset by 30 minutes)
Yup, time zones are a real pain. I live in the UK, and we have the unfortunate situation of being on GMT which is the same as UCT, until we go to DST on BST, and then things stop working.
1 Like
This is a great thread.
I am currently struggling with the predictions on my three PV arrays.
I’m trying to modify your code you shared above, but I’m having trouble with the fact that I have two inverters, thus I don’t know how to configure solar energy hr30 properly.
I guess I will have to add the two readings together huh?
Yup. If you have two inverters then you will need to sum them somewhere.
Either
- have a template sensor to add together the two inverter power figures, then into one Reimann sum etc
- have two Reimann sums, then a template sensor to add the two energy figures into one utility meter
- have two energy utility meters, and a template sensor to add them together and pick that up in the code
- or two utility meters and change the Node-RED code to pick up both and add them together there
Great, thanks for the reply.
I have one more question.
I have composed the code according to you, added my api data.
I have three fields. All of them are giving me a prediction, but the save record node still reports an error. I can’t find it.
But is it by adding another field.
I added P3 in the Init/Shift array, but I couldn’t find anything else that needed to be added.
Do you have any ideas?
I wrote this (some time back) to deal with two planes.
The node ‘init/shift array’ does the work of setting up the solar array context variable if it does not exist.
In this code, in the ‘init’ part, it will set up the data object. You should find a line reading something like
solarFC.lastRead={P1: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}, P2: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}};
and you can see that this adds object P1 and P2.
The error you are seeing in the save record node is where this has read the solarFC.lastRead object and is trying to update the time. Since you have three planes, you are trying to write to P3 which does not exist, hence ‘cannot set the properties of undefined’ as P3 is not there!
Answer - find the single code line as above and add in the P3 bit (replace it as follows)
solarFC.lastRead={P1: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}, P2: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}, P3: {time: null, watts: null, todayTotal: null, tomorrowTotal: null}};
I can’t guarantee this is 100% correct, but hopefully you get the idea!
Also, you need to go into the context tab on the debug window, find the SolarForecast variable and delete it. This will force the ‘init/shift’ function node to rebuild the array (since it is not there) and this should add the missing P3 object to the ‘last read’ section, and it should all work.
Fingers crossed.
Edit - there is going to be other places where the code is setup for plane 1 and 2, and will not include 3. Don’t have time to look at it at the moment, but hopefully you can see how far you get yourself!
I found this line and I already edited it, but it didn’t help.
I tried overwriting it with your code, but still nothing.
As you can see in the screen, I have debug 9 connected to the Init/Shift array and this is how P3 will give its data. But if I link debug9 to save record the P3 data will not come.
Your debug output for node debug 9 shows that P1, P2 and P3 records are being produced.
The error comes from ‘save record’ and only when it gets the P3 record. The code as was only deals with seeing P1 and P2.
The code in ‘save record’ reads the context variable and expects P3 to be there, but it is not.
If you look at the context variables in the debug window, look at SolarForecast, and look at LastRead you should see objects P1 and P2. But no P3.
The new code will create P1 and P2 and P3 for a new array, but this only runs when the node can’t find the array.
So, you need to delete the existing variable, then the next time the init function node runs it will not find it and create it from scratch, and with the new code it will now add P3 (which you can check by looking at the context variable in the debug window).
I don’t know where to look for a existing variable to delete … 
Yes it takes a while to find all these things. Node-RED is quite complicated but amazing in what it can do.
The debug window on the right.
Select the tab for the context (variable)
Under ‘flow’ you should see the SolarForecast variable.
There are icons for refreshing these variables, and icons for copying (very useful), and an icon for deleting.
Once it has gone you should be good to go.
As I said, there are other places where my code adds in P1, then looks to see if there is a P2, but clearly I did not write this for P3/4/5…
I’m here again.
As predicted after midnight the calculation at the bottom of the flow stopped working.
I found some possibilities where P3 is missing in the Update Table, but it’s too complicated for me and I don’t understand it at all.
Yes indeed.
Well, I wrote this for myself with two planes, and I tried to write the code so that it would work for anyone with just one plane (it must have P1, then it looks to see if P2 exists rather than assuming it does). Clearly I did not write this for someone with three planes, so the code has to be adjusted to add in for P3.
It has been several months since I last touched this - thankfully I added some comments so I have some hope of understanding what I did, otherwise I would be just as puzzled as you…
I think we have Init/shift array ok, and yes I was expecting issues from Update Table.
I don’t have the time to rebuild the flow to cope with ‘n’ planes, so this is going to have to be a patch job.
var FC = flow.get("SolarForecast")
// save actual from msg.payload index/value. index = last hour
// this sould be energy Wh between hh-:30 and hh+:30
var index=msg.payload.index;
FC.solarTable[index].actualWh=msg.payload.value;
// save pre-update forecast for last hour to forecast history
FC.solarTable[index].oldfcW=FC.solarTable[index].fcW;
var today=FC.dates.today;
var plane2=(FC.lastRead.P2.time.substring(0,10)==today); // true if P2 last update date is today
var plane3=(FC.lastRead.P3.time.substring(0,10)==today); // extra P3
FC.energyToday=FC.lastRead.P1.todayTotal;
FC.energyTomorrow=FC.lastRead.P1.tomorrowTotal;
if (plane2) {
FC.energyToday+=FC.lastRead.P2.todayTotal;
FC.energyTomorrow+=FC.lastRead.P2.tomorrowTotal;
}
if (plane3) {
FC.energyToday+=FC.lastRead.P3.todayTotal;
FC.energyTomorrow+=FC.lastRead.P3.tomorrowTotal;
}
var i, j;
var key, value, date, hour;
var wattArray=FC.lastRead.P1.watts;
var keyArray=Object.keys(wattArray);
// clear all hour forecasts to zero
for (i=24; i<72; i++) {FC.solarTable[i].fcW=0}
// save P1 last-read values to table
for (i=0; i<keyArray.length; i++) {
key=keyArray[i];
value=wattArray[key];
date=key.substring(0,10);
hour=parseInt(key.substring(11,13),10);
j=hour+24;
if (date==today){
if (value==0){ // capture start/end hour
if (hour<12) {FC.start=key}
else {FC.stop=key}
}
} else {j+=24}
if(value>0) {FC.solarTable[j].fcW=value}
}
// add in P2 values if required
if (plane2) {
wattArray=FC.lastRead.P2.watts;
keyArray=Object.keys(wattArray);
for (i=0; i<keyArray.length; i++) {
key=keyArray[i];
value=wattArray[key];
date=key.substring(0,10);
hour=parseInt(key.substring(11,13),10);
j=hour+24;
if (date>today) {j+=24}
if (value>0) {FC.solarTable[j].fcW+=value}
}
}
// add in P3 values if required
// extra for P3
if (plane3) {
wattArray=FC.lastRead.P3.watts;
keyArray=Object.keys(wattArray);
for (i=0; i<keyArray.length; i++) {
key=keyArray[i];
value=wattArray[key];
date=key.substring(0,10);
hour=parseInt(key.substring(11,13),10);
j=hour+24;
if (date>today) {j+=24}
if (value>0) {FC.solarTable[j].fcW+=value}
}
}
// recalculate watt-hours for each hr:-30 to hr:+30 and
// post to hh:00 using a+b+b+c/4
// run from index (last hour) to end of today
for (i=index; i<48; i++) {
value=FC.solarTable[i-1].fcW;
if (i==index) {value=FC.solarTable[i-1].oldfcW}
value+=2*FC.solarTable[i].fcW;
value+=FC.solarTable[i+1].fcW;
FC.solarTable[i].efcWh=Math.floor(value/4);
}
flow.set("SolarForecast", FC);
return msg;
I have not tested this (never a good idea…) but I have copied out the code and added in what I think is required.
You should
- copy the Update Table node (just use ^c and ^v, paste it to one side in the flow - this keeps a copy just in case…)
- edit the original Update Table node (in the flow)
- select the entire text in the function node, and delete it
- select (use the copy option) the entire text from the text box above
- paste the new code into the node, save, and deploy.
You will need to do a bit of digging in the context tab to check that the SolarForecast variable is ok - this should have P1 and P2 and P3 under LastRead. You can just run and test the flow to see if this variable updates correctly and the graph is OK.
The error you are getting on the Analyse Forecast is not something I can debug without seeing the data. I suggest for the mean time that you turn this bit of the code off - it is a nice to have but not necessary at the moment. Just edit the Analyse Forecast node, and at the very bottom left use the ‘enabled’ option to disable - save and redeploy.
Hope it now works
Although you should not have to, again if it does not work try deleting the SolarForecast variable in context - this will force the Init function to recreate the variable table from scratch.
I don’t have time at the moment to look deeper into this, and I really need copies of the actual data so as to debug what is going on. In general terms, there is not a great deal of interest in Node-RED stuff as most like to use HA directly, and you are only the second person I know of who has tried to use this code, so I will put the need for an ‘update’ to (say) four planes on my list of nice things to do when I get around to it.
Looking at the code it does need a bit of a tidy up, and I would probably use JSONata now and not JavaScript too. Of course, there is nothing stopping you having a go at editing it yourself! 
Thank you very much.
Solar table is already running well, there are P1,P2 and P3.
The forecast is still reporting errors.
So I have it turned off. Too bad, I wanted to compare the forecast results with another option.
Maybe someone else who has multiple PV arrays will chime in and know where the problem is.
Thanks for this at least.
Home assistant connection error suggests that the sensor nodes are not connected to the home assistant server node correctly.
Again, another reason to revisit this code (when I have time). The original code probably still uses entity nodes, which were replaced by the new sensor nodes last September. You will have to check that the FC Solar Table node is a sensor node, and that they have an ‘entity config’ set up as a configuration node, and this node connects to home assistant server (node) correctly.