Well check your sensor for P_load, it is also shwoing fixed values in Home Assistant?
Check the optimization status in the add-on logs, maybe that with those values the optimization problem is just unfeasible, hence the treat_def_as_semi_cont constraint not being respected.
Regarding P_load: Yes it does, but this is a sensor from EMHASS right? My sensor.power_load_without_variable_loads sensor is working.
Ok, and the optimization status? Did you check that? If the optimization is by some reason unfeasible, then the results may not be realiable.
David - after playing around, I am still lost - see examples below:
Time step: 30 (for all examples)
Example 1 with prediction_horizon of 10 and lists with 20 elements creates forecast for 5h despite providing data for 10 hours => forecast seems to be based on prediction_horizon * timestep / 60
post_mpc_optim: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"pv_power_forecast\":[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], \"load_cost_forecast\":[0.415, 0.3793, 0.3793, 0.3664, 0.3664, 0.3488, 0.3488, 0.3366, 0.3366, 0.333, 0.333, 0.3271, 0.3271, 0.321, 0.321, 0.3186, 0.3186, 0.3162, 0.3162, 0.32], \"prediction_horizon\":10,\"soc_init\":0.82,\"soc_final\":0.4,\"def_total_hours\":[8,3]}' http://localhost:5000/action/naive-mpc-optim"
Example 2: same as above but prediction_horizon set to 20 (and also providing lists with elements). This provides forecast for 10h:
post_mpc_optim: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"pv_power_forecast\":[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], \"load_cost_forecast\":[0.415, 0.3793, 0.3793, 0.3664, 0.3664, 0.3488, 0.3488, 0.3366, 0.3366, 0.333, 0.333, 0.3271, 0.3271, 0.321, 0.321, 0.3186, 0.3186, 0.3162, 0.3162, 0.32], \"prediction_horizon\":20,\"soc_init\":0.81,\"soc_final\":0.4,\"def_total_hours\":[8,3]}' http://localhost:5000/action/naive-mpc-optim"
Example 3: Same but providing list with 48 elements and set prediction horizon to 48. This provides forecast for 24h:
post_mpc_optim: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"pv_power_forecast\":[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1028.1, 3451.2999999999997, 5664.7, 7271.200000000001, 8743.2, 9840.3, 10859.199999999999, 11683.1, 12126.0, 12374.0, 12200.2, 11792.8, 11252.5, 10275.3, 9167.3, 7614.2, 6092.1, 4160.9, 559.4, 0.0, 0.0, 0.0, 0.0, 0.0], \"load_cost_forecast\":[0.3793, 0.3793, 0.3664, 0.3664, 0.3488, 0.3488, 0.3366, 0.3366, 0.333, 0.333, 0.3271, 0.3271, 0.321, 0.321, 0.3186, 0.3186, 0.3162, 0.3162, 0.32, 0.32, 0.3516, 0.3516, 0.3765, 0.3765, 0.3998, 0.3998, 0.354, 0.354, 0.2948, 0.2948, 0.2928, 0.2928, 0.2912, 0.2912, 0.2912, 0.2912, 0.2912, 0.2912, 0.3025, 0.3025, 0.3292, 0.3292, 0.3615, 0.3615, 0.3715, 0.3715, 0.3246, 0.3246], \"prediction_horizon\":48,\"soc_init\":0.81,\"soc_final\":0.4,\"def_total_hours\":[8,3]}' http://localhost:5000/action/naive-mpc-optim"
So far, so good. All working as expected.
But I immediately get an error when I use a prediction_horizon of more than 48; here with 52 and also list with 52 elements:
post_mpc_optim: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"pv_power_forecast\":[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1028.1, 3451.2999999999997, 5664.7, 7271.200000000001, 8743.2, 9840.3, 10859.199999999999, 11683.1, 12126.0, 12374.0, 12200.2, 11792.8, 11252.5, 10275.3, 9167.3, 7614.2, 6092.1, 4160.9, 559.4, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], \"load_cost_forecast\":[0.3793, 0.3793, 0.3664, 0.3664, 0.3488, 0.3488, 0.3366, 0.3366, 0.333, 0.333, 0.3271, 0.3271, 0.321, 0.321, 0.3186, 0.3186, 0.3162, 0.3162, 0.32, 0.32, 0.3516, 0.3516, 0.3765, 0.3765, 0.3998, 0.3998, 0.354, 0.354, 0.2948, 0.2948, 0.2928, 0.2928, 0.2912, 0.2912, 0.2912, 0.2912, 0.2912, 0.2912, 0.3025, 0.3025, 0.3292, 0.3292, 0.3615, 0.3615, 0.3715, 0.3715, 0.3246, 0.3246, 0.3156, 0.3156, 0.3017, 0.3017], \"prediction_horizon\":52,\"soc_init\":0.81,\"soc_final\":0.4,\"def_total_hours\":[8,3]}' http://localhost:5000/action/naive-mpc-optim"
Always this error:
[2023-02-07 20:03:50,850] ERROR in app: Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 135, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 119, in set_input_data_dict
df_input_data_dayahead = copy.deepcopy(df_input_data_dayahead)[df_input_data_dayahead.index[0]:df_input_data_dayahead.index[prediction_horizon-1]]
File "/usr/local/lib/python3.9/dist-packages/pandas/core/indexes/base.py", line 5039, in __getitem__
return getitem(key)
File "/usr/local/lib/python3.9/dist-packages/pandas/core/arrays/datetimelike.py", line 341, in __getitem__
"Union[DatetimeLikeArrayT, DTScalarOrNaT]", super().__getitem__(key)
File "/usr/local/lib/python3.9/dist-packages/pandas/core/arrays/_mixins.py", line 272, in __getitem__
result = self._ndarray[key]
IndexError: index 51 is out of bounds for axis 0 with size 48
When using 56, the error reads:
IndexError: index 55 is out of bounds for axis 0 with size 48
Would you mind checking? There seems to be somewhere a limitation for the prediction_horizon of 48!?
BTW: i can provide a list with 96 elements and set the prediction_horizon to 48. That provides a forecast for 24h but ignores the data for the second 24h. It works without error but just seems to ignore the data provided that is past the prediction_horizon.
thanks!
You will need to provide your complete config to check that.
Well yes MPC will not go any further than 24h into the future. Why will you ever need to do that? Just simply use a day-ahead optimization in that case. MPC is designed for short-term forecasts (example the next hour) and relatively high frequency optimizations, for example every 10min.
That change of status from one day to another indicates that there is a problem with the data that you are feeding to emhass. Either the load sensor or any of the forecasts
I understood that the purpose of dayahead optimization is to do a forecast for the next 24h but I cannot a forecast of more or less than 24h?
Here is what I am trying to do:
Electricity prices (Tibber) are updated every day around 2pm for the following day. At that time, I have electricity prices for c. 36h (remaining today plus tomorrow)
Electricity prices tomorrow might be much higher or lower than for rest of today. Also solar production might differ substantially.
The optimization for deferrable loads and use of battery might therefore change significantly. I.e.:
If I have very cheap electricity and lots of solar today => should fully charge the battery today and only use it tomorrow. Also use deferrable loads mostly today and minimize their use tomorrow.
I am basically trying to move deferrable loads between days and also the use of the battery to optimize for PV self use and minimize price from grid electricity - not just for for the next 24h but for a longer period.
The longer forecast, the less precise it gets which is why I have been trying to use MPC in order to continually roll it forward in case PV forecast changes a lot or actual consumption / battery SOC is changing a lot.
I could try to implement this with rules but felt Emhass can do this much better.
Any ideas how to implement?
I do exactly this using the MPC optimisation, my prices for next 24 hours are updated at 1230 each day.
My MPC optimisation runs every minute as my prices are changing every 5 minutes, my solar PV and household load also change. The weather changes so I may need more air-conditioning, my EV connects and disconnects from the charging point during the day so EMHASS is constantly updating the optimisation during the day using EMHASS.
I often see after the 1230 update things like charging my EV are deferred to the next day when the prices are cheaper or I have more solar production forecast for tomorrow.
https://twitter.com/MarkSPurcell/status/1620256092323459072?t=kBefoZBy271ylSaxq7ar7Q&s=19
Managed to get deferral of loads until tomorrows low prices again today with the 1230 prices update, when my energy provider provides price forecasts for the next 24 hours.
At 1200 today you can see my EV charging scheduled for 2pm, along with pool pumps and hot water heating.
After the 1230 price update for the next 24 hours, you can see the prices for tomorrow are lower, so the scheduling for my EV charging and pool pumps have been shifted to tomorrows low prices, leaving the rest of todays scheduling for my household load and battery charging.
Hi guys. New to HA . I’m in Australia so gear is a bit different? Saw this integration and loved the idea.
Few initial stumbling blocks. How close is close enough with selection of the panels and inverter? I have Hanwah Q.Peak DUO ML G10+ 415W panels, but closest I see is 390W panels in the spreadsheet.
Same for the inverter. I have a solaredge SE10000H-AUS3MBX14 maybe the (SE10000H-US) is the closest?
I don’t have the skills to do the solcast option.
I would say ballpark is fine there are some many other variations to the modelling. You really just want to have a relevant solar production curve.
So go for something close with 415 W panels and 10kW inverter.
thanks , I’ll give it a go.
Thanks a lot and makes sense. Will try it accordingly.
I got the EV back from the repair shop this week and with some very patient help from Mark P I got and MPC optimisation up and running. Solcast for solar forecasting, no dynamic energy pricing but I have a TOU tariff. The automation to change the EV charger amps is working nicely. I just need to add the automation to track to the battery SOC forecast. Just wanted the thank Mark for his help and congratulate David for an awesome add on.
I have more deferrable loads coming as the pool comes online in May. My plan is the “simulate” what it would be like to be on Amber by using a feed of NEM forward looking wholesale price forecast (credit to Mark for finding http://nemlog.com.au/api/pdrgn/all/json). This will be the first time since I got my solar installed in 2020 that I will using it to its full capacity with just the right amount of local consumption.
Very much looking forward to helping extend the add-on to support a more sophisticated approach to load forecasting and support for bi-directional EVs 
2 Likes
I have tried running the mpc optim. But have some questions. Why is this needed instead of the next day optim? will this remember when the last deferable load was run if i run this once an hour to update the forcast? And do the mpc-optim command look correct? I have modified it to get nordpool prices and not forcast longer than today if there if its before 3 pm and after pm it will also include the next day.
emhass_mpc_optim: 'curl -i -H ''Content-Type: application/json'' -X POST -d ''{"load_cost_forecast":{{((state_attr(''sensor.nordpool_kwh_krsand_nok_2_095_025'', ''raw_today'')|map(attribute=''value'') |list + state_attr(''sensor.nordpool_kwh_krsand_nok_2_095_025'', ''raw_tomorrow'')|map(attribute=''value'')| list))[now().hour:][:24]}},"prod_price_forecast":{{((state_attr(''sensor.nordpool_kwh_krsand_nok_3_10_0'', ''raw_today'')|map(attribute=''value'') |list + state_attr(''sensor.nordpool_kwh_krsand_nok_3_10_0'', ''raw_tomorrow'')|map(attribute=''value'')| list))[now().hour:][:24]}}, "prediction_horizon":''{{ (24 if now().hour <= 15 else 48) - now().hour}}'', "def_total_hours":[8,3]}'' http://localhost:5000/action/naive-mpc-optim'
I find the frequency to run the optimisation depends on how quickly your inputs are charging.
If your are on a fixed or time of use tariff and your solar production forecast is good then daily optimisation would be sufficient.
In my case my electricity tariff (buy and sell prices) is updated every minute and can vary widely during the day from negative to over $10 / kWh. So I run the mpc optimisation every minute.
My understanding with Nordpool is your pricing is fixed once a day so you would get a reasonable optimisation plan running just after the prices are published. You can test how reasonable it is by running mpc a few times during the day and see if there is wild variation in the optimisation plan.
EMHASS doesn’t record any state between optimisation runs so it doesn’t know if it has just operated your load before the last run. I don’t actually find this to be a problem even though I am optimising every minute. My long run loads over 8 hours still work and my short run loads 1 hour sometimes might pick up 2 hours operations. One challenge I have mitigated in my automations it switching loads two quickly, on/off/on/off every minute. Once an automation starts a load I force it to continue for at least 15 minutes.
To further check your script, can you copy and paste into DeveloperTool:Template and post those results so we can see what all the variable expansion results are.
I understand, so the reason why i wanted to do this is that I have expirienced some thimes my hot water is getting cold because it can have a long time between the heating. if I have a day with cheap prices and next day is got production it can go more than 24 hours between heating. Also wather can change so i want to run it more times.
OK, so how do you fix this for the deferable load then when you run it like this?