Oh, ok so the array size can change from Amber? I didn’t know that. I expect its safer to use 33 and ignore the tail end of the data as its calculating the forecast every minute anyway.
Amber/ AEMO publishes the 24 hour Forecast after 12:30 each day so then you have 48x30 minute forecasts.
They continue to update this until 0330, when the forecasts stop, so between 0330 and 1230 you have incrementally less than 48 forecasts. At 1200 you have the smallest number of forecasts 1200-0330 the next day which is 33x30 minute forecasts.
1 Like
This looks pretty good, what results do you get for the optimisation?
I’m getting errors in the log file.
2023-07-12 17:42:00,052 - web_server - INFO - Setting up needed data
2023-07-12 17:42:00,054 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-12 17:42:00,667 - web_server - INFO - Retrieving weather forecast data using method = list
2023-07-12 17:42:00,668 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-07-12 17:42:00,669 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-12 17:42:01,933 - web_server - INFO - >> Performing naive MPC optimization...
2023-07-12 17:42:01,933 - web_server - INFO - Performing naive MPC optimization
2023-07-12 17:42:01,942 - web_server - INFO - Perform an iteration of a naive MPC controller
2023-07-12 17:42:01,943 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2190, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1486, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 199, in action_call
opt_res = naive_mpc_optim(input_data_dict, app.logger)
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 275, in naive_mpc_optim
opt_res_naive_mpc = input_data_dict['opt'].perform_naive_mpc_optim(
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 573, in perform_naive_mpc_optim
self.opt_res = self.perform_optimization(df_input_data, P_PV.values.ravel(), P_load.values.ravel(),
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 156, in perform_optimization
if self.optim_conf['treat_def_as_semi_cont'][k]:
IndexError: list index out of range
2023-07-12 17:42:01,976 - web_server - INFO - Setting up needed data
2023-07-12 17:42:01,978 - web_server - INFO - >> Publishing data...
2023-07-12 17:42:01,978 - web_server - INFO - Publishing data to HASS instance
2023-07-12 17:42:01,996 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 922
2023-07-12 17:42:02,007 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 1085.43
2023-07-12 17:42:02,016 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2023-07-12 17:42:02,026 - web_server - INFO - Successfully posted to sensor.p_deferrable1 = 0.0
2023-07-12 17:42:02,037 - web_server - INFO - Successfully posted to sensor.p_deferrable2 = 0.0
2023-07-12 17:42:02,046 - web_server - INFO - Successfully posted to sensor.p_deferrable3 = 0.0
2023-07-12 17:42:02,054 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 163.43
2023-07-12 17:42:02,063 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 68.83
2023-07-12 17:42:02,072 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 0.0
2023-07-12 17:42:02,084 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = -2.22
2023-07-12 17:42:02,096 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 0.19
2023-07-12 17:42:02,106 - web_server - INFO - Successfully posted to sensor.unit_prod_price = 0.09
This seems to be causing some loss of data.
Not sure what to do to correct the error?
do you have the correct number of devices setup for this variable?
Oops! I removed all the deferrable loads except one. Simplify the setup for the time being. But missed the ‘number_of_deferrable_loads’ setting. Still has 4. Thanks again?!
By the way if I change the ‘list_peak_hours_periods_start_hours’ and end, does that restrict the operation of the deferrable load to that time period?
Also the one deferrable load left is the pool pump. I assume that ‘treat_deferrable_load_as_semi_cont: true’ is appropriate? The pump is simple one speed 1500w motor.
number_of_deferrable_loads: 4
list_nominal_power_of_deferrable_loads:
- nominal_power_of_deferrable_loads: 1500
list_operating_hours_of_each_deferrable_load:
- operating_hours_of_each_deferrable_load: 2
list_peak_hours_periods_start_hours:
- peak_hours_periods_start_hours: "02:54"
list_peak_hours_periods_end_hours:
- peak_hours_periods_end_hours: "15:24"
list_treat_deferrable_load_as_semi_cont:
- treat_deferrable_load_as_semi_cont: true
number_of_deferrable_loads: 1
list_nominal_power_of_deferrable_loads:
- nominal_power_of_deferrable_loads: 1500
list_operating_hours_of_each_deferrable_load:
- operating_hours_of_each_deferrable_load: 2
list_peak_hours_periods_start_hours:
- peak_hours_periods_start_hours: "02:54"
list_peak_hours_periods_end_hours:
- peak_hours_periods_end_hours: "15:24"
list_treat_deferrable_load_as_semi_cont:
- treat_deferrable_load_as_semi_cont: true
Still doing strange things like discharging at 1am and I can’t see any amber data after midnight. Just stuck on FiT and price from before midnight
Better logs now though:
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun emhass (no readiness notification)
s6-rc: info: service legacy-services successfully started
2023-07-12 18:08:00,599 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2023-07-12 18:08:00,599 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2023-07-12 18:08:00,599 - web_server - INFO - The data path is: /share
2023-07-12 18:08:00,601 - web_server - INFO - Using core emhass version: 0.4.13
waitress INFO Serving on http://0.0.0.0:5000
2023-07-12 18:09:00,064 - web_server - INFO - Setting up needed data
2023-07-12 18:09:00,120 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-12 18:09:00,734 - web_server - INFO - Retrieving weather forecast data using method = list
2023-07-12 18:09:00,735 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-07-12 18:09:00,736 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-12 18:09:01,919 - web_server - INFO - >> Performing naive MPC optimization...
2023-07-12 18:09:01,919 - web_server - INFO - Performing naive MPC optimization
2023-07-12 18:09:01,932 - web_server - INFO - Perform an iteration of a naive MPC controller
2023-07-12 18:09:02,012 - web_server - INFO - Status: Optimal
2023-07-12 18:09:02,013 - web_server - INFO - Total value of the Cost function = -0.11
2023-07-12 18:09:02,446 - web_server - INFO - Setting up needed data
2023-07-12 18:09:02,448 - web_server - INFO - >> Publishing data...
2023-07-12 18:09:02,449 - web_server - INFO - Publishing data to HASS instance
2023-07-12 18:09:02,477 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 0
2023-07-12 18:09:02,490 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 1078.94
2023-07-12 18:09:02,504 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2023-07-12 18:09:02,516 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 1078.94
2023-07-12 18:09:02,533 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 45.32
2023-07-12 18:09:02,546 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 0.0
2023-07-12 18:09:02,556 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = -0.11
2023-07-12 18:09:02,566 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 0.2
2023-07-12 18:09:02,577 - web_server - INFO - Successfully posted to sensor.unit_prod_price = 0.09
At 1am your sell price is very high at 20¢/ kWh so it makes sense for it to discharge.
Can you include both loads and prices on the graphic as that tells the whole story.
The table has prices after midnight so I suspect there is an issue with your apex card.
1 Like
As you are injecting Amber prices these settings are not used in your optimisation.
For a single speed pool pump you want this to be true.
1 Like
I had to delete and recreate the Amber API token and reconfigure the amber entry? Just stopped working.
Looks ok.
Charges battery on low prices at 2am.
Discharges battery for morning peak at 7am
Starts pool pump at 10am
I suspect 2pm is beyond your 33 forecasts which is why pool pump and battery appear stuck, but that should resolve as the time gets closer.
Yes running perfectly now.
1 Like
Fantastic project and I look forward to getting things running.
But i am a newbie and trying to discover HA
My imput sensors.
Running?
The log
Pushing any button:
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun emhass (no readiness notification)
s6-rc: info: service legacy-services successfully started
2023-07-12 15:10:40,354 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2023-07-12 15:10:40,355 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2023-07-12 15:10:40,355 - web_server - INFO - The data path is: /share
2023-07-12 15:10:40,358 - web_server - INFO - Using core emhass version: 0.4.13
waitress INFO Serving on http://0.0.0.0:5000
2023-07-12 15:11:50,749 - web_server - INFO - EMHASS server online, serving index.html...
2023-07-12 15:11:50,761 - web_server - WARNING - The data container dictionary is empty... Please launch an optimization task
2023-07-12 15:12:35,396 - web_server - INFO - Setting up needed data
2023-07-12 15:12:35,457 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-12 15:12:35,486 - web_server - ERROR - Exception on /action/perfect-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2190, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1486, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 174, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 78, in set_input_data_dict
rh.get_data(days_list, var_list,
File "/usr/local/lib/python3.9/dist-packages/emhass/retrieve_hass.py", line 124, in get_data
data = response.json()[0]
KeyError: 0
What can be the problem?
Regards
renaatdb
Any thoughts on this restful command debug log would be appreciated!
The command is
rest_command:
naive_mpc_optim:
url: http://localhost:5000/action/naive-mpc-optim
method: POST
content_type: 'application/json'
payload: >-
{
"prod_price_forecast": {{
([states('sensor.home_feed_in_price')|float(0)] +
(state_attr('sensor.home_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list))
| tojson
}},
"load_cost_forecast": {{
([states('sensor.home_general_price')|float(0)] +
state_attr('sensor.home_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)
| tojson
}},
"load_power_forecast": {{
([states('sensor.power_load_no_var_loads')|int(0)] +
(states('input_text.fi_fo_buffer').split(', ')|map('multiply',1000)|map('int')|list)[1:]
)| tojson
}},
"pv_power_forecast": {{([states('sensor.solarnet_power_photovoltaics')|int(0)] +
state_attr('sensor.forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list +
state_attr('sensor.forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list
)
}},
"prediction_horizon": {{
min(48, (state_attr('sensor.home_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1)
}},
"alpha": 1,
"beta": 0,
"num_def_loads": 6,
"def_total_hours": [1],
"treat_def_as_semi_cont": [1],
"set_def_constant": [0],
"soc_init": {{ (states('sensor.battery_level')|int(0))/100 }},
"soc_final": 0.05
}
publish_data:
url: http://localhost:5000/action/publish-data
method: POST
content_type: 'application/json'
payload: '{}'
In dev templates I get the following output
Result type: string
rest_command:
naive_mpc_optim:
url: http://localhost:5000/action/naive-mpc-optim
method: POST
content_type: 'application/json'
payload: >-
{
"prod_price_forecast": [0.11, 0.13, 0.16, 0.32, 0.15, 0.14, 0.07, 0.06, 0.03, 0.06, 0.03, 0.0, -0.03, -0.04, -0.04, -0.04, -0.02, 0.0, 0.03, 0.0, 0.0, 0.03, 0.07, 0.12, 0.17, 0.16, 0.16, 0.15, 0.15, 0.16, 0.16, 0.13, 0.12, 0.12, 0.1, 0.12, 0.12, 0.1, 0.09, 0.07, 0.07, 0.06, 0.06, 0.04, 0.04],
"load_cost_forecast": [0.2, 0.22, 0.26, 0.43, 0.25, 0.23, 0.15, 0.14, 0.12, 0.14, 0.12, 0.08, 0.04, 0.03, 0.03, 0.03, 0.06, 0.08, 0.12, 0.08, 0.08, 0.11, 0.16, 0.21, 0.27, 0.25, 0.25, 0.24, 0.24, 0.25, 0.25, 0.22, 0.21, 0.21, 0.18, 0.21, 0.21, 0.19, 0.18, 0.15, 0.15, 0.14, 0.14, 0.12, 0.12],
"load_power_forecast": [0, 1000, 800, 1500, 1000, 900, 1100, 1000, 900, 900, 1000, 900, 1000, 1000, 1000, 900, 2200, 300, 6700, 6700, 1200, -5000, -9000, 1600, 1000, 1200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
"pv_power_forecast": [0, 0, 75, 861, 2225, 3537, 4782, 6165, 7513, 8731, 9694, 10341, 10796, 11079, 11089, 10726, 9812, 8680, 7341, 6005, 4359, 2525, 695, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 75, 805, 2130, 3468, 4754, 6209, 7694, 8890, 9493, 9948, 10409, 10538, 10390, 9991, 9267, 8231, 6960, 5637, 4092, 2344, 758, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
"prediction_horizon": 45,
"alpha": 1,
"beta": 0,
"num_def_loads": 1,
"def_total_hours": [0],
"P_deferrable_nom": [0],
"treat_def_as_semi_cont": [1],
"set_def_constant": [0],
"soc_init": 0.48,
"soc_final": 0.05
}
publish_data:
url: http://localhost:5000/action/publish-data
method: POST
content_type: 'application/json'
payload: '{}'
This template updates at the start of each minute.
This template listens for the following state changed events:
Entity: input_text.fi_fo_buffer
Entity: sensor.battery_level
Entity: sensor.forecast_today
Entity: sensor.forecast_tomorrow
Entity: sensor.home_feed_in_forecast
Entity: sensor.home_feed_in_price
Entity: sensor.home_general_forecast
Entity: sensor.home_general_price
Entity: sensor.power_load_no_var_loads
Entity: sensor.solarnet_power_photovoltaics
Then in EMHASS i have the following debug log
2023-07-13 05:42:14,172 - web_server - INFO - Setting up needed data
2023-07-13 05:42:14,187 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-13 05:42:14,548 - web_server - INFO - Retrieving weather forecast data using method = list
2023-07-13 05:42:14,557 - web_server - INFO - >> Performing naive MPC optimization...
2023-07-13 05:42:14,557 - web_server - INFO - Performing naive MPC optimization
2023-07-13 05:42:14,592 - web_server - INFO - Perform an iteration of a naive MPC controller
2023-07-13 05:42:14,611 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2190, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1486, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 199, in action_call
opt_res = naive_mpc_optim(input_data_dict, app.logger)
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 275, in naive_mpc_optim
opt_res_naive_mpc = input_data_dict['opt'].perform_naive_mpc_optim(
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 573, in perform_naive_mpc_optim
self.opt_res = self.perform_optimization(df_input_data, P_PV.values.ravel(), P_load.values.ravel(),
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 271, in perform_optimization
rhs = def_total_hours[k]*self.optim_conf['P_deferrable_nom'][k])
TypeError: 'int' object is not subscriptable
2023-07-13 05:42:14,623 - web_server - INFO - Setting up needed data
2023-07-13 05:42:14,626 - web_server - INFO - >> Publishing data...
2023-07-13 05:42:14,626 - web_server - INFO - Publishing data to HASS instance
2023-07-13 05:42:14,649 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = -13.5
2023-07-13 05:42:14,662 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 997.03
2023-07-13 05:42:14,675 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2023-07-13 05:42:14,688 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 1010.53
2023-07-13 05:42:14,701 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 62.55
2023-07-13 05:42:14,715 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 0.0
2023-07-13 05:42:14,727 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = 1.66
2023-07-13 05:42:14,741 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 0.24
2023-07-13 05:42:14,753 - web_server - INFO - Successfully posted to sensor.unit_prod_price = 0.15
Cheers
Tom
These variables are different to your payload above.
Maybe set P_deferrable_nom to the expected power consumption value in W of your deferrable load, e.g. 1500 for a 1.5 kW pump?
sometimes I wonder about the analysis.
Seems to be cycling the battery up to 70% overnight at a loss.
Charges at a general price of 16c to 17c at 2:30 am through to 5am and then starts to discharge the battery at 5:20am at as low as 8c FiT? Perhaps I’ve got some data feed wrong?
Not sure what these settings whould be:
Manual says:
- set_battery_dynamic: Set a power dynamic limiting condition to the battery power. This is an additional constraint
on the battery dynamic in power per unit of time. - battery_dynamic_max: The maximum positive battery power dynamic. This is the power variation in percentage
of battery maximum power. - battery_dynamic_min: The minimum negative battery power dynamic. This is the power variation in percentage
of battery maximum power.
No idea what that means?