You can setup two helper sensors to change the forecasts for both prices.
I presume one is feed in and the other is there import price.
This is what it means.
1 Like
Exctly, there is a good example sensor template code in this discussion: https://github.com/davidusb-geek/emhass/issues/203
For your module this seems like a good match. Same brand same power: Trina_Solar_TSM_465DEG15VC_20_II_
For the inverter this seems like a good match for hybrid and power:
SolaX_Power_Network_Technology__Zhe_jiang__Co___Ltd___A1_Hybrid_7_6_US__240V_
The workaround I have successfully implemented is that I also reduce the def_end_timestep to force the load to the beginning. This seems to be very buggy as the constant load then gets sometimes multiplied or devided by 2! Also the optimzer state then goes to infeasble.
This is a bug:
I have a template to work around this.
Let Deferrable load = x and def_end_timestep = y
If y = 2x the bug occurs. So:
“set_def_constant”: {{ ‘false’ if y <= x * 2) else ‘true’ }}
This is not a bug, it is just as this type of optimization work. We can’t add constraint upon contraint and expect that it will always go smoothly. As some point there is just too much constraint or constraints that are not compatible.
The problem has been identified but there is not much to do code wise. That issue will be closed soon
1 Like
Very strange. The 2024.4.4 is out, probably got fixed with that. Tried searching at the release notes but I did not found anything related.
Now that a lot of people does optimisation much more frequently than once a day, I was thinking how this could be improved without having to majorly rewrite EMHASS.
An idea would be to allow another parameter to be sent to EMHASS which tells EMHASS the deferrable is currently running so keep it running. EMHASS just has to put that in timeslot 0 and in successive timeslots till the job is finished. It could also send how much hows is left in that particular deferrable as EMHASS doesn’t keep state. This would be easy for us to send from HA as its easy to find out of the deferrable > 0W and when it last started. As an added bonus, this could be used to signify a job has completed for the day and not try to schedule it later.
eg.
4 hour job not started
def_remainder_time = [4]
4 hour job midway
def_remainder_time = [2]
4 hour job completed
def_remainder_time = [0]
If this requires too much rewrite, then just a simple flag stating whether the job is currently running and we use existing p_def_total_hours to send remaining hours left.
def_running = [1]
1 Like
Thanks Mark and David…
Will calling the day optimisation once whenever price changes over time be enough? unlike in Amber I call it like every 2-5 minutes?
Okey. To me it seem a little odd that you can’t use the MPC with def_end_timestep in combination with set_def_constant = true. Is it not the point with “receding horizon principle” ?
If you set set_def_constant = false everything works fine. So I must use the template hack to make my application work.
If this is how it will be I think the documentation needs to be updated so it’s clear that if you use set_def_constant = true you can’t use def_end_timestep in some cases.
Example:
If I have a load with def_total_hours = 6. If it allocate the 6 hours to the end of the window it will fail, look at the figure below. The spooky thing is that it will say that the optimisation is optimal when it’s not.
- 1 on this. Would really want the ability to tell EMHASS that I have a load running and I want it to run the whole timeslot.
Another thing:
If you run the optimisation every 5 minutes in some cases it can for example run the load 12:25 to 12:30. In my case this leads to 5 min run and 25 is missed. If I want a load to run 1 hour in the period and it does this two times I will get 10 min run instead of 60 min.
1 Like
Hi Mikael,
What I do for my pool pump (needs roughly 8 hours per day running) is calculate the amount it has run. So even if your example would happen, it would just schedule it again because the 8 hours aren’t done yet.
Example to calculate the amount of time a switch has been on:
- platform: history_stats
name: "pool_waterpump_hours_on_today"
entity_id: switch.eh_zwembad_pump_runstop
state: "on"
type: time
start: "{{ now().replace(hour=0, minute=0, second=0) }}"
end: "{{ now() }}"
I’m super frustrated by an issue that started out of the blue this morning. My naive-mpc-optim fails with an error stating that my no var loads sensor has no history, but I can for sure say that it does. I’m using emhass 0.8.6 docker standalone. The issue came out of the blue this morning. No restarts, no config changes no nothing.
I’ve restarted both emhass and ha. Can it be that emhass for some reason can’t connect to ha? No errors in ha logs.
2024-04-14 19:35:00,617 - web_server - INFO - Passed runtime parameters: {'prediction_horizon': 24, 'soc_init': 0.05, 'soc_final': 0.5, 'def_total_hours': [], 'load_cost_forecast': [0.55, 0.37, 0.35, 0.35, 0.35, 0.32, 0.33, 0.34, 0.35, 0.35, 0.59, 0.94, 2.45, 1.86, 1.07, 1.07, 1.01, 0.97, 0.83, 0.63, 0.78, 0.87, 1.0, 0.99], 'prod_price_forecast': [0.32, 0.14, 0.12, 0.12, 0.12, 0.09, 0.1, 0.11, 0.12, 0.12, 0.36, 0.71, 2.22, 1.63, 0.84, 0.84, 0.78, 0.74, 0.6, 0.4, 0.55, 0.64, 0.77, 0.76], 'pv_power_forecast': [109, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 222, 567, 1174, 2092, 3042, 4154, 5169, 6031, 6454, 6285, 5663, 4634, 2908]}
2024-04-14 19:35:00,617 - web_server - INFO - >> Setting input data dict
2024-04-14 19:35:00,617 - web_server - INFO - Setting up needed data
2024-04-14 19:35:00,620 - web_server - INFO - Retrieve hass get data method initiated...
2024-04-14 19:35:02,155 - web_server - INFO - Retrieving weather forecast data using method = list
2024-04-14 19:35:02,157 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2024-04-14 19:35:02,157 - web_server - INFO - Retrieve hass get data method initiated...
2024-04-14 19:35:02,227 - web_server - ERROR - The retrieved JSON is empty, A sensor:sensor.template_emhass_no_var_load_2 may have 0 days of history or passed sensor may not be correct
2024-04-14 19:35:02,229 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 108, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 128, in set_input_data_dict
df_input_data_dayahead = pd.concat([P_PV_forecast, P_load_forecast], axis=1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/reshape/concat.py", line 372, in concat
op = _Concatenator(
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/reshape/concat.py", line 462, in __init__
raise TypeError(msg)
TypeError: cannot concatenate object of type '<class 'bool'>'; only Series and DataFrame objs are valid
But the sensor exists and has a long history:
Great idea! Thank you. 
Guys my day ahead service has stopped working lately following this error bellow… Does anyone got it before?
2024-04-14 23:10:31,382 - web_server - INFO - Passed runtime parameters: {'set_def_constant': '[True, True, True]'}
2024-04-14 23:10:31,385 - web_server - INFO - >> Setting input data dict
2024-04-14 23:10:31,385 - web_server - INFO - Setting up needed data
2024-04-14 23:10:31,399 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 108, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 57, in set_input_data_dict
params, retrieve_hass_conf, optim_conf, plant_conf = utils.treat_runtimeparams(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/utils.py", line 298, in treat_runtimeparams
optim_conf['set_def_constant'] = [eval(str(k).capitalize()) for k in runtimeparams['set_def_constant']]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/utils.py", line 298, in <listcomp>
optim_conf['set_def_constant'] = [eval(str(k).capitalize()) for k in runtimeparams['set_def_constant']]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 1
[
^
SyntaxError: '[' was never closed
What does your dayahead-optim POST look like?
Take the template component from the dayahead-optim POST and see what it renders to in the developer tools template renderer.
Check the output is valid JSON format in a JSON format tester
Here are my shell commands:
dayahead_optim: 'curl -i -H ''Content-Type:application/json'' -X POST -d ''{"set_def_constant": "[True, True, True]"}'' http://localhost:5000/action/dayahead-optim'
publish_data: 'curl -i -H ''Content-Type:application/json'' -X POST -d ''{"set_def_constant": "[True, True, True]"}'' http://localhost:5000/action/publish-data'
The template didn’t render anything apparently!
Is there a good way to control if you want the hot water heater not to have to be turned on/off. I have a cut-off temperature that means that if the temperature is above 47 degrees, the geothermal heat pump does not start.
As it is today, it calculates that it should run every day even if it is not needed as a day when it wants to run now between 13-15 but it will not start because it is up to 55 degrees C, Then it can miss that it needs heat the water when it is below 47 degrees.
I have searched the forum and see that you have discussed the topic, but does anyone have a working solution to the problem that can show me examples.
I have seen a strange error in the past days (not happening at the moment).
does somebody know why?
moreover: is there a way to access a longer history of the log? I only see the last minutes/hours…
TypeError: Cannot compare dtypes datetime64[ns] and datetime64[ns, Europe/Vienna]
2024-04-13 07:00:00,361 - web_server - INFO - Passed runtime parameters: {}
2024-04-13 07:00:00,361 - web_server - INFO - >> Setting input data dict
2024-04-13 07:00:00,362 - web_server - INFO - Setting up needed data
2024-04-13 07:00:00,368 - web_server - INFO - >> Publishing data...
2024-04-13 07:00:00,368 - web_server - INFO - Publishing data to HASS instance
2024-04-13 07:00:00,378 - web_server - ERROR - Exception on /action/publish-data [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 115, in action_call
_ = publish_data(input_data_dict, app.logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 474, in publish_data
idx_closest = opt_res_latest.index.get_indexer([now_precise], method='ffill')[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 3740, in get_indexer
return self._get_indexer_non_comparable(target, method=method, unique=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 5990, in _get_indexer_non_comparable
raise TypeError(f"Cannot compare dtypes {self.dtype} and {other.dtype}")
TypeError: Cannot compare dtypes datetime64[ns] and datetime64[ns, Europe/Vienna]
2024-04-13 07:00:05,519 - web_server - INFO - Passed runtime parameters: {'set_def_constant': [True, True], 'pv_power_forecast': [1700.6, 2534.6000000000004, 3145.7999999999997, 5189.4, 7504.3, 9848.5, 12046.099999999999, 13966.300000000001, 15586.4, 16895.399999999998, 18093.9, 18766.2, 19143.5, 19171.1, 18994.7, 18526.9, 17510.899999999998, 16375.1, 15036.4, 13049.300000000001, 11129.300000000001, 9396.5, 7051.5, 3999.8], 'def_total_hours': [1.0, 0], 'prediction_horizon': 24, 'load_cost_forecast': [11.919599999999999, 11.919599999999999, 10.4244, 10.4244, 8.8956, 8.8956, 7.8372, 7.8372, 6.6432, 6.6432, 4.849200000000001, 4.849200000000001, 1.8396000000000008, 1.8396000000000008, 1.2384000000000004, 1.2384000000000004, 1.8504000000000014, 1.8504000000000014, 5.5920000000000005, 5.5920000000000005, 7.849200000000001, 7.849200000000001, 12.216, 12.216], 'prod_price_forecast': [9.26, 9.26, 9.25, 9.25, 9.24, 9.24, 9.23, 9.23, 9.22, 9.22, 9.21, 9.21, 9.2, 9.2, 9.21, 9.21, 9.22, 9.22, 9.23, 9.23, 9.24, 9.24, 9.25, 9.25]}
2024-04-13 07:00:05,520 - web_server - INFO - >> Setting input data dict
2024-04-13 07:00:05,520 - web_server - INFO - Setting up needed data
2024-04-13 07:00:05,529 - web_server - INFO - Retrieve hass get data method initiated...
2024-04-13 07:00:08,410 - web_server - INFO - Retrieving weather forecast data using method = list
2024-04-13 07:00:08,413 - web_server - INFO - Retrieving data from hass for load forecast using method = mlforecaster
2024-04-13 07:00:08,415 - web_server - INFO - Retrieve hass get data method initiated...
2024-04-13 07:00:42,568 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 108, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 127, in set_input_data_dict
P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'], set_mix_forecast=True, df_now=df_input_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 659, in get_load_forecast
forecast_out = mlf.predict(data_last_window)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/machine_learning_forecaster.py", line 205, in predict
exog = MLForecaster.generate_exog(data_last_window, self.lags_opt, self.var_model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/machine_learning_forecaster.py", line 103, in generate_exog
forecast_dates = pd.date_range(start=data_last_window.index[-1]+data_last_window.index.freq,
~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for +: 'Timestamp' and 'NoneType'
2024-04-13 07:05:00,363 - web_server - INFO - Passed runtime parameters: {}
2024-04-13 07:05:00,363 - web_server - INFO - >> Setting input data dict
2024-04-13 07:05:00,364 - web_server - INFO - Setting up needed data
2024-04-13 07:05:00,370 - web_server - INFO - >> Publishing data...
2024-04-13 07:05:00,371 - web_server - INFO - Publishing data to HASS instance
2024-04-13 07:05:00,378 - web_server - ERROR - Exception on /action/publish-data [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 115, in action_call
_ = publish_data(input_data_dict, app.logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 474, in publish_data
idx_closest = opt_res_latest.index.get_indexer([now_precise], method='ffill')[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 3740, in get_indexer
return self._get_indexer_non_comparable(target, method=method, unique=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 5990, in _get_indexer_non_comparable
raise TypeError(f"Cannot compare dtypes {self.dtype} and {other.dtype}")
TypeError: Cannot compare dtypes datetime64[ns] and datetime64[ns, Europe/Vienna]