Great idea! Thank you. 
Guys my day ahead service has stopped working lately following this error bellowā¦ Does anyone got it before?
2024-04-14 23:10:31,382 - web_server - INFO - Passed runtime parameters: {'set_def_constant': '[True, True, True]'}
2024-04-14 23:10:31,385 - web_server - INFO - >> Setting input data dict
2024-04-14 23:10:31,385 - web_server - INFO - Setting up needed data
2024-04-14 23:10:31,399 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 108, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 57, in set_input_data_dict
params, retrieve_hass_conf, optim_conf, plant_conf = utils.treat_runtimeparams(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/utils.py", line 298, in treat_runtimeparams
optim_conf['set_def_constant'] = [eval(str(k).capitalize()) for k in runtimeparams['set_def_constant']]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/utils.py", line 298, in <listcomp>
optim_conf['set_def_constant'] = [eval(str(k).capitalize()) for k in runtimeparams['set_def_constant']]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 1
[
^
SyntaxError: '[' was never closed
What does your dayahead-optim POST look like?
Take the template component from the dayahead-optim POST and see what it renders to in the developer tools template renderer.
Check the output is valid JSON format in a JSON format tester
Here are my shell commands:
dayahead_optim: 'curl -i -H ''Content-Type:application/json'' -X POST -d ''{"set_def_constant": "[True, True, True]"}'' http://localhost:5000/action/dayahead-optim'
publish_data: 'curl -i -H ''Content-Type:application/json'' -X POST -d ''{"set_def_constant": "[True, True, True]"}'' http://localhost:5000/action/publish-data'
The template didnāt render anything apparently!
Is there a good way to control if you want the hot water heater not to have to be turned on/off. I have a cut-off temperature that means that if the temperature is above 47 degrees, the geothermal heat pump does not start.
As it is today, it calculates that it should run every day even if it is not needed as a day when it wants to run now between 13-15 but it will not start because it is up to 55 degrees C, Then it can miss that it needs heat the water when it is below 47 degrees.
I have searched the forum and see that you have discussed the topic, but does anyone have a working solution to the problem that can show me examples.
I have seen a strange error in the past days (not happening at the moment).
does somebody know why?
moreover: is there a way to access a longer history of the log? I only see the last minutes/hoursā¦
TypeError: Cannot compare dtypes datetime64[ns] and datetime64[ns, Europe/Vienna]
2024-04-13 07:00:00,361 - web_server - INFO - Passed runtime parameters: {}
2024-04-13 07:00:00,361 - web_server - INFO - >> Setting input data dict
2024-04-13 07:00:00,362 - web_server - INFO - Setting up needed data
2024-04-13 07:00:00,368 - web_server - INFO - >> Publishing data...
2024-04-13 07:00:00,368 - web_server - INFO - Publishing data to HASS instance
2024-04-13 07:00:00,378 - web_server - ERROR - Exception on /action/publish-data [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 115, in action_call
_ = publish_data(input_data_dict, app.logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 474, in publish_data
idx_closest = opt_res_latest.index.get_indexer([now_precise], method='ffill')[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 3740, in get_indexer
return self._get_indexer_non_comparable(target, method=method, unique=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 5990, in _get_indexer_non_comparable
raise TypeError(f"Cannot compare dtypes {self.dtype} and {other.dtype}")
TypeError: Cannot compare dtypes datetime64[ns] and datetime64[ns, Europe/Vienna]
2024-04-13 07:00:05,519 - web_server - INFO - Passed runtime parameters: {'set_def_constant': [True, True], 'pv_power_forecast': [1700.6, 2534.6000000000004, 3145.7999999999997, 5189.4, 7504.3, 9848.5, 12046.099999999999, 13966.300000000001, 15586.4, 16895.399999999998, 18093.9, 18766.2, 19143.5, 19171.1, 18994.7, 18526.9, 17510.899999999998, 16375.1, 15036.4, 13049.300000000001, 11129.300000000001, 9396.5, 7051.5, 3999.8], 'def_total_hours': [1.0, 0], 'prediction_horizon': 24, 'load_cost_forecast': [11.919599999999999, 11.919599999999999, 10.4244, 10.4244, 8.8956, 8.8956, 7.8372, 7.8372, 6.6432, 6.6432, 4.849200000000001, 4.849200000000001, 1.8396000000000008, 1.8396000000000008, 1.2384000000000004, 1.2384000000000004, 1.8504000000000014, 1.8504000000000014, 5.5920000000000005, 5.5920000000000005, 7.849200000000001, 7.849200000000001, 12.216, 12.216], 'prod_price_forecast': [9.26, 9.26, 9.25, 9.25, 9.24, 9.24, 9.23, 9.23, 9.22, 9.22, 9.21, 9.21, 9.2, 9.2, 9.21, 9.21, 9.22, 9.22, 9.23, 9.23, 9.24, 9.24, 9.25, 9.25]}
2024-04-13 07:00:05,520 - web_server - INFO - >> Setting input data dict
2024-04-13 07:00:05,520 - web_server - INFO - Setting up needed data
2024-04-13 07:00:05,529 - web_server - INFO - Retrieve hass get data method initiated...
2024-04-13 07:00:08,410 - web_server - INFO - Retrieving weather forecast data using method = list
2024-04-13 07:00:08,413 - web_server - INFO - Retrieving data from hass for load forecast using method = mlforecaster
2024-04-13 07:00:08,415 - web_server - INFO - Retrieve hass get data method initiated...
2024-04-13 07:00:42,568 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 108, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 127, in set_input_data_dict
P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'], set_mix_forecast=True, df_now=df_input_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 659, in get_load_forecast
forecast_out = mlf.predict(data_last_window)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/machine_learning_forecaster.py", line 205, in predict
exog = MLForecaster.generate_exog(data_last_window, self.lags_opt, self.var_model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/machine_learning_forecaster.py", line 103, in generate_exog
forecast_dates = pd.date_range(start=data_last_window.index[-1]+data_last_window.index.freq,
~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for +: 'Timestamp' and 'NoneType'
2024-04-13 07:05:00,363 - web_server - INFO - Passed runtime parameters: {}
2024-04-13 07:05:00,363 - web_server - INFO - >> Setting input data dict
2024-04-13 07:05:00,364 - web_server - INFO - Setting up needed data
2024-04-13 07:05:00,370 - web_server - INFO - >> Publishing data...
2024-04-13 07:05:00,371 - web_server - INFO - Publishing data to HASS instance
2024-04-13 07:05:00,378 - web_server - ERROR - Exception on /action/publish-data [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 115, in action_call
_ = publish_data(input_data_dict, app.logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 474, in publish_data
idx_closest = opt_res_latest.index.get_indexer([now_precise], method='ffill')[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 3740, in get_indexer
return self._get_indexer_non_comparable(target, method=method, unique=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 5990, in _get_indexer_non_comparable
raise TypeError(f"Cannot compare dtypes {self.dtype} and {other.dtype}")
TypeError: Cannot compare dtypes datetime64[ns] and datetime64[ns, Europe/Vienna]
I donāt know if I understood correctly, but I think you are talking about the hysteresis. So, if your water is still hot enough (47Ā°C and above), it will not start, correct?
I solved the problem, by setting the target temperature im my automation (triggered by the deferrable sensor) to a much higher value for some minutes. This will force it to start, and when itās running, you can decrease the target temperature again. At least my heatpump then will run until the target temperature is met.
I continue on this story.
If I try to use the sensor from the UI to run an ML forecaster, I get the same error:
Iāve tested making a CURL call to my HA REST API from within the container an it works well:
root@2b43e81d5699:/app# curl -H "Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" -H "Content-Type: application/json" 'https://my.homeassistant.url:8123/api/states/sensor.template_emhass_no_var_load_2' |python3 -m json.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 312 100 312 0 0 4517 0 --:--:-- --:--:-- --:--:-- 4588
{
"entity_id": "sensor.template_emhass_no_var_load_2",
"state": "1162.89",
"attributes": {
"unit_of_measurement": "W",
"device_class": "power"
},
"last_changed": "2024-04-15T14:07:46.406620+00:00",
"last_updated": "2024-04-15T14:07:46.406620+00:00",
"context": {
"id": "01HVH1BP74Q72AZJ8Q7MAD7JCT",
"parent_id": null,
"user_id": null
}
}
So apparently, no connection or authorization issues either. Querying the sensor history works just as well, but itās a bit too much to post here. 
Why has emhass suddenly stopped recognizing my sensor and itās history?
Unfortunately my pump is not that smart and I can block the production and force it to start but at the same time it feels unnecessary if it is not needed.
I would like to have a kind of flexible plan that when it is below 47 degrees that within say 2-6 hours, it should start and if it is above 47 nothing at all.
Hi Johan, I think you should tackle that within HA. I do more or less the same for my electric car. If it is connected & needs charging for more than X hours, HA will put that into the def_total_hours for deferable load 0.
ok do you have an example you can share?
I answer myself.
I think I solved it now
Made a template that shows 0 if water heater temp is above 47 degrees and 3 if it is below.
(It seems to work but donāt know if emhass likes the 0 hour)
Submitting the value with
ādef_total_hoursā:[{{ states(āsensor.p_deferrable0_hoursā) | default(0) }},1]
However, I have an override that checks so that the water heater does not fall below 39 degrees, then it forces the production of hot water
2 Likes
This is exactly the correct way to do this.
EMHASS actually loves 0 run hours as it means it doesnāt have to schedule the additional load.
You can even get more sophisticated and charge the rules to gradually increase the run hours as the temperature drops.
Over 47 Ā°C = 0 run hours
46Ā° = 1 hour
45Ā° = 2 hours
44Ā° and below = 3 hours
In my location I also need to ensure the water temp gets to 60Ā°C at least once a week to kill all bacteria, Iām not sure if that is a requirement for you.
1 Like
The optimal frequency to call the optimisation depends on how quickly the external factors are changing.
If your prices are fixed for the next 24 hours, you have a stable household and solar profile. Then calling the optimisation once a day and publishing those results every five minutes will give you excellent results. If your case if the price windows are known in advance and donāt change, then you donāt even need to call the optimisation at each price change, as long as they are reflected in the price forecast.
The reason with Amber that I call MPC frequently is the Amber price forecast for 6pm maybe initially 30 Ā¢/ kWh, but then that forecast changes, sometimes the forecast for 6pm goes down to 20 Ā¢/ kWh other times it goes up to 10 $/ kWh. Every time that forecast changes I need to update my optimisation.
With your TOU schedule you know in advance that the 6pm price is always going to be 47 Ā¢/ kWh and it never changes.
1 Like
Wow, Iāll try to test that.
legionella is automatically run by the pump once a week
When I started using EMHASS I had a lot if issues with history. I couldnāt restart Home Assistant without then having to wait 2 days before I could run MPC Optimisation again. Had to survive with day-ahead for two days.
This didnāt stop until I deleted my recorder database and started afresh. At that time I also started restricting the size of the recorder database by only including the entities of interest to me and purging to 10 days.
Now with a much smaller recorder db that issue hasnāt returned.
By the way, I came up with one thing, donāt I have to put a timer in the automation so that the automation doesnāt switch off when it gets over 47, 46, 45 degrees?
Yes, I had similar issues initially, but havenāt had any issues since.
The weird thing is that I can get full history for the sensor if I call HA REST API using curl, even from within the emhass container, but emhass still thinks the returned json is empty. So something that emhass does is different from what my curl command does.
Iāll see if I can set up emhass in a test environment to debug it and see what on earth it actually does.
I even tested setting up a second no_var_load sensor, but same issue with that. Emhass canāt read its history.
Perfect working!