Greetings everyone.
For those who want to develop on EMHASS/EMHASS-Add-on, some documents have been updated that may help:
Thanks, but can you have a look at the code you posted again? The syntax seems to get messed up by the forum for some reason, and I’m not good enough with code to fix it despite my best efforts 
@ThorAlex ,
sorry. I used the wrong format.
- name: entsoe_prices_forecast_30
unique_id: entsoe_prices_forecast_30
state: >
{{ '30 minute entsoe forecast prices'}}
attributes:
prices: >
{% set today = now().strftime('%Y-%m-%d') %}
{% set tomorrow = (now().timestamp()+86400) | timestamp_custom('%Y-%m-%d') %}
{% set x = (state_attr('sensor.entsoe_average_electricity_price_today', 'prices') | selectattr('time', 'match', today ) | map(attribute='price') | list)[now().hour:]
+ state_attr('sensor.entsoe_average_electricity_price_today', 'prices') | selectattr('time', 'match', tomorrow ) | map(attribute='price') | list %}
{% set ns = namespace(z=[]) %}
{% for i in x %}
{% if loop.index == 1 and now().minute >29 %}
{% set ns.z = ns.z + [i] %}
{% else %}
{% set ns.z = ns.z + [i] + [i] %}
{% endif %}
{% endfor %}
{% set l = ns.z|length %}
{% set y = states('sensor.entsoe_average_electricity_price_today')|float %}
{% if l < 48 %}
{% for i in range(0,48-l) %}
{% set ns.z = ns.z + [y] %}
{% endfor %}
{% endif %}
{% set ns.z = ns.z[:48] %}
{{ ns.z }}
And the mpc-optim
\"load_cost_forecast\": {{ state_attr('sensor.entsoe_prices_forecast_30', 'prices') |tojson }},
\"prod_price_forecast\": {{ state_attr('sensor.entsoe_prices_forecast_30', 'prices') |tojson }},
\"pv_power_forecast\": {{ ([states('sensor.apf_solar_entity')|int(0)] + state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast') | selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list + state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedForecast') | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list)[:48] |tojson }},
\"prediction_horizon\": {{ min(48, state_attr('sensor.entsoe_prices_forecast_30', 'prices')|length |int) |tojson }},
Hope this will work for you. If you have any questions, feel free…
Hi,
I did the latest HA update today.
I was checking logs after that and I saw this.
Not sure since when this started.
Logger: homeassistant.components.automation
Source: components/automation/__init__.py:1019
Integration: Automatisering (documentation, issues)
First occurred: 13:35:00 (152 occurrences)
Last logged: 16:40:00
Error evaluating condition in 'Afwasmachine aan Emhass': In 'condition': In 'numeric_state': In 'numeric_state' condition: unknown entity sensor.p_deferrable1
Error evaluating condition in 'Wasmachine aan Emhass': In 'condition': In 'numeric_state': In 'numeric_state' condition: unknown entity sensor.p_deferrable0
Error evaluating condition in 'afwasmachine uit Emhass': In 'condition': In 'numeric_state': In 'numeric_state' condition: unknown entity sensor.p_deferrable1
Error evaluating condition in 'Wasmachine uit Emhass': In 'condition': In 'numeric_state': In 'numeric_state' condition: unknown entity sensor.p_deferrable0
That’s because sensor.p_deferrableX is unknown until your rest/shell command the first time publishes data.
I’m currently stuck on 0.4.2 of the addon, updating to any version after that causes the same error I see below
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 12, in <module>
import pandas as pd
File "/usr/local/lib/python3.11/dist-packages/pandas/__init__.py", line 16, in <module>
raise ImportError(
ImportError: Unable to import required dependencies:
numpy: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
I checked previous issues that had this in the GitHub repo as well as on this post and nothing has helped.
I’ve tried a clean install, removing the docker images as suggested in the forums and nothing seems to help.
I’m running HA OS on a ASUS Tinker Board (which is armv7l which I believe is still support unlike armhf). Checking the docker image and it is using the armv7 image.
Any ideas on how to get this working on newer versions, or is armv7l also no longer supported?
Happy to provide any other information needed.
@davidusb I get this after the latest emhass update.
I first tried a model fit.
2024-03-01 12:04:05,435 - web_server - INFO - >> Setting input data dict
2024-03-01 12:04:05,435 - web_server - INFO - Setting up needed data
2024-03-01 12:04:05,444 - web_server - INFO - Retrieve hass get data method initiated...
2024-03-01 12:04:12,986 - web_server - INFO - >> Performing perfect optimization...
2024-03-01 12:04:12,987 - web_server - INFO - Performing perfect forecast optimization
2024-03-01 12:04:12,998 - web_server - INFO - Perform optimization for perfect forecast scenario
2024-03-01 12:04:13,002 - web_server - INFO - Solving for day: 21-2-2024
2024-03-01 12:04:13,009 - web_server - ERROR - Exception on /action/perfect-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 118, in action_call
opt_res = perfect_forecast_optim(input_data_dict, app.logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 207, in perfect_forecast_optim
opt_res = input_data_dict['opt'].perform_perfect_forecast_optim(df_input_data, input_data_dict['days_list'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/optimization.py", line 535, in perform_perfect_forecast_optim
data_tp = df_input_data.copy().loc[pd.date_range(start=day_start, end=day_end, freq=self.freq)]
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexing.py", line 1103, in __getitem__
return self._getitem_axis(maybe_callable, axis=axis)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexing.py", line 1332, in _getitem_axis
return self._getitem_iterable(key, axis=axis)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexing.py", line 1272, in _getitem_iterable
keyarr, indexer = self._get_listlike_indexer(key, axis)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexing.py", line 1462, in _get_listlike_indexer
keyarr, indexer = ax._get_indexer_strict(key, axis_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 5877, in _get_indexer_strict
self._raise_if_missing(keyarr, indexer, axis_name)
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 5941, in _raise_if_missing
raise KeyError(f"{not_found} not in index")
KeyError: "[Timestamp('2024-02-21 12:00:00+0100', tz='UTC+01:00'), Timestamp('2024-02-21 12:30:00+0100', tz='UTC+01:00'), Timestamp('2024-02-21 13:00:00+0100', tz='UTC+01:00'), Timestamp('2024-02-21 13:30:00+0100', tz='UTC+01:00'), Timestamp('2024-02-21 14:00:00+0100', tz='UTC+01:00'), Timestamp('2024-02-21 14:30:00+0100', tz='UTC+01:00')] not in index"
I have this error after last update
(mpc update)
My PV power forecast is more then 24, this hasn’t been a problem in the past.
Is this the problem of the error?
Edit: updating all arrays to 24, doesn’t fix it
2024-03-01 16:50:03,601 - web_server - INFO - Retrieving data from hass for load forecast using method = mlforecaster
2024-03-01 16:50:03,601 - web_server - INFO - Retrieve hass get data method initiated...
2024-03-01 16:50:05,623 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 103, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 127, in set_input_data_dict
P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'], set_mix_forecast=True, df_now=df_input_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 626, in get_load_forecast
data = pd.DataFrame.from_dict(data_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/frame.py", line 1760, in from_dict
return cls(data, index=index, columns=columns, dtype=dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/frame.py", line 709, in __init__
mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/construction.py", line 481, in dict_to_mgr
return arrays_to_mgr(arrays, columns, index, dtype=dtype, typ=typ, consolidate=copy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/construction.py", line 115, in arrays_to_mgr
index = _extract_index(arrays)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/construction.py", line 655, in _extract_index
raise ValueError("All arrays must be of the same length")
ValueError: All arrays must be of the same length
This is my output rest
{
"pv_power_forecast": [
202,
64,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
90,
615,
1518,
1958,
2027,
1900,
1386,
857,
429,
239,
200,
64,
0,
0,
0,
0,
0
],
"prod_price_forecast": [
0.0725,
0.0894,
0.0919,
0.0872,
0.0775,
0.0716,
0.0701,
0.0664,
0.06,
0.056,
0.056,
0.0555,
0.0549,
0.056,
0.0566,
0.0607,
0.0638,
0.0658,
0.0604,
0.0611,
0.0578,
0.0549,
0.0599,
0.0661
],
"load_cost_forecast": [
0.1928,
0.2107,
0.2134,
0.2084,
0.1981,
0.1919,
0.1902,
0.1863,
0.1795,
0.1754,
0.1753,
0.1748,
0.1741,
0.1753,
0.1759,
0.1803,
0.1835,
0.1857,
0.18,
0.1807,
0.1773,
0.1742,
0.1794,
0.186
],
"soc_init": 0.96,
"prediction_horizon": 24
}
Resolved itself After a new model fit.
Strange anyway
I’m also seeing this exact issue on a completely fresh install. From at least 0.6.6 to 0.8.1 davidusb/image-armv7-emhass:0.8.1 on an odroid-xu4.
I had some problems with my recorder database, I had removed the emhass related files in the share folder to get emhass working again.
Emhass works, but there are no new files created in the share folder.
We are having a hard time supporting armhf and recently armv7 archs.
Hopefully full support for armv7 will be back on track soon.
This was changed recently. Now the files are saved internally in the docker container and they are persistent on reboot and add-on updates tasks.
We can still access them if we need to by entering inside the container and looking at the /data folder.
Is the container dedicated to the add-on instance as well?
So might I be successful this time, if I try to install EMHASS add-on twice as the executions, files from one instance would not interfere/overwrite the ones from the other?
Yeah sure. And you can have as many instances as you want. You can easily achieve this using it as a local add-on. Copy the contents of the add-on repo in the add-on folder on HA. Change the add-on version number in config.yaml to something like version: 0.8.1.instanceA and do the same for other instances B, C, D, etc.
These instances should appear in the Add-on Store page as local add-ons.
I guess any update will have to be managed manually, in such scenario. Right?
I just had a quick question on the new def load start/end config parameters. They are a step number in the current prediction horizon. So if you regularly call the MPC optimiser and you want your def loads to start and stop at the same “real time” each day, then you need to recalculate you start/end timesteps to reflect the current time when the MPC optimiser is run? That is a long question, so let me try again. If I want my pool pumps to only be scheduled between 7:00 and 17:00 then I need to keep recalculating def_start_timestep and def_end_timestep such that their timestep values equate to 7:00 and 17:00 in real time?
I can see this being quick complex given the need to keep a constant real time window for loads being scheduled while there is a sliding prediction window going on. Has anyone written example code for this?
I never have low prices overnight, so my pool only ever gets scheduled during the day.
I do however use the end timestep in the manner you outlined for my EV charging. I have an input time helper to set the stop time. Above you can see the EV charging scheduled before 11am, but the pool heater is unconstrained so gets scheduled for the afternoon.
I then have an helper with the following template value:
sensor.emhass_deferrable2_end_timeslots
{% set current_time = now() %}
{% set sensor_time_str = states('input_datetime.emhass_p_deferable_2_end_time') %}
{% set sensor_time = current_time.replace(hour=0, minute=0, second=0, microsecond=0) + timedelta(hours=(sensor_time_str.split(':')[0]|int), minutes=(sensor_time_str.split(':')[1]|int)) %}
{% if sensor_time <= current_time %}
{% set sensor_time = sensor_time + timedelta(days=1) %}
{% endif %}
{% set time_difference = sensor_time - current_time %}
{{ time_difference.total_seconds() / 3600 * 2 }}
Then I use the following payload in my MPC call:
"def_end_timestep":[0, 0,
{{max(0,states('sensor.emhass_deferrable2_end_timeslots')|int(0))}},
0, 0, 0],
Awesome as usual Mark. I will find a place to add this to the EMHASS docs.