Another question out of interest
Does the EMHASS model makes a prediction based on all past data and does it distinguish the day of the week?
For example, on weekdays we have a different load profile than weekend.
Can it predict this?
Or is it better in this case to make manual load profiles (csv or list) and then pass them to EMHASS?
Yes definetly. This is actually one the good reasons to use this ML model. These variables are passed as covariates: year, month of the year, day_of_week, day_of_year, day, hour.
So along with the history data of your sensor, these variables will be also used to generate the future forecasts.
Thanks,
Because I always have the feeling it is based on day-1.
Is this because I do a naive-mpc-optim every 5 minutes? should I turn this off, or is there another way to pass SOC to EMHASS?
2024-02-01 11:56:11,391 - web_server - ERROR - The retrieved JSON is empty, check that correct day or variable names are passed
2024-02-01 11:56:11,392 - web_server - ERROR - Either the names of the passed variables are not correct or days_to_retrieve is larger than the recorded history of your sensor (check your recorder settings)
2024-02-01 11:56:11,392 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File “/usr/local/lib/python3.11/dist-packages/flask/app.py”, line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.11/dist-packages/flask/app.py”, line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.11/dist-packages/flask/app.py”, line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.11/dist-packages/flask/app.py”, line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.11/dist-packages/emhass/web_server.py”, line 50, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.11/dist-packages/emhass/command_line.py”, line 120, in set_input_data_dict
P_load_forecast = fcst.get_load_forecast(method=optim_conf[‘load_forecast_method’], set_mix_forecast=True, df_now=df_input_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.11/dist-packages/emhass/forecast.py”, line 586, in get_load_forecast
rh.get_data(days_list, var_list)
File “/usr/local/lib/python3.11/dist-packages/emhass/retrieve_hass.py”, line 150, in get_data
self.df_final = pd.concat([self.df_final, df_day], axis=0)
^^^^^^
UnboundLocalError: cannot access local variable ‘df_day’ where it is not associated with a value
Any idea how to resolve? I didn’t change anything
By default it only uses day -1, if you want to use more history you need to train and utilise the machine learning forecaster.
https://emhass.readthedocs.io/en/latest/mlforecaster.html
You don’t need to turn off MPC for this feature.
@davidusb Hi,
Tried using solar.forecast instead of scrapper but I get this error.
KeyError: 'solar_forecast_kwp'
s6-supervise emhass: warning: unable to spawn ./finish: Permission denied
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 222, in <module>
params = build_params(params, params_secrets, options, args.addon, app.logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/utils.py", line 612, in build_params
params['retrieve_hass_conf']['solar_forecast_kwp'] = options.get('optional_solar_forecast_kwp',params['retrieve_hass_conf']['solar_forecast_kwp'])
Does this mean you always need to start the optimization of a deferrable load with 0 W?
If yes, does this meant it will work only with the day-ahead optim, as you are not shifting the time you are running the elaboration, which with MPC would sooner or later correspond to the moment you want to start the load, hence power not being zero anymore and potentially causing the load to be split?
All the sensors contains data. See also this screenshot:
costfun: profit
logging_level: DEBUG
set_total_pv_sell: false
set_nocharge_from_grid: false
set_nodischarge_to_grid: false
sensor_power_photovoltaics: sensor.solaredge_ac_power
sensor_power_load_no_var_loads: sensor.house_usage_in_watt
number_of_deferrable_loads: 2
list_nominal_power_of_deferrable_loads:
- nominal_power_of_deferrable_loads: 800
- nominal_power_of_deferrable_loads: 10800
list_operating_hours_of_each_deferrable_load:
- operating_hours_of_each_deferrable_load: 2
- operating_hours_of_each_deferrable_load: 4
list_start_timesteps_of_each_deferrable_load:
- start_timesteps_of_each_deferrable_load: 0
- start_timesteps_of_each_deferrable_load: 0
list_end_timesteps_of_each_deferrable_load:
- end_timesteps_of_each_deferrable_load: 0
- end_timesteps_of_each_deferrable_load: 0
list_peak_hours_periods_start_hours:
- peak_hours_periods_start_hours: "02:54"
- peak_hours_periods_start_hours: "17:24"
list_peak_hours_periods_end_hours:
- peak_hours_periods_end_hours: "15:24"
- peak_hours_periods_end_hours: "20:24"
list_treat_deferrable_load_as_semi_cont:
- treat_deferrable_load_as_semi_cont: false
- treat_deferrable_load_as_semi_cont: false
list_set_deferrable_load_single_constant:
- set_deferrable_load_single_constant: false
- set_deferrable_load_single_constant: false
load_peak_hours_cost: 0.1907
load_offpeak_hours_cost: 0.1419
photovoltaic_production_sell_price: 0.065
maximum_power_from_grid: 17250
list_pv_module_model:
- pv_module_model: Jinko_Solar_Co___Ltd_JKM410M_72HL
list_pv_inverter_model:
- pv_inverter_model: SolarEdge_Technologies_Ltd___SE9K__208V_
list_surface_tilt:
- surface_tilt: 45
list_surface_azimuth:
- surface_azimuth: 210
list_modules_per_string:
- modules_per_string: 26
list_strings_per_inverter:
- strings_per_inverter: 1
set_use_battery: true
battery_nominal_energy_capacity: 14336
hass_url: empty
long_lived_token: empty
optimization_time_step: 60
historic_days_to_retrieve: 2
method_ts_round: first
lp_solver: COIN_CMD
lp_solver_path: /usr/bin/cbc
set_battery_dynamic: false
battery_dynamic_max: 0.9
battery_dynamic_min: -0.9
load_forecast_method: naive
battery_discharge_power_max: 3400
battery_charge_power_max: 3400
battery_discharge_efficiency: 0.94
battery_charge_efficiency: 0.94
battery_minimum_state_of_charge: 0.15
battery_maximum_state_of_charge: 0.9
battery_target_state_of_charge: 0.15
No, it means that we need to find a way to count that deferrable starting at a value different than zero as a startup occurrence. But of course the first value can be either zero or non zero, so it should be compatible for all: day-ahead, MPC, etc.
What is your load forecast method?
{
"prod_price_forecast": {{
((state_attr('sensor.nordpool_purchase_price', 'raw_today') | map(attribute='value') | list + state_attr('sensor.nordpool_purchase_price', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24]
| tojson
}},
"load_cost_forecast": {{
((state_attr('sensor.nordpool_purchase_price', 'raw_today') | map(attribute='value') | list + state_attr('sensor.nordpool_purchase_price', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24]
| tojson
}},
"pv_power_forecast": {{
([states('sensor.solaredge_ac_power')|int(0)] +
state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedHourly')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list +
state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedHourly')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list
)| tojson
}},
"prediction_horizon": {{
min(24, (((state_attr('sensor.nordpool_purchase_price', 'raw_today')|map(attribute='value')|list + state_attr('sensor.nordpool_purchase_price', 'raw_tomorrow') | map(attribute='value')| list)[now().hour:][:24]|list|length)))
}},
"alpha": 1,
"beta": 0,
"num_def_loads": 2,
"def_total_hours": [{{states('input_number.def_total_hours_ww')}},{{states('input_number.def_total_hours_ev')}}],
"treat_def_as_semi_cont": [0,0],
"set_def_constant": [0,0],
"soc_init": {{ (states('sensor.sofarcontroller_battery_soc')|int(0))/100 }}
}
{
"prod_price_forecast": [
0.259,
0.255,
0.29,
0.29,
0.272,
0.263,
0.254,
0.253,
0.242,
0.231,
0.226,
0.224,
0.22,
0.218,
0.22,
0.234,
0.248,
0.262,
0.258,
0.251,
0.244,
0.232,
0.229,
0.231
],
"load_cost_forecast": [
0.259,
0.255,
0.29,
0.29,
0.272,
0.263,
0.254,
0.253,
0.242,
0.231,
0.226,
0.224,
0.22,
0.218,
0.22,
0.234,
0.248,
0.262,
0.258,
0.251,
0.244,
0.232,
0.229,
0.231
],
"pv_power_forecast": [
507,
2102,
2,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
26,
218,
548,
744,
773,
707,
560,
307,
105,
2,
0,
0,
0,
0,
0,
0
],
"prediction_horizon": 24,
"alpha": 1,
"beta": 0,
"num_def_loads": 2,
"def_total_hours": [
0,
0
],
"treat_def_as_semi_cont": [
0,
0
],
"set_def_constant": [
0,
0
],
"soc_init": 0.68
}
But after the prices for tomorrow (available today since 13.30) it’s working again. Never had this issue before.
That didn’t answered the question about the load forecast method. But if it is working again it’s ok. So it’s probably related to the availability of those data that you are passing as lists.
Definetly a bug to be fixed!
There were a lot of changes latetly on how we deal with passed configuration parameters and default options. This by-passed by the current tests.
sorry, using the naive method
There is currently an issue with both solcast and solar.forcast parameters being passed in via the add-on config options.
Hopefully both of the current pull requests will fix this issue. Sorry about that, I should be more clear here when I find an issue.
(#169 is a predecessor of #167)
I have a problem:
the command:
curl -i -H "Content-Type: application/json" -X POST -d '{"set_def_constant":[true],"pv_power_forecast":[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],"def_total_hours":[4],"prediction_horizon":10,"load_cost_forecast":[16.895712000000003, 16.65222, 15.634992, 14.522592, 13.86504, 13.5684, 13.159284, 13.22232, 13.748856, 16.443336000000002],"prod_price_forecast":[9.24, 9.23, 9.22, 9.21, 9.2, 9.21, 9.22, 9.23, 9.24, 9.25]}' http://localhost:5000/action/naive-mpc-optim
the error:
2024-02-01 21:34:27,356 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2024-02-01 21:34:27,357 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2024-02-01 21:34:27,357 - web_server - INFO - The data path is: /share
2024-02-01 21:34:27,361 - web_server - INFO - Using core emhass version: 0.7.2
waitress INFO Serving on http://0.0.0.0:5000
2024-02-01 21:34:33,361 - web_server - INFO - Setting up needed data
2024-02-01 21:34:33,447 - web_server - INFO - Retrieve hass get data method initiated...
2024-02-01 21:34:33,989 - web_server - INFO - Retrieving weather forecast data using method = list
2024-02-01 21:34:33,992 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2024-02-01 21:34:33,993 - web_server - INFO - Retrieve hass get data method initiated...
2024-02-01 21:34:34,205 - web_server - ERROR - The inferred freq from data is not equal to the defined freq in passed parameters
2024-02-01 21:34:34,221 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 50, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 120, in set_input_data_dict
P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'], set_mix_forecast=True, df_now=df_input_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 598, in get_load_forecast
forecast_out.index = self.forecast_dates
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/generic.py", line 6002, in __setattr__
return object.__setattr__(self, name, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pandas/_libs/properties.pyx", line 69, in pandas._libs.properties.AxisProperty.__set__
File "/usr/local/lib/python3.11/dist-packages/pandas/core/generic.py", line 730, in _set_axis
self._mgr.set_axis(axis, labels)
File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/managers.py", line 225, in set_axis
self._validate_set_axis(axis, new_labels)
File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/base.py", line 70, in _validate_set_axis
raise ValueError(
ValueError: Length mismatch: Expected axis has 1 elements, new values have 10 elements
here my config:
costfun: profit
logging_level: INFO
set_total_pv_sell: false
set_nocharge_from_grid: false
set_nodischarge_to_grid: false
sensor_power_photovoltaics: sensor.reading_power_inverter_ac_output
sensor_power_load_no_var_loads: sensor.derived_power_consumption_total_no_var_loads
number_of_deferrable_loads: 1
list_nominal_power_of_deferrable_loads:
- nominal_power_of_deferrable_loads: 4000
list_operating_hours_of_each_deferrable_load:
- operating_hours_of_each_deferrable_load: 4
list_start_timesteps_of_each_deferrable_load:
- start_timesteps_of_each_deferrable_load: 0
list_end_timesteps_of_each_deferrable_load:
- end_timesteps_of_each_deferrable_load: 0
list_peak_hours_periods_start_hours:
- peak_hours_periods_start_hours: "00:00"
list_peak_hours_periods_end_hours:
- peak_hours_periods_end_hours: "06:00"
list_treat_deferrable_load_as_semi_cont:
- treat_deferrable_load_as_semi_cont: true
list_set_deferrable_load_single_constant:
- set_deferrable_load_single_constant: false
load_peak_hours_cost: 19.07
load_offpeak_hours_cost: 14.19
photovoltaic_production_sell_price: 12.464
maximum_power_from_grid: 20000
list_pv_module_model:
- pv_module_model: Canadian_Solar_Inc__CS3U_375P
list_pv_inverter_model:
- pv_inverter_model: Fronius_International_GmbH__Fronius_Symo_20_0_3_480__480V_
list_surface_tilt:
- surface_tilt: 38
list_surface_azimuth:
- surface_azimuth: 205
list_modules_per_string:
- modules_per_string: 66
list_strings_per_inverter:
- strings_per_inverter: 1
set_use_battery: false
battery_nominal_energy_capacity: 5000
hass_url: empty
long_lived_token: empty
optimization_time_step: 60
historic_days_to_retrieve: 7
method_ts_round: first
lp_solver: COIN_CMD
lp_solver_path: /usr/bin/cbc
set_battery_dynamic: false
battery_dynamic_max: 0.9
battery_dynamic_min: -0.9
weight_battery_discharge: 1
weight_battery_charge: 1
load_forecast_method: naive
battery_discharge_power_max: 1000
battery_charge_power_max: 1000
battery_discharge_efficiency: 0.95
battery_charge_efficiency: 0.95
battery_minimum_state_of_charge: 0.3
battery_maximum_state_of_charge: 0.9
battery_target_state_of_charge: 0.6
any Idea why this happens?
some background: I had an error today (1 hour ago) but I still was on version 0.5.4 (? - the latest before 0.6). sadly I did not capture it, I thought, it’s better to just upgrade to 0.6.1, as this might solve the issue. but it didn’t. I’m not sure if the error is the same now, at least it doesn’t work either…
Whilst trying to diagnose an issue Home Assistant was restored from a backup from a week ago…
Couldn’t get EMHASS to reload, and of course finally came to the realisation that the backup doesn’t contain the minimum of days days data.
The minimum 2 days is a blocker to new installs, but also who system restores, I’m wondering if there is another way to satisfy EMHASS during these initial setups.
The only reason to keep this 2 day data history need is to produce the needed forecasts: PV production, load consumption, PV sell price and load consumption cost.
If you are able to pass all these 4 forecasts as lists then this 2-day data restriction can be lifted. But I don’t think anyone is passing all these 4 forecasts. The load consumption forecast is the hardest, most people let EMHASS handle at least that one.
What method could be used to bypass this restriction?
Maybe a simple method like passing a daily mean value just for the temporary initial period after a fresh install or a backup as it was your case.
Open to any ideas
I guess it is a similar reason my case…
But in my case it was my own fault… 
I changed the load sensor of my heatpump, which is essential for the no_var_loads-sensor:
so the only workaround is to pass a list until there is enough history, correct?
Maybe somehow just use the oldest available data to substitute for any older data which is required but unavailable?
