I need som help with mlforecast. Emhass addon gives error about “All arrays must be of the same length” when I run dayahead-optim when I use load forecast method “mlforecaster”.
2023-07-24 22:07:59,185 - web_server - INFO - The data path is: /share
2023-07-24 22:07:59,186 - web_server - INFO - Using core emhass version: 0.4.14
waitress INFO Serving on http://0.0.0.0:5000
2023-07-24 22:08:03,444 - web_server - INFO - Setting up needed data
2023-07-24 22:08:03,471 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-07-24 22:08:05,384 - web_server - INFO - Retrieving data from hass for load forecast using method = mlforecaster
2023-07-24 22:08:05,385 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-24 22:08:07,717 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2190, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1486, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 174, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 91, in set_input_data_dict
P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'])
File "/usr/local/lib/python3.9/dist-packages/emhass/forecast.py", line 620, in get_load_forecast
data = pd.DataFrame.from_dict(data_dict)
File "/usr/local/lib/python3.9/dist-packages/pandas/core/frame.py", line 1677, in from_dict
return cls(data, index=index, columns=columns, dtype=dtype)
File "/usr/local/lib/python3.9/dist-packages/pandas/core/frame.py", line 636, in __init__
mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
File "/usr/local/lib/python3.9/dist-packages/pandas/core/internals/construction.py", line 502, in dict_to_mgr
return arrays_to_mgr(arrays, columns, index, dtype=dtype, typ=typ, consolidate=copy)
File "/usr/local/lib/python3.9/dist-packages/pandas/core/internals/construction.py", line 120, in arrays_to_mgr
index = _extract_index(arrays)
File "/usr/local/lib/python3.9/dist-packages/pandas/core/internals/construction.py", line 674, in _extract_index
raise ValueError("All arrays must be of the same length")
ValueError: All arrays must be of the same length
When I use “naive” as load forecast method I do not have any errors:
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun emhass (no readiness notification)
s6-rc: info: service legacy-services successfully started
2023-07-24 22:10:12,784 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2023-07-24 22:10:12,784 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2023-07-24 22:10:12,784 - web_server - INFO - The data path is: /share
2023-07-24 22:10:12,785 - web_server - INFO - Using core emhass version: 0.4.14
waitress INFO Serving on http://0.0.0.0:5000
2023-07-24 22:10:16,701 - web_server - INFO - Setting up needed data
2023-07-24 22:10:16,722 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-07-24 22:10:18,520 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-07-24 22:10:18,521 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-24 22:10:20,967 - web_server - INFO - >> Performing dayahead optimization...
2023-07-24 22:10:20,967 - web_server - INFO - Performing day-ahead forecast optimization
2023-07-24 22:10:20,976 - web_server - INFO - Perform optimization for the day-ahead
2023-07-24 22:10:21,073 - web_server - INFO - Status: Optimal
2023-07-24 22:10:21,073 - web_server - INFO - Total value of the Cost function = -63.34
I use nordpool as source for “load_cost_forecast” and “prod_price_forecast”. Nordpool integration publish 24 hour price data (price for every hour).
I use this shell_command for setting up mlforecast with num_lags: 24 because the time step is 1 hour:
# I run this first. It gives no errors. Using default mlforecast config.
shell_command:
ml_forecast_fit: "curl -i -H \"Content-Type:application/json\" -X POST -d '{\"num_lags\": 24}' http://localhost:5000/action/forecast-model-fit"
# Then I run this after. It gives no errors.
ml_forecast_tune: "curl -i -H \"Content-Type:application/json\" -X POST -d '{\"num_lags\": 24}' http://localhost:5000/action/forecast-model-tune"
I run dayahead-optim each day at 13:55. Nordpool publish prices every day at 13:15
shell_command:
trigger_nordpool_forecast: "curl -i -H \"Content-Type: application/json\" -X POST -d '{
\"load_cost_forecast\":{{((state_attr('sensor.nordpool', 'raw_today') | map(attribute='value') | list + state_attr('sensor.nordpool', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24] }},
\"prod_price_forecast\":{{((state_attr('sensor.nordpool_uten_avgifter', 'raw_today') | map(attribute='value') | list + state_attr('sensor.nordpool_uten_avgifter', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24]}},
\"def_total_hours\":{{states('sensor.list_operating_hours_of_each_deferrable_load')}}
}' http://localhost:5000/action/dayahead-optim"
Here is my emhass config:
root@5b918bf2-emhass:/data# cat options.json
{
"hass_url": "empty",
"long_lived_token": "empty",
"costfun": "profit",
"logging_level": "INFO",
"optimization_time_step": 60,
"historic_days_to_retrieve": 2,
"method_ts_round": "nearest",
"set_total_pv_sell": false,
"lp_solver": "COIN_CMD",
"lp_solver_path": "/usr/bin/cbc",
"set_nocharge_from_grid": false,
"set_nodischarge_to_grid": false,
"set_battery_dynamic": false,
"battery_dynamic_max": 0.9,
"battery_dynamic_min": -0.9,
"load_forecast_method": "naive", # I change this setting to "mlforecaster" when I try to use mlforecast
"sensor_power_photovoltaics": "sensor.ecu_current_power",
"sensor_power_load_no_var_loads": "sensor.power_load_no_var_loads",
"number_of_deferrable_loads": 7,
"list_nominal_power_of_deferrable_loads": [
{
"nominal_power_of_deferrable_loads": 1948
},
{
"nominal_power_of_deferrable_loads": 7230
},
{
"nominal_power_of_deferrable_loads": 1408
},
{
"nominal_power_of_deferrable_loads": 640
},
{
"nominal_power_of_deferrable_loads": 900
},
{
"nominal_power_of_deferrable_loads": 895
},
{
"nominal_power_of_deferrable_loads": 5517
}
],
"list_operating_hours_of_each_deferrable_load": [
{
"operating_hours_of_each_deferrable_load": 5
},
{
"operating_hours_of_each_deferrable_load": 4
},
{
"operating_hours_of_each_deferrable_load": 4
},
{
"operating_hours_of_each_deferrable_load": 4
},
{
"operating_hours_of_each_deferrable_load": 4
},
{
"operating_hours_of_each_deferrable_load": 4
},
{
"operating_hours_of_each_deferrable_load": 6
}
],
"list_peak_hours_periods_start_hours": [
{
"peak_hours_periods_start_hours": "06:00"
}
],
"list_peak_hours_periods_end_hours": [
{
"peak_hours_periods_end_hours": "22:00"
}
],
"list_treat_deferrable_load_as_semi_cont": [
{
"treat_deferrable_load_as_semi_cont": true
},
{
"treat_deferrable_load_as_semi_cont": true
},
{
"treat_deferrable_load_as_semi_cont": true
},
{
"treat_deferrable_load_as_semi_cont": true
},
{
"treat_deferrable_load_as_semi_cont": true
},
{
"treat_deferrable_load_as_semi_cont": true
},
{
"treat_deferrable_load_as_semi_cont": true
}
],
"load_peak_hours_cost": 0.1907,
"load_offpeak_hours_cost": 0.1419,
"photovoltaic_production_sell_price": 0.065,
"maximum_power_from_grid": 5310,
"list_pv_module_model": [
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
},
{
"pv_module_model": "REC_Solar_REC295TP2"
}
],
"list_pv_inverter_model": [
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___QS1__240V_"
},
{
"pv_inverter_model": "Altenergy_Power_System_Inc___YC600__240V_"
}
],
"list_surface_tilt": [
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 38
},
{
"surface_tilt": 41
},
{
"surface_tilt": 41
},
{
"surface_tilt": 41
},
{
"surface_tilt": 41
},
{
"surface_tilt": 41
},
{
"surface_tilt": 41
}
],
"list_surface_azimuth": [
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
},
{
"surface_azimuth": 225
}
],
"list_modules_per_string": [
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
},
{
"modules_per_string": 1
}
],
"list_strings_per_inverter": [
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
},
{
"strings_per_inverter": 1
}
],
"set_use_battery": false,
"battery_discharge_power_max": 1000,
"battery_charge_power_max": 1000,
"battery_discharge_efficiency": 0.95,
"battery_charge_efficiency": 0.95,
"battery_nominal_energy_capacity": 5000,
"battery_minimum_state_of_charge": 0.3,
"battery_maximum_state_of_charge": 0.9,
"battery_target_state_of_charge": 0.6
