EMHASS: An Energy Management for Home Assistant

[JHk (joop)]

Hoi All,
Yesterday may system stopt working with this errors. I do not no why.

[2025-07-26 11:29:25 +0200] [95642] [INFO] Successfully posted to sensor.total_cost_fun_value = 5.36
[2025-07-26 11:29:25 +0200] [95642] [INFO] Successfully posted to sensor.optim_status = Optimal
[2025-07-26 11:29:25 +0200] [95642] [INFO] Successfully posted to sensor.unit_load_cost = 0.239
[2025-07-26 11:29:25 +0200] [95642] [INFO] Successfully posted to sensor.unit_prod_price = 0.239
[2025-07-26 11:30:00 +0200] [95642] [INFO]  >> Obtaining params: 
[2025-07-26 11:30:00 +0200] [95642] [INFO] Passed runtime parameters: {'prediction_horizon': 13, 'prod_price_forecast': [0.239, 0.231, 0.213, 0.212, 0.232, 0.239, 0.251, 0.267, 0.293, 0.307, 0.314, 0.304, 0.293], 'load_cost_forecast': [0.239, 0.231, 0.213, 0.212, 0.232, 0.239, 0.251, 0.267, 0.293, 0.307, 0.314, 0.304, 0.293], 'operating_hours_of_each_deferrable_load': [0, 0, 0, 0], 'start_timesteps_of_each_deferrable_load': [0, 0, 0, 0], 'end_timesteps_of_each_deferrable_load': [0, 0, 0, 0], 'treat_deferrable_load_as_semi_cont': [0, 0, 0, 0], 'set_deferrable_load_single_constant': [1, 1, 1, 1], 'def_current_state': [1, 1, 1, 1], 'nominal_power_of_deferrable_loads': [2000, 2000, 0, 2000], 'alpha': 0.25, 'beta': 0.75, 'continual_publish': 0}
[2025-07-26 11:30:00 +0200] [95642] [INFO]  >> Setting input data dict
[2025-07-26 11:30:00 +0200] [95642] [INFO] Setting up needed data
[2025-07-26 11:30:00 +0200] [95642] [INFO] Retrieve hass get data method initiated...
[2025-07-26 11:30:04 +0200] [95642] [INFO] Retrieving weather forecast data using method = open-meteo
[2025-07-26 11:30:04 +0200] [95642] [INFO] Loading existing cached Open-Meteo JSON file: /data/cached-open-meteo-forecast.json
[2025-07-26 11:30:04 +0200] [95642] [INFO] The cached Open-Meteo JSON file is recent (age=27m, max_age=30m)
[2025-07-26 11:30:12 +0200] [95642] [INFO] Retrieving data from hass for load forecast using method = naive
[2025-07-26 11:30:12 +0200] [95642] [INFO] Retrieve hass get data method initiated...
[2025-07-26 11:30:19 +0200] [95642] [INFO]  >> Performing naive MPC optimization...
[2025-07-26 11:30:19 +0200] [95642] [INFO] Performing naive MPC optimization
[2025-07-26 11:30:19 +0200] [95642] [INFO] Perform an iteration of a naive MPC controller
[2025-07-26 11:30:24 +0200] [95642] [INFO] Status: Optimal
[2025-07-26 11:30:24 +0200] [95642] [INFO] Total value of the Cost function = 5.35
[2025-07-26 11:30:24 +0200] [95642] [INFO] Optimization status: Optimal
[2025-07-26 11:30:25 +0200] [22] [ERROR] Worker (pid:95642) was sent SIGKILL! Perhaps out of memory?
[2025-07-26 11:30:25 +0200] [97240] [INFO] Booting worker with pid: 97240
[2025-07-26 11:30:47 +0200] [97240] [INFO] Obtaining parameters from config.json:
[2025-07-26 11:30:47 +0200] [97240] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 608, in spawn_worker
    worker.init_process()
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/workers/gthread.py", line 94, in init_process
    super().init_process()
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/workers/base.py", line 135, in init_process
    self.load_wsgi()
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/workers/base.py", line 147, in load_wsgi
    self.wsgi = self.app.wsgi()
                ^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/app/base.py", line 66, in wsgi
    self.callable = self.load()
                    ^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/app/wsgiapp.py", line 57, in load
    return self.load_wsgiapp()
           ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/app/wsgiapp.py", line 47, in load_wsgiapp
    return util.import_app(self.app_uri)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/util.py", line 423, in import_app
    app = app(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^
  File "/app/src/emhass/web_server.py", line 64, in create_app
    main()
  File "/app/src/emhass/web_server.py", line 650, in main
    injection_dict = pickle.load(fid)
                     ^^^^^^^^^^^^^^^^
EOFError: Ran out of input
[2025-07-26 11:30:47 +0200] [97240] [INFO] Worker exiting (pid: 97240)
[2025-07-26 11:30:48 +0200] [22] [ERROR] Worker (pid:97240) exited with code 3
[2025-07-26 11:30:48 +0200] [22] [ERROR] Shutting down: Master

[JHk (joop)]

It is not possible to start EMHASS anymore. I restarted Home assistant.
What can I do?

EOFError: Ran out of input
[2025-07-27 12:37:07 +0200] [23] [INFO] Worker exiting (pid: 23)
[2025-07-27 12:37:08 +0200] [22] [ERROR] Worker (pid:23) exited with code 3
[2025-07-27 12:37:08 +0200] [22] [ERROR] Shutting down: Master
[2025-07-27 12:37:08 +0200] [22] [ERROR] Reason: Worker failed to boot.
   Building emhass @ file:///app
      Built emhass @ file:///app
Uninstalled 1 package in 42ms
Installed 1 package in 5ms
[2025-07-27 12:37:16 +0200] [22] [INFO] Starting gunicorn 23.0.0
[2025-07-27 12:37:16 +0200] [22] [INFO] Listening at: http://0.0.0.0:5000 (22)
[2025-07-27 12:37:16 +0200] [22] [INFO] Using worker: gthread
[2025-07-27 12:37:16 +0200] [23] [INFO] Booting worker with pid: 23
[2025-07-27 12:37:35 +0200] [23] [INFO] Obtaining parameters from config.json:
[2025-07-27 12:37:35 +0200] [23] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 608, in spawn_worker
    worker.init_process()
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/workers/gthread.py", line 94, in init_process
    super().init_process()
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/workers/base.py", line 135, in init_process
    self.load_wsgi()
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/workers/base.py", line 147, in load_wsgi
    self.wsgi = self.app.wsgi()
                ^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/app/base.py", line 66, in wsgi
    self.callable = self.load()
                    ^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/app/wsgiapp.py", line 57, in load
    return self.load_wsgiapp()
           ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/app/wsgiapp.py", line 47, in load_wsgiapp
    return util.import_app(self.app_uri)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/gunicorn/util.py", line 423, in import_app
    app = app(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^
  File "/app/src/emhass/web_server.py", line 64, in create_app
    main()
  File "/app/src/emhass/web_server.py", line 650, in main
    injection_dict = pickle.load(fid)
                     ^^^^^^^^^^^^^^^^
EOFError: Ran out of input
[2025-07-27 12:37:35 +0200] [23] [INFO] Worker exiting (pid: 23)
[2025-07-27 12:37:36 +0200] [22] [ERROR] Worker (pid:23) exited with code 3
[2025-07-27 12:37:36 +0200] [22] [ERROR] Shutting down: Master
[2025-07-27 12:37:36 +0200] [22] [ERROR] Reason: Worker failed to boot.

[RikBast]

Erase injection_dict.pkl in you share folder and restart EMHASS

[JHk (joop)]

I’m not that familliar with EMHASS. Where can I find that file?

[RikBast]

Open terminal in HA and go to your share directory (hope you set it up)

[110hs]

Hello Folks,
I’m back on EMHASS after a few months and I’m now trying to finalize the transition to open-meteo.
I was testing a bit what I did a few months ago and I can successfully run the code to pull the data from open-meteo for multiple days (2 confirmed so far) with 60 and 30 minutes resolution.
But when I try to switch to 15 minutes resolution I get the following error.
The array lengths look ok to me: 96 items for 24 hours.
Any idea?
Thanks!

[2025-07-28 01:17:49 +0200] [21] [INFO]  >> Obtaining params: 
[2025-07-28 01:17:49 +0200] [21] [INFO] Passed runtime parameters: {'optimization_time_step ': 15, 'delta_forecast': 1, 'prediction_horizon': 96, 'number_of_deferrable_loads': 0, 'list_nominal_power_of_deferrable_loads': [0], 'list_operating_hours_of_each_deferrable_load': [1], 'list_treat_deferrable_load_as_semi_cont': [1], 'def_current_state': [0], 'list_set_deferrable_load_single_constant': [0], 'list_start_timesteps_of_each_deferrable_load': [0], 'list_end_timesteps_of_each_deferrable_load': [0], 'soc_init': 0.906, 'soc_final': 0.2, 'battery_charge_power_max': 7100, 'battery_discharge_power_max': 7100, 'load_cost_forecast': [0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.151309, 0.151309, 0.151309, 0.151309, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.133427, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.151309, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025, 0.131025], 'prod_price_forecast': [0.087516, 0.087516, 0.087516, 0.08437, 0.08437, 0.08437, 0.08437, 0.08557, 0.08557, 0.08557, 0.08557, 0.087032, 0.087032, 0.087032, 0.087032, 0.088336, 0.088336, 0.088336, 0.088336, 0.108251, 0.108251, 0.108251, 0.108251, 0.117824, 0.117824, 0.117824, 0.117824, 0.123084, 0.123084, 0.123084, 0.123084, 0.119286, 0.119286, 0.119286, 0.119286, 0.110986, 0.110986, 0.110986, 0.110986, 0.112564, 0.112564, 0.112564, 0.112564, 0.11633, 0.11633, 0.11633, 0.11633, 0.115257, 0.115257, 0.115257, 0.115257, 0.117456, 0.117456, 0.117456, 0.117456, 0.109871, 0.109871, 0.109871, 0.109871, 0.101507, 0.101507, 0.101507, 0.101507, 0.105211, 0.105211, 0.105211, 0.105211, 0.10986, 0.10986, 0.10986, 0.10986, 0.12098, 0.12098, 0.12098, 0.12098, 0.122558, 0.122558, 0.122558, 0.122558, 0.122021, 0.122021, 0.122021, 0.122021, 0.111512, 0.111512, 0.111512, 0.111512, 0.105431, 0.105431, 0.105431, 0.105431, 0.099256, 0.099256, 0.099256, 0.099256, 0.087516], 'alpha': 0.25, 'beta': 0.75, 'model_type': 'KNN'}
[2025-07-28 01:17:49 +0200] [21] [INFO]  >> Setting input data dict
[2025-07-28 01:17:49 +0200] [21] [INFO] Setting up needed data
[2025-07-28 01:17:49 +0200] [21] [INFO] Retrieve hass get data method initiated...
[2025-07-28 01:17:52 +0200] [21] [INFO] Retrieving weather forecast data using method = open-meteo
[2025-07-28 01:17:52 +0200] [21] [INFO] Loading existing cached Open-Meteo JSON file: /data/cached-open-meteo-forecast.json
[2025-07-28 01:17:52 +0200] [21] [INFO] The cached Open-Meteo JSON file is recent (age=27m, max_age=30m)
[2025-07-28 01:17:52 +0200] [21] [INFO] Retrieving data from hass for load forecast using method = naive
[2025-07-28 01:17:52 +0200] [21] [INFO] Retrieve hass get data method initiated...
[2025-07-28 01:17:56 +0200] [21] [ERROR] Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/flask/app.py", line 1511, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/flask/app.py", line 919, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/src/emhass/web_server.py", line 411, in action_call
    input_data_dict = set_input_data_dict(
                      ^^^^^^^^^^^^^^^^^^^^
  File "/app/src/emhass/command_line.py", line 439, in set_input_data_dict
    df_input_data_dayahead.index[0] : df_input_data_dayahead.index[
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/pandas/core/indexes/base.py", line 5389, in __getitem__
    return getitem(key)
           ^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/pandas/core/arrays/datetimelike.py", line 381, in __getitem__
    result = cast("Union[Self, DTScalarOrNaT]", super().__getitem__(key))
                                                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/pandas/core/arrays/_mixins.py", line 284, in __getitem__
    result = self._ndarray[key]
             ~~~~~~~~~~~~~^^^^^
IndexError: index 95 is out of bounds for axis 0 with size 48

[JHk (joop)]

Hi Rik,

Erased injection_dict.pkl file. EMHASS is still not starting.

[JHk (joop)]

After removing and reinstallation everything is working fine.

[RikBast]

How do you guys measure ‘power no var loads’? At the moment I use the Huawei integration and calculate the ’ power no var loads ', but if data come slow or are unavailable I get strange readings, it seems that HA does some statistic on the data.

But if use ‘Show more’ I get this:

Using MLforecaster gives a very bad fit because of all these spikes I think?

Will a direct measurement be more robust? Like Aeotec Home Energy Meter 8? Any advice?

[altonius (Alton(ius) Blom)]

For my solar generation data I do a filter to clear out some of those spikes, and use the filtered data in EMHASS. Maybe that’d help with your power no vars too:

  - platform: filter
    name: "filtered solar generation"
    entity_id: sensor.power_solar_generation
    filters:
      - filter: lowpass
        time_constant: 10
        precision: 2
      - filter: range
        lower_bound: 0
        upper_bound: 12500

[RikBast]

Hi,

I tried that also, but still a lot of spikes.
Now I am trying this:

  - trigger:
      - platform: time_pattern
        minutes: "/1"  # Update every 1 minute
    sensor:
      - name: "House Other Loads"
        unique_id: house_other_loads
        unit_of_measurement: "W"
        icon: mdi:home-lightning-bolt
        state: >
          {% set inverter = states('sensor.inverter_werkelijk_vermogen') | float(0) %}
          {% set grid = states('sensor.power_meter_vermogen') | float(0) %}
          {% set dryer = states('sensor.waskeuken_wasdroger_waskeuken_618_power') | float(0) %}
          {% set washer = states('sensor.waskeuken_wasmachine_waskeuken_611_power') | float(0) %}
          {% set phev = states('sensor.garage_phev_switch_803_power') | float(0) %}

          {% set house_load = inverter - grid %}
          {% set var_loads = dryer + washer + phev %}
          {% set other_loads = house_load - var_loads %}
          {{ (other_loads if other_loads > 0 else 0) | round(1) }}

This sensor logs data every minute, up till now it’s much better, spikes I can’t prevent.:

[markpurcell]

Paul,

Just picking up on an old thread.

Have you actually spun up a flexmeasures instance, it does look interesting.

[sladey]

Hey Mark,

Funny you should ask. I have setup flex measures locally, I have also used their hosted instance (Seita’s hosted instance). This week I am chatting to the team about how to use flex measures as a simulation tool for different combinations of battery, solar and fleet ev charging.

[soco]

Hi everyone,
been playing around with EMHASS for a few weeks and currently trying to use the model-fit for load cost and load power forecasts.
Unfortunately I always get negative R2 scores. Anybody else experienced this issue?

Here is a Debug log from the load cost forecast:

[2025-08-04 07:38:40 +0200] [23] [INFO] Passed runtime parameters: {'historic_days_to_retrieve': 17, 'model_type': 'load_cost_forecast', 'var_model': 'sensor.nordpool_kwh_ger_eur_3_10_0', 'sklearn_model': 'KNeighborsRegressor', 'num_lags': 48, 'split_date_delta': '48h', 'perform_backtest': 'False'}
[2025-08-04 07:38:40 +0200] [23] [INFO]  >> Setting input data dict
[2025-08-04 07:38:40 +0200] [23] [INFO] Setting up needed data
[2025-08-04 07:38:40 +0200] [23] [DEBUG] Initialized Optimization with retrieve_hass_conf: {'optimization_time_step': Timedelta('0 days 00:30:00'), 'historic_days_to_retrieve': 17, 'sensor_power_photovoltaics': 'sensor.total_dc_power', 'sensor_power_photovoltaics_forecast': '', 'sensor_power_load_no_var_loads': 'sensor.load_power', 'load_negative': False, 'set_zero_min': True, 'sensor_replace_zero': ['sensor.total_dc_power'], 'sensor_linear_interp': ['sensor.load_power'], 'method_ts_round': 'first', 'continual_publish': False, 'hass_url': 'http://supervisor/core/api', 'long_lived_token': '', 'time_zone': <DstTzInfo 'Europe/Berlin' LMT+0:53:00 STD>, 'Latitude': XX.XXXXX, 'Longitude': XX.XXXXX, 'Altitude': XXX, 'solcast_api_key': 'XXX', 'solcast_rooftop_id': 'XXX'}
[2025-08-04 07:38:40 +0200] [23] [DEBUG] Optimization configuration: {'costfun': 'self-consumption', 'logging_level': 'DEBUG', 'set_use_pv': True, 'set_use_adjusted_pv': False, 'adjusted_pv_regression_model': 'LassoRegression', 'adjusted_pv_solar_elevation_threshold': 10, 'set_use_battery': True, 'number_of_deferrable_loads': 5, 'nominal_power_of_deferrable_loads': [600, 400, 450, 3500, 2000], 'minimum_power_of_deferrable_loads': [0, 0, 0, 1440, 0], 'operating_hours_of_each_deferrable_load': [0, 0, 0, 0, 0], 'treat_deferrable_load_as_semi_cont': [True, True, True, False, True], 'set_deferrable_load_single_constant': [True, True, True, False, True], 'set_deferrable_startup_penalty': [0, 0, 0, 0, 0], 'delta_forecast_daily': Timedelta('1 days 00:00:00'), 'load_forecast_method': 'mlforecaster', 'load_cost_forecast_method': 'hp_hc_periods', 'load_peak_hours_cost': 0.354, 'load_offpeak_hours_cost': 0.354, 'production_price_forecast_method': 'constant', 'photovoltaic_production_sell_price': 0.084, 'set_total_pv_sell': False, 'lp_solver': 'COIN_CMD', 'lp_solver_path': '/usr/bin/cbc', 'lp_solver_timeout': 45, 'num_threads': 0, 'set_nocharge_from_grid': False, 'set_nodischarge_to_grid': True, 'set_battery_dynamic': False, 'battery_dynamic_max': 0.9, 'battery_dynamic_min': -0.9, 'weight_battery_discharge': 1, 'weight_battery_charge': 1, 'weather_forecast_method': 'solcast', 'open_meteo_cache_max_age': 30, 'start_timesteps_of_each_deferrable_load': [0, 0, 0, 0, 0], 'end_timesteps_of_each_deferrable_load': [0, 0, 0, 0, 0], 'load_peak_hour_periods': {'period_hp_1': [{'start': '00:00'}, {'end': '00:00'}]}}
[2025-08-04 07:38:40 +0200] [23] [DEBUG] Plant configuration: {'maximum_power_from_grid': 22000, 'maximum_power_to_grid': 6000, 'pv_module_model': ['CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M'], 'pv_inverter_model': ['Sungrow_Power_Supply_Co___Ltd___SG60KU_M__480V_'], 'surface_tilt': [35], 'surface_azimuth': [165], 'modules_per_string': [14], 'strings_per_inverter': [1], 'inverter_is_hybrid': True, 'compute_curtailment': False, 'battery_discharge_power_max': 6000, 'battery_charge_power_max': 6000, 'battery_discharge_efficiency': 0.95, 'battery_charge_efficiency': 0.95, 'battery_nominal_energy_capacity': 9600, 'battery_minimum_state_of_charge': 0.05, 'battery_maximum_state_of_charge': 1, 'battery_target_state_of_charge': 1}
[2025-08-04 07:38:40 +0200] [23] [DEBUG] Solver configuration: lp_solver=COIN_CMD, lp_solver_path=/usr/bin/cbc
[2025-08-04 07:38:40 +0200] [23] [DEBUG] Number of threads: 2
[2025-08-04 07:38:40 +0200] [23] [INFO] Retrieve hass get data method initiated...
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 27, on day: 2025-07-18 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 30, on day: 2025-07-19 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 27, on day: 2025-07-20 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 30, on day: 2025-07-21 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 27, on day: 2025-07-22 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 27, on day: 2025-07-23 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 27, on day: 2025-07-24 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 27, on day: 2025-07-25 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 30, on day: 2025-07-26 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 27, on day: 2025-07-27 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 36, on day: 2025-07-28 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 30, on day: 2025-07-29 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 27, on day: 2025-07-30 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 30, on day: 2025-07-31 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 34, on day: 2025-08-01 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 31, on day: 2025-08-02 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [DEBUG] sensor:sensor.nordpool_kwh_ger_eur_3_10_0 retrieved Dataframe count: 42, on day: 2025-08-03 00:00:00+00:00. This is less than freq value passed: 0 days 00:30:00
[2025-08-04 07:38:40 +0200] [23] [INFO]  >> Performing a machine learning forecast model fit...
[2025-08-04 07:38:40 +0200] [23] [INFO] Performing a forecast model fit for load_cost_forecast
[2025-08-04 07:38:40 +0200] [23] [INFO] Training a KNeighborsRegressor model
[2025-08-04 07:38:41 +0200] [23] [INFO] Elapsed time for model fit: 0.030734539031982422
[2025-08-04 07:38:41 +0200] [23] [INFO] Prediction R2 score of fitted model on test data: -0.41213566715192806
[2025-08-04 07:38:41 +0200] [23] [DEBUG] saved model to /share/load_cost_forecast_mlf.pkl

and here is the plot:

I already deleted the pkl-file but still the same issue.

Thanks in advance!

[RudolfRendier]

You probably need more data, 2 weeks is not a lot for a ML-model.
Adjust your HA recorder settings to retain more.

I keep 30 days, and the model is still way off.
I haven’t tried saving more.

[eekhoorn]

I got these results for the PV forecast with 8 days data:

[INFO] PV adjust Training metrics: RMSE = 1417.9528530229886, R2 = 0.662969382420331

According to chatgtp this is ‘reasonable’, but I don’t know if it is better than the standard approach and if the data requirement is totally different than for a load forecast.

[soco]

I just rerun the model-fit for load cost and load power.
I currently have 21 days for load cost and 10 days of data for load power (had to recreate the sensor…).
For load cost I got an R2 score of 0.41 and for load power of -0.66.

Updated my recorder to held 60 days of data and will observe, if I get better results.

Thanks!

[swiftycoder (Sampath D)]

This is the dashboard and setup I need. Also keen to see where you’ve taken this automation.

[soco]

I can confirm, that using operating_timesteps_of_each_deferrable_load results in “infeasible”.
Here is a part of the optimization configuration using timesteps:

'number_of_deferrable_loads': 6, 
'nominal_power_of_deferrable_loads': [600, 400, 450, 3500, 2000, 250], 
'minimum_power_of_deferrable_loads': [0, 0, 0, 1440, 0, 0], 
--> 'operating_hours_of_each_deferrable_load': [0, 0, 0, 0, 0, 0],
'treat_deferrable_load_as_semi_cont': [1, 1, 1, 0, 1, 1], 
'set_deferrable_load_single_constant': [1, 1, 1, 0, 0, 1], 
'set_deferrable_startup_penalty': [1, 1, 1, 1, 1, 1], 
'start_timesteps_of_each_deferrable_load': [0, 0, 0, 0, 0, 0], 
'end_timesteps_of_each_deferrable_load': [0, 0, 0, 0, 0, 96], 
--> 'operating_timesteps_of_each_deferrable_load': [0, 0, 0, 0, 4, 96], 

and this is from the optimization configuration using hours:

'number_of_deferrable_loads': 6, 
'nominal_power_of_deferrable_loads': [600, 400, 450, 3500, 2000, 250], 
'minimum_power_of_deferrable_loads': [0, 0, 0, 1440, 0, 0], 
--> 'operating_hours_of_each_deferrable_load': [0, 0, 0, 0, 1, 24], 
'treat_deferrable_load_as_semi_cont': [1, 1, 1, 0, 1, 1], 
'set_deferrable_load_single_constant': [1, 1, 1, 0, 0, 1], 
'set_deferrable_startup_penalty': [1, 1, 1, 1, 1, 1],
'start_timesteps_of_each_deferrable_load': [0, 0, 0, 0, 0, 0], 
'end_timesteps_of_each_deferrable_load': [0, 0, 0, 0, 0, 96],

as you can see, when passing the timesteps parameter, it additonally passes the operating_hours parameter with all zeros.

[RikBast]

Yes, and that seems to be the issue, but how to solve? Seems a bug to me.