EMHASS: An Energy Management for Home Assistant

110hs · November 13, 2024, 4:44pm

Thank you @RikBast, as I was coming from the add-on and these NaN options could be left to default as hard-coded before, but it seems it’s not the case.

Let’s see if any of the maintainers have details about the 9 days warning, I would prefer not to increase it if not strictly needed as I’m not using any ML method to predict tomorrow’s load.

EDIT:
Anyway I may be wrong but if I read it correctly the number of days to retrieve is automatically set to 9 if the value is lower

2024-11-13 17:45:56,774 - web_server - WARNING - warning `days_to_retrieve` is set to a value less than 9, this could cause an error with the fit
2024-11-13 17:45:56,774 - web_server - WARNING - setting`passed_data:days_to_retrieve` to 9 for fit/predict/tune

110hs · November 13, 2024, 4:55pm

After upgrading to the latest version I’ve tested all the commands I previously developed (in use or not).
Everything works but the model predict one (I’m not currently using).
Below is the log; anybody else who maybe had the same error can share a quick solution/cause?
Thanks

2024-11-13 17:52:42,280 - web_server - INFO - Passed runtime parameters: {'days_to_retrieve': 21, 'model_type': 'KNN', 'var_model': 'sensor.consumption_w', 'model_predict_publish': 'False', 'model_predict_entity_id': 'sensor.p_load_forecast_custom_model', 'model_predict_unit_of_measurement': 'W', 'model_predict_friendly_name': 'Load Power Forecast custom ML model'}
2024-11-13 17:52:42,280 - web_server - INFO -  >> Setting input data dict
2024-11-13 17:52:42,280 - web_server - INFO - Setting up needed data
2024-11-13 17:52:42,285 - web_server - INFO - Retrieve hass get data method initiated...
2024-11-13 17:53:07,702 - web_server - INFO -  >> Performing a machine learning forecast model predict...
2024-11-13 17:53:07,706 - web_server - ERROR - Exception on /action/forecast-model-predict [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 3653, in get_loc
    return self._engine.get_loc(casted_key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pandas/_libs/index.pyx", line 147, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 176, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'sensor.consumption_filtered_w'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 401, in action_call
    df_pred = forecast_model_predict(input_data_dict, app.logger)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 575, in forecast_model_predict
    predictions = mlf.predict(data_last_window)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/machine_learning_forecaster.py", line 208, in predict
    last_window=data_last_window[self.var_model],
                ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/frame.py", line 3761, in __getitem__
    indexer = self.columns.get_loc(key)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 3655, in get_loc
    raise KeyError(key) from err
KeyError: 'sensor.consumption_filtered_w'

RikBast · November 13, 2024, 5:08pm

Just ran a quick model fit and model predict, no errors, just the warning.

2024-11-13 18:01:28,385 - web_server - INFO - Passed runtime parameters: {'days_to_retrieve': 2, 'model_type': 'load_forecast', 'var_model': 'sensor.power_load_no_var_loads', 'sklearn_model': 'KNeighborsRegressor', 'num_lags': 96, 'split_date_delta': '96h', 'perform_backtest': 'True'}
2024-11-13 18:01:28,385 - web_server - INFO -  >> Setting input data dict
2024-11-13 18:01:28,386 - web_server - INFO - Setting up needed data
2024-11-13 18:01:28,391 - web_server - WARNING - warning `days_to_retrieve` is set to a value less than 9, this could cause an error with the fit
2024-11-13 18:01:28,391 - web_server - WARNING - setting`passed_data:days_to_retrieve` to 9 for fit/predict/tune
2024-11-13 18:01:28,395 - web_server - INFO - Retrieve hass get data method initiated...
2024-11-13 18:01:30,440 - web_server - INFO -  >> Performing a machine learning forecast model fit...
2024-11-13 18:01:30,441 - web_server - INFO - Performing a forecast model fit for load_forecast
2024-11-13 18:01:30,445 - web_server - INFO - Training a KNeighborsRegressor model
2024-11-13 18:01:30,499 - web_server - INFO - Elapsed time for model fit: 0.05338883399963379
2024-11-13 18:01:30,521 - web_server - INFO - Prediction R2 score of fitted model on test data: -0.2688222269701517
2024-11-13 18:01:30,521 - web_server - INFO - Performing simple backtesting of fitted model

  0%|          | 0/2 [00:00<?, ?it/s]
100%|██████████| 2/2 [00:00<00:00, 116.77it/s]
2024-11-13 18:01:30,543 - web_server - INFO - Elapsed backtesting time: 0.021650075912475586
2024-11-13 18:01:30,546 - web_server - INFO - Backtest R2 score:    neg_r2_score
0      0.751414
2024-11-13 18:02:06,841 - web_server - INFO - EMHASS server online, serving index.html...
2024-11-13 18:03:24,180 - web_server - INFO - Passed runtime parameters: {'model_type': 'load_forecast', 'model_predict_publish': 'True', 'model_predict_entity_id': 'sensor.power_load_no_var_loads_custom_model', 'model_predict_unit_of_measurement': 'W', 'model_predict_friendly_name': 'Power load no var ML model'}
2024-11-13 18:03:24,181 - web_server - INFO -  >> Setting input data dict
2024-11-13 18:03:24,181 - web_server - INFO - Setting up needed data
2024-11-13 18:03:24,185 - web_server - INFO - Retrieve hass get data method initiated...
2024-11-13 18:03:25,982 - web_server - INFO -  >> Performing a machine learning forecast model predict...
2024-11-13 18:03:26,029 - web_server - INFO - Successfully posted to sensor.power_load_no_var_loads_custom_model = 1416.39
2024-11-13 18:04:11,527 - web_server - INFO - EMHASS server online, serving index.html...

110hs · November 13, 2024, 7:15pm

Thanks @RikBast for the patience. It seems solved now.

s2719632 · November 13, 2024, 8:01pm

Hi,

I am having issues with the deferrable load since upgrading. The issue is that the planning seems to preferably use what is in the config.json rather than the parameters in the rest command. In the example below it is planning to run the load for 2 hours as per the config but I really only need to run it for one hour as per rest command. If I change the number of def loads in the config.json to zero then I get no deferrable output. Any ideas?

Here is my config.json

{
  "battery_charge_efficiency": 1,
  "battery_charge_power_max": 10000,
  "battery_discharge_efficiency": 1,
  "battery_discharge_power_max": 10000,
  "battery_dynamic_max": 0.9,
  "battery_dynamic_min": -0.9,
  "battery_maximum_state_of_charge": 1,
  "battery_minimum_state_of_charge": 0.04,
  "battery_nominal_energy_capacity": 19200,
  "battery_target_state_of_charge": 0.04,
  "compute_curtailment": true,
  "continual_publish": true,
  "costfun": "profit",
  "delta_forecast_daily": 1,
  "end_timesteps_of_each_deferrable_load": [
    0
  ],
  "historic_days_to_retrieve": 9,
  "inverter_is_hybrid": false,
  "load_cost_forecast_method": "hp_hc_periods",
  "load_forecast_method": "naive",
  "load_negative": false,
  "load_offpeak_hours_cost": 0.1419,
  "load_peak_hour_periods": {
    "period_hp_1": [
      {
        "start": "02:54"
      },
      {
        "end": "15:24"
      }
    ],
    "period_hp_2": [
      {
        "start": "17:24"
      },
      {
        "end": "20:24"
      }
    ]
  },
  "load_peak_hours_cost": 0.1907,
  "logging_level": "INFO",
  "lp_solver": "COIN_CMD",
  "lp_solver_path": "/usr/bin/cbc",
  "maximum_power_from_grid": 25000,
  "maximum_power_to_grid": 15000,
  "method_ts_round": "first",
  "modules_per_string": [
    16
  ],
  "nominal_power_of_deferrable_loads": [
    11500
  ],
  "number_of_deferrable_loads": 1,
  "operating_hours_of_each_deferrable_load": [
    2
  ],
  "optimization_time_step": 30,
  "photovoltaic_production_sell_price": 0.1419,
  "production_price_forecast_method": "constant",
  "pv_inverter_model": [
    "Fronius_International_GmbH__Fronius_Primo_5_0_1_208_240__240V_"
  ],
  "pv_module_model": [
    "CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M"
  ],
  "sensor_linear_interp": [
    "sensor.solarnet_power_photovoltaics",
    "sensor.power_load_no_var_loads"
  ],
  "sensor_power_load_no_var_loads": "sensor.power_load_no_var_loads",
  "sensor_power_photovoltaics": "sensor.solarnet_power_photovoltaics",
  "sensor_replace_zero": [
    "sensor.solarnet_power_photovoltaics",
    "sensor.power_load_no_var_loads"
  ],
  "set_battery_dynamic": true,
  "set_deferrable_load_single_constant": [
    false
  ],
  "set_deferrable_startup_penalty": [
    0
  ],
  "set_nocharge_from_grid": false,
  "set_nodischarge_to_grid": false,
  "set_total_pv_sell": false,
  "set_use_battery": true,
  "set_zero_min": true,
  "start_timesteps_of_each_deferrable_load": [
    0
  ],
  "strings_per_inverter": [
    1
  ],
  "surface_azimuth": [
    205
  ],
  "surface_tilt": [
    30
  ],
  "treat_deferrable_load_as_semi_cont": [
    false
  ],
  "weather_forecast_method": "solar.forecast",
  "weight_battery_charge": 0,
  "weight_battery_discharge": 0.15
}

And here is my rest command

rest_command:
  naive_mpc_optim:
    url: http://localhost:5000/action/naive-mpc-optim
    method: POST
    content_type: "application/json"
    payload: >-
      {
        "prod_price_forecast": {{
          ([states('sensor.home_feed_in_price')|float(0)] +
          (state_attr('sensor.home_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list))
          | tojson 
        }},
        "load_cost_forecast": {{
          ([states('sensor.home_general_price')|float(0)] + 
          state_attr('sensor.home_general_forecast', 'forecasts') |map(attribute='per_kwh')|list) 
          | tojson 
        }},
        "load_power_forecast": {{
          ([states('sensor.power_load_no_var_loads')|int(0)] +
          (states('input_text.fi_fo_buffer').split(', ')|map('multiply',1000)|map('int')|list)[1:]
          )| tojson 
        }},
        "pv_power_forecast": {{([states('sensor.solarnet_power_photovoltaics')|int(0)] +
          state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list +
          state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list
          )
          }},
        "prediction_horizon": {{
          min(48, (state_attr('sensor.home_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1)
        }},
        "alpha": 1,
        "beta": 0,
        "number_of_deferrable_loads": 1,
        "list_operating_hours_of_each_deferrable_load": [{{states('sensor.def_total_hours_ev')|int(0)}}],
        "list_nominal_power_of_deferrable_loads": [11500],
        "list_treat_deferrable_load_as_semi_cont": [0],
        "list_set_deferrable_load_single_constant": [0],
        "soc_init": {{ (states('sensor.battery_level_nominal')|int(0))/100 }},
        "soc_final": 0.05
      }

Here are some of the results from the rest command

        "number_of_deferrable_loads": 1,
        "list_operating_hours_of_each_deferrable_load": [1],
        "list_nominal_power_of_deferrable_loads": [11500],
        "list_treat_deferrable_load_as_semi_cont": [0],
        "list_set_deferrable_load_single_constant": [0],
        "soc_init": 0.06,
        "soc_final": 0.05

And here is the plan showing around 2 hours of deferrable load. Certainly more than the 1 hour I want to plan for

Appreciate any thoughts. Thank you

jezhall · November 14, 2024, 2:10am

Hi All,

I need help with stopping the car from starting and stopping its charge so often, It gets to the point at night the cars not even full and im unsure as to how why other then the stop starts from deferable loads.

Ive tried even setting the hours to a fixed value but EMHASS constantly moves it around .

logging_level: DEBUG
data_path: default
costfun: profit
sensor_power_photovoltaics: sensor.solar_total_true
sensor_power_load_no_var_loads: sensor.consumption
set_total_pv_sell: false
set_nocharge_from_grid: false
set_nodischarge_to_grid: false
maximum_power_from_grid: 12000
maximum_power_to_grid: 25000
number_of_deferrable_loads: 2
list_nominal_power_of_deferrable_loads:

nominal_power_of_deferrable_loads: 2000
list_operating_hours_of_each_deferrable_load:
operating_hours_of_each_deferrable_load: 4
list_start_timesteps_of_each_deferrable_load:
start_timesteps_of_each_deferrable_load: 0
list_end_timesteps_of_each_deferrable_load:
end_timesteps_of_each_deferrable_load: 0
list_peak_hours_periods_start_hours:
peak_hours_periods_start_hours: “14:00”
list_peak_hours_periods_end_hours:
peak_hours_periods_end_hours: “21:00”
list_treat_deferrable_load_as_semi_cont:
treat_deferrable_load_as_semi_cont: true
list_set_deferrable_load_single_constant:
set_deferrable_load_single_constant: true
list_set_deferrable_startup_penalty:
set_deferrable_startup_penalty: 0
load_peak_hours_cost: 0.1907
load_offpeak_hours_cost: 0.1419
photovoltaic_production_sell_price: 0.065
list_pv_module_model:
pv_module_model: CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M
list_pv_inverter_model:
pv_inverter_model: Fronius_International_GmbH__Fronius_Primo_5_0_1_208_240__240V_
list_surface_tilt:
surface_tilt: 22
list_surface_azimuth:
surface_azimuth: 75
list_modules_per_string:
modules_per_string: 20
list_strings_per_inverter:
strings_per_inverter: 1
inverter_is_hybrid: false
compute_curtailment: true
set_use_battery: true
battery_nominal_energy_capacity: 48000
hass_url: empty
long_lived_token: empty
optimization_time_step: 30
historic_days_to_retrieve: 2
method_ts_round: first
lp_solver: COIN_CMD
lp_solver_path: /usr/bin/cbc
set_battery_dynamic: true
battery_dynamic_max: 0.9
battery_dynamic_min: -0.9
weight_battery_discharge: 0.2
weight_battery_charge: 0.2
load_forecast_method: naive
battery_discharge_power_max: 11600
battery_charge_power_max: 11600
battery_discharge_efficiency: 0.95
battery_charge_efficiency: 0.7
battery_maximum_state_of_charge: 1
set_zero_min: true
battery_minimum_state_of_charge: 0.2

Ive set my config

rest_command:
  naive_mpc_optim:
    url: http://localhost:5000/action/naive-mpc-optim
    method: POST
    content_type: "application/json"
    payload: >-
     {
       "prod_price_forecast": {{ 
        (
        state_attr('sensor.forecast', 'forecast') 
        | map(attribute='earningsFlexUp') 
        | map('float') 
        | map('round', 2) 
        | list
        + [0] * (48 - (state_attr('sensor.forecast', 'forecast') | map(attribute='earningsFlexUp') | list | length))
        )[::6][:48]
        | tojson 
        }},
        "load_cost_forecast": {{ 
        (
         state_attr('sensor.forecast', 'forecast') 
        | map(attribute='costsFlexUp') 
        | map('float') 
        | map('round', 2) 
        | list
        + [0] * (48 - (state_attr('sensor.forecast', 'forecast') | map(attribute='earningsFlexUp') | list | length))
        )[::6][:48]
        | tojson 
        }},

        "pv_power_forecast": {{
          ([states('sensor.solar_total_true')|int(0)] +
          state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list +
          state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list
          )| tojson
        }},
        "load_power_forecast": {{
          ([states('sensor.consumption')|int(0)] +
          (states('input_text.fi_fo_buffer').split(', ')|map('multiply',1000)|map('int')|list)[1:]
          )| tojson 
        }},
        "prediction_horizon": {{
          min(48, (state_attr('sensor.home_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1)
        }},
        "num_def_loads": 2,
        "def_total_hours": [4, {{ states('sensor.def_total_hours_ev') | int | round(0)}}],
        "def_end_timestep": [0, 0],
        
        "P_deferrable_nom": [
        {% set shelly_power = states('sensor.shellyplus1pm_441793cf2d38_switch_0_power') | float(0) %}
        {{ 500 if shelly_power == 0 else shelly_power }},
        {% if is_state('switch.charger_availability', 'off') %}
        7200
        {% else %}
        1
        {% endif %}],


        "treat_def_as_semi_cont": [0, 0],
        "set_def_constant": [1, 1],
        "def_start_penalty": [1, 1],      
        "soc_init": {{(states('sensor.battery_total_2')|int(0))/400}},
        "soc_final": 0.2,
        "alpha": 1,
        "beta": 0
      }

RudolfRendier · November 14, 2024, 5:06pm

That sounds as if you’re moving the horizon of the MPC. (i.e. postponing the end). Are you?

Johan71 · November 14, 2024, 5:11pm

How is it that as soon as I send endsteps in my code, it becomes infesable without error?
Current version: v0.11.2

post_naive_mpc_shell_carnow: >
  curl -i -H "Content-Type: application/json" -X POST -d '{
    "load_cost_forecast":{{((state_attr("sensor.nordpool_tibber", "raw_today") | map(attribute="value") | list  + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value") | list))[now().hour:][:24] }},
    "prod_price_forecast":{{((state_attr("sensor.nordpool_tibber", "raw_today") | map(attribute="value") | list  + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value") | list))[now().hour:][:24] }},
    "prediction_horizon":{{min(24, (((state_attr("sensor.nordpool_tibber", "raw_today")|map(attribute="value")|list + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value")| list)[now().hour:][:24]|list|length)))}},
    "pv_power_forecast":{{([states("sensor.inverter_input_power")|int(0)] + state_attr("sensor.solcast_pv_forecast_forecast_today", "detailedHourly")|selectattr("period_start","gt",utcnow()) | map(attribute="pv_estimate")|map("multiply",1000)|map("int")|list + state_attr("sensor.solcast_pv_forecast_forecast_tomorrow", "detailedHourly")|selectattr("period_start","gt",utcnow()) | map(attribute="pv_estimate")|map("multiply",1000)|map("int")|list)| tojson}},
    "load_power_forecast":{{[states('sensor.power_load_novar_loads')|int] +(states('input_text.fifo_buffer').split(',')|map('int')|list)[1:]}},
    "soc_init":{{ max(0,states("sensor.battery_state_of_capacity")|int(0))/100 }},
    "soc_final":{{ max(100,states("number.battery_end_of_discharge_soc")|int(0))/100 }},
    "P_deferrable_nom": [{{ states('sensor.go_echarger_265216_nrg_12')|int }}, 0],
    "delta_forecast":2,
    "alpha": 1,
    "beta": 0,
    "def_total_hours":[{{ states('sensor.charger_def_hours') | default(0) }}, 0],
    "def_end_timestep":[{{ states('sensor.endslots') | default(0) }}, 0]
  }' http://localhost:5000/action/naive-mpc-optim

2024-11-14 18:05:35,872 - web_server - INFO - Passed runtime parameters: {'load_cost_forecast': [0.56, 0.55, 0.5, 0.47, 0.43, 0.36, 0.35, 0.33, 0.29, 0.28, 0.29, 0.34, 0.37, 0.49, 0.68, 0.68, 0.65, 0.48, 0.39, 0.38, 0.38, 0.37, 0.37, 0.37], 'prod_price_forecast': [0.56, 0.55, 0.5, 0.47, 0.43, 0.36, 0.35, 0.33, 0.29, 0.28, 0.29, 0.34, 0.37, 0.49, 0.68, 0.68, 0.65, 0.48, 0.39, 0.38, 0.38, 0.37, 0.37, 0.37], 'prediction_horizon': 24, 'pv_power_forecast': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 233, 1291, 2405, 3204, 3455, 3156, 2276, 151, 0, 0, 0, 0, 0, 0, 0, 0], 'load_power_forecast': [2679, 1896, 1341, 1163, 2263, 1362, 3245, 1563, 1338, 1202, 1254, 1201, 1208, 1230, 1241, 1299, 2371, 1491, 1465, 1384, 718, 1506, 1422, 1566, 2188, 2368, 1188, 724, 1168, 901, 2099, 3209, 2637, 2533, 2698, 1240, 674, 1289, 2023, 1447, 2452, 1116, 2234, 1887, 1111, 2123, 896, 886], 'soc_init': 0.16, 'soc_final': 1.0, 'P_deferrable_nom': [0, 0], 'delta_forecast': 2, 'alpha': 1, 'beta': 0, 'def_total_hours': [5, 0], 'def_end_timestep': [5, 0]}
2024-11-14 18:05:35,872 - web_server - INFO -  >> Setting input data dict
2024-11-14 18:05:35,872 - web_server - INFO - Setting up needed data
2024-11-14 18:05:35,875 - web_server - INFO - Retrieve hass get data method initiated...
2024-11-14 18:05:36,180 - web_server - INFO - Retrieving weather forecast data using method = list
2024-11-14 18:05:36,183 - web_server - INFO -  >> Performing naive MPC optimization...
2024-11-14 18:05:36,184 - web_server - INFO - Performing naive MPC optimization
2024-11-14 18:05:36,194 - web_server - INFO - Perform an iteration of a naive MPC controller
2024-11-14 18:05:36,269 - web_server - INFO - Status: Infeasible
2024-11-14 18:05:36,269 - web_server - INFO - Total value of the Cost function = -8.75
2024-11-14 18:05:36,275 - web_server - INFO - Publishing data to HASS instance
2024-11-14 18:05:36,280 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 0
2024-11-14 18:05:36,282 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 2679
2024-11-14 18:05:36,284 - web_server - INFO - Successfully posted to sensor.p_hybrid_inverter = 1330.0
2024-11-14 18:05:36,286 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 1.0
2024-11-14 18:05:36,288 - web_server - INFO - Successfully posted to sensor.p_deferrable1 = 0.0
2024-11-14 18:05:36,290 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 1330.0
2024-11-14 18:05:36,292 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 2.0
2024-11-14 18:05:36,295 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 1350.0
2024-11-14 18:05:36,297 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = -8.6
2024-11-14 18:05:36,298 - web_server - INFO - Successfully posted to sensor.optim_status = Infeasible
2024-11-14 18:05:36,300 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 0.56
2024-11-14 18:05:36,302 - web_server - INFO - Successfully posted to sensor.unit_prod_price = 0.56
2024-11-14 18:05:41,333 - web_server - INFO - EMHASS server online, serving index.html...

mk2lehe · November 14, 2024, 5:36pm

One guess is that your current battery state (soc_init) has dropped below your battery_minimum_state_of_charge (in config.json).

Johan71 · November 14, 2024, 5:50pm

That is not the case, as soon as I send

= Optimal

"def_total_hours":[0, 0],
"def_end_timestep":[0, 0]

Results

"def_total_hours":[0, 0],
 "def_end_timestep": [0, 0]

####################################################
= Infeasible

"def_total_hours":[{{ states('sensor.charger_def_hours') | default(0) }}, 0],
"def_end_timestep": [0, 0]

Results

"def_total_hours":[5, 0],
"def_end_timestep": [0, 0]

RudolfRendier · November 14, 2024, 6:01pm

Probably easiest to confirm that by pasting the json into the box-input in the EMHASS web UI.

altonius · November 14, 2024, 6:13pm

Almost everytime i’ve got an infeasible, it’s to do with my SOC levels.
What’s your configuration for battery_minimum_state_of_charge (in config.json)?

Johan71 · November 14, 2024, 6:32pm

Right now the battery is discharging and is at 4% should go down to 2%. However, I know it happens to me sometimes and then the SOC is lower than they should be.

The funny thing is it was working fine before I upgraded to v0.11

Johan71 · November 14, 2024, 6:43pm

Edited my shell script to
‘def_total_hours’: [5, 0], ‘def_end_timestep’: [7, 0]}
Then it worked immediately

post_naive_mpc_shell_nocar: >
  curl -i -H "Content-Type: application/json" -X POST -d '{
    "load_cost_forecast":{{((state_attr("sensor.nordpool_tibber", "raw_today") | map(attribute="value") | list  + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value") | list))[now().hour:][:24] }},
    "prod_price_forecast":{{((state_attr("sensor.nordpool_tibber", "raw_today") | map(attribute="value") | list  + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value") | list))[now().hour:][:24] }},
    "prediction_horizon":{{min(24, (((state_attr("sensor.nordpool_tibber", "raw_today")|map(attribute="value")|list + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value")| list)[now().hour:][:24]|list|length)))}},
    "pv_power_forecast":{{([states("sensor.inverter_input_power")|int(0)] + state_attr("sensor.solcast_pv_forecast_forecast_today", "detailedHourly")|selectattr("period_start","gt",utcnow()) | map(attribute="pv_estimate")|map("multiply",1000)|map("int")|list + state_attr("sensor.solcast_pv_forecast_forecast_tomorrow", "detailedHourly")|selectattr("period_start","gt",utcnow()) | map(attribute="pv_estimate")|map("multiply",1000)|map("int")|list)| tojson}},
    "load_power_forecast":{{[states('sensor.power_load_novar_loads')|int] +(states('input_text.fifo_buffer').split(',')|map('int')|list)[1:]}},
    "soc_init":{{ max(0,states("sensor.battery_state_of_capacity")|int(0))/100 }},
    "soc_final":{{ max(100,states("number.battery_end_of_discharge_soc")|int(0))/100 }},
    "delta_forecast":2,
    "alpha": 1,
    "beta": 0,
    "def_total_hours":[5, 0],
    "def_end_timestep":[7, 0]
  }' http://localhost:5000/action/naive-mpc-optim

VS Dont work

post_naive_mpc_shell_nocar: >
  curl -i -H "Content-Type: application/json" -X POST -d '{
    "load_cost_forecast":{{((state_attr("sensor.nordpool_tibber", "raw_today") | map(attribute="value") | list  + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value") | list))[now().hour:][:24] }},
    "prod_price_forecast":{{((state_attr("sensor.nordpool_tibber", "raw_today") | map(attribute="value") | list  + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value") | list))[now().hour:][:24] }},
    "prediction_horizon":{{min(24, (((state_attr("sensor.nordpool_tibber", "raw_today")|map(attribute="value")|list + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value")| list)[now().hour:][:24]|list|length)))}},
    "pv_power_forecast":{{([states("sensor.inverter_input_power")|int(0)] + state_attr("sensor.solcast_pv_forecast_forecast_today", "detailedHourly")|selectattr("period_start","gt",utcnow()) | map(attribute="pv_estimate")|map("multiply",1000)|map("int")|list + state_attr("sensor.solcast_pv_forecast_forecast_tomorrow", "detailedHourly")|selectattr("period_start","gt",utcnow()) | map(attribute="pv_estimate")|map("multiply",1000)|map("int")|list)| tojson}},
    "load_power_forecast":{{[states('sensor.power_load_novar_loads')|int] +(states('input_text.fifo_buffer').split(',')|map('int')|list)[1:]}},
    "soc_init":{{ max(0,states("sensor.battery_state_of_capacity")|int(0))/100 }},
    "soc_final":{{ max(100,states("number.battery_end_of_discharge_soc")|int(0))/100 }},
    "delta_forecast":2,
    "alpha": 1,
    "beta": 0,
    "def_total_hours":[{{ states('sensor.charger_def_hours') | default(0) }}, 0],
    "def_end_timestep": [0, 0]
  }' http://localhost:5000/action/naive-mpc-optim

mk2lehe · November 14, 2024, 8:37pm

delta_forecast has been changed to delta_forecast_daily, maybe that’s it

110hs · November 15, 2024, 12:39pm

@davidusb @GeoDerp
I’ve noticed this in the supervisor log, not sure if I should track it among the issues of EMAHSS on github.
Just let me know:

EDIT
Solution here

2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'logging_level' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'costfun' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'sensor_power_photovoltaics' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'sensor_power_load_no_var_loads' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'set_total_pv_sell' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'set_nocharge_from_grid' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'set_nodischarge_to_grid' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'maximum_power_from_grid' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'maximum_power_to_grid' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'number_of_deferrable_loads' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_nominal_power_of_deferrable_loads' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_operating_hours_of_each_deferrable_load' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_start_timesteps_of_each_deferrable_load' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_end_timesteps_of_each_deferrable_load' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_peak_hours_periods_start_hours' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_peak_hours_periods_end_hours' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_treat_deferrable_load_as_semi_cont' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_set_deferrable_load_single_constant' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_set_deferrable_startup_penalty' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'load_peak_hours_cost' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'load_offpeak_hours_cost' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'photovoltaic_production_sell_price' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.083 WARNING (MainThread) [supervisor.addons.options] Option 'list_pv_module_model' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'list_pv_inverter_model' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'list_surface_tilt' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'list_surface_azimuth' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'list_modules_per_string' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'list_strings_per_inverter' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'inverter_is_hybrid' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'compute_curtailment' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'set_use_battery' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_nominal_energy_capacity' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'optimization_time_step' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'historic_days_to_retrieve' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'method_ts_round' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'lp_solver' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'lp_solver_path' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'set_battery_dynamic' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_dynamic_max' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_dynamic_min' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'weight_battery_discharge' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'weight_battery_charge' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'load_forecast_method' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_discharge_power_max' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_charge_power_max' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_discharge_efficiency' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_charge_efficiency' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_minimum_state_of_charge' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_maximum_state_of_charge' does not exist in the schema for EMHASS (5b918bf2_emhass)
2024-11-15 13:34:26.084 WARNING (MainThread) [supervisor.addons.options] Option 'battery_target_state_of_charge' does not exist in the schema for EMHASS (5b918bf2_emhass)

heindlalexander · November 15, 2024, 1:23pm

Hi!

I’m still on 0.10.6 using the addon in Home assistant and I fear to upgrade to 0.11.2.
I use only one deferrable load (no battery) which is my heating system.
What if I upgrade? Are there any manual steps to be done? It seems the configuration is different, so, do I have to migrate that manually? How?
I have never done a restore. What would be the most convenient way to restore back to 0.10.6, if needed? Is this procedure described anywhere (step by step)?

Thanks!

110hs · November 15, 2024, 5:33pm

Before upgrading make sure to copy somewhere your add-on configuration text (here it is shown how to access it, valid also for previous versions of the add-on). You’ll need it later at step 4.
Before upgrading make sure the flag “backup” is on.
Click on upgrade
After the upgrade, which may take a while, follow the instructions below (I linked the minute where the relevant part for you starts) to make sure your configuration is transferred to the new “format”.

Then make sure your sensors (not the default ones) are being used for " Sensor to replace NAN values with 0s" and " Sensor to replace NAN values with linear interpolation". If not, edit the names/lists as needed.
here

That’s it. The shell/restful commands should work straight away.

If you experience any problems you can roll back the add-on: go to Settings → System → Backup.
You should see something like addon_5b918bf2_emhass_0.10.6 (the backup that was created at steps 2 and 3). Select it, check the checkboxes, restore. When the process is over you’ll have your previous version of the add-on back.

RudolfRendier · November 15, 2024, 7:07pm

I’d like to add
6. Remove the add-on yaml config. (Or most of it)

That’s where your logs in the supervisor come from.
More information is here: "Battery settings for the add-on incorrect" in HA Supervisor logs · Issue #103 · davidusb-geek/emhass-add-on · GitHub

Jeza · November 16, 2024, 11:30pm

If this is posted to me, im not 100% on that answer, i would assume it is but how can i verify?

Ive basicly copied marks system and robs within these threads

So if they have a moving mpc then yer this actually would make sence.

I havnt upgraded to the new emhass as its taken me so long to get this system work correctly im too scarsd to go through the upgrade just yet.

How would i get the car to charge and stay charging?