EMHASS: An Energy Management for Home Assistant

What does your jinja code produce when you test in Developer Tools Template? What does your pricing sensor look like?
My dayahead jinja code looks like this:

{
  "load_cost_forecast": {{
    ([states('sensor.cecil_st_general_price')|float(0)] + 
    state_attr('sensor.cecil_st_general_forecast', 'forecasts') |map(attribute='per_kwh')|list) 
    | tojson 
  }},
  "prod_price_forecast": {{
    ([states('sensor.cecil_st_feed_in_price')|float(0)] +
    (state_attr('sensor.cecil_st_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list))
    | tojson 
  }},
  "pv_power_forecast": {{
    ([states('sensor.sonnenbatterie_84324_production_w')|int(0)] +
    state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list +
    state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list
    )| tojson
  }},
  "num_def_loads": 2,
  "def_total_hours": [3,0],
  "P_deferrable_nom":  [1300, 7360],
  "treat_def_as_semi_cont": [1, 0]  
}

Result from Developer Tools Template looks like this:

{
  "load_cost_forecast": [
    0.16,
    0.16,
    0.16,
    0.16,
    0.16,
    0.16,
    0.16,
    0.15,
    0.13,
    0.13,
    0.13,
    0.16,
    0.16,
    0.16,
    0.16,
    0.13,
    0.11,
    0.1,
    0.1,
    0.1,
    0.1,
    0.08,
    0.04,
    0.03,
    0.03,
    0.02,
    0.02,
    0.02,
    0.02,
    0.27,
    0.27,
    0.28,
    0.29,
    0.3,
    0.32,
    0.35,
    0.36,
    0.38,
    0.42,
    0.42,
    0.45,
    0.19,
    0.16,
    0.16,
    0.16,
    0.15,
    0.13,
    0.15,
    0.13
  ],
  "prod_price_forecast": [
    0.06,
    0.06,
    0.06,
    0.06,
    0.06,
    0.06,
    0.06,
    0.05,
    0.04,
    0.04,
    0.04,
    0.06,
    0.06,
    0.06,
    0.06,
    0.04,
    0.02,
    0.01,
    0.01,
    0.01,
    0.01,
    -0.03,
    -0.07,
    -0.07,
    -0.08,
    -0.08,
    -0.08,
    -0.08,
    -0.09,
    0.2,
    0.2,
    0.21,
    0.21,
    0.22,
    0.24,
    0.27,
    0.28,
    0.29,
    0.33,
    0.33,
    0.36,
    0.09,
    0.06,
    0.06,
    0.06,
    0.06,
    0.04,
    0.06,
    0.04
  ],
  "pv_power_forecast": [
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    5,
    58,
    345,
    861,
    1411,
    1992,
    2500,
    2941,
    3314,
    3608,
    3856,
    4004,
    4049,
    4064,
    4090,
    4020,
    3854,
    3642,
    3354,
    3020,
    2599,
    2130,
    1554,
    1031,
    503,
    89,
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    0,
    0
  ],
  "num_def_loads": 2,
  "def_total_hours": [
    3,
    0
  ],
  "P_deferrable_nom": [
    1300,
    7360
  ],
  "treat_def_as_semi_cont": [
    1,
    0
  ]
}

My pricing data looks like this:

Deferrable loads are things like EV charging, hot water, pool pump etc. These loads need to be automated so EMHASS can control them and their consumption subtracted from household consumption.
Maybe this might help my doco

1 Like

Thanks, that looks like it works.

I’m still struckling with the battery forecast SOC and the actual SOC of the battery.

I don’t use the forecast soc value in a automation.

The capacity is correctly configured. Any idea how to solve this?

See also the screenshot:

The blue line is the Battery forecast SOC of EMHASS, purple the actual SOC from the inverter.

See my config:

hass_url: empty
long_lived_token: empty
costfun: profit
logging_level: INFO
optimization_time_step: 60
historic_days_to_retrieve: 3
method_ts_round: first
set_total_pv_sell: false
lp_solver: COIN_CMD
lp_solver_path: /usr/bin/cbc
set_nocharge_from_grid: false
set_nodischarge_to_grid: false
set_battery_dynamic: false
battery_dynamic_max: 0.92
battery_dynamic_min: -0.92
load_forecast_method: naive
sensor_power_photovoltaics: sensor.solaredge_ac_power
sensor_power_load_no_var_loads: sensor.house_usage_in_watt
number_of_deferrable_loads: 2
list_nominal_power_of_deferrable_loads:
  - nominal_power_of_deferrable_loads: 800
  - nominal_power_of_deferrable_loads: 10800
list_operating_hours_of_each_deferrable_load:
  - operating_hours_of_each_deferrable_load: 2
  - operating_hours_of_each_deferrable_load: 4
list_peak_hours_periods_start_hours:
  - peak_hours_periods_start_hours: "02:54"
  - peak_hours_periods_start_hours: "17:24"
list_peak_hours_periods_end_hours:
  - peak_hours_periods_end_hours: "15:24"
  - peak_hours_periods_end_hours: "20:24"
list_treat_deferrable_load_as_semi_cont:
  - treat_deferrable_load_as_semi_cont: false
  - treat_deferrable_load_as_semi_cont: false
load_peak_hours_cost: 0.1907
load_offpeak_hours_cost: 0.1419
photovoltaic_production_sell_price: 0.065
maximum_power_from_grid: 17250
list_pv_module_model:
  - pv_module_model: Jinko_Solar_Co___Ltd_JKM410M_72HL
list_pv_inverter_model:
  - pv_inverter_model: SolarEdge_Technologies_Ltd___SE9K__208V_
list_surface_tilt:
  - surface_tilt: 45
list_surface_azimuth:
  - surface_azimuth: 210
list_modules_per_string:
  - modules_per_string: 26
list_strings_per_inverter:
  - strings_per_inverter: 1
set_use_battery: true
battery_discharge_power_max: 3300
battery_charge_power_max: 3300
battery_discharge_efficiency: 0.95
battery_charge_efficiency: 0.95
battery_nominal_energy_capacity: 14500
battery_minimum_state_of_charge: 0.1
battery_maximum_state_of_charge: 1
battery_target_state_of_charge: 0.1

This is mine. Don’t know if it indicates anything is wrong.

I have a powerwall and I use the SOC forecast to set the backup reserve %.

The battery responds like this:

  • set reserve < actual SOC - stores excess solar and discharges battery to meet excess household load.
  • reserve = SOC - stores excess solar and doesn’t discharge battery excess household load is drawn from the grid
  • reserve > SOC - forces battery to charge slowly (5.5 kW) from solar+grid

I also use the p_batt_forecast to change charging rates

  • batt_forecast < -5.5 kW backup reserve % above
  • batt_forecast < -10 kW set mode backup - forces charging at 10 kW
  • batt_forecast < -15 kW set mode time_based_control if battery tou is off_peak - forces charging at 15 kW
  • batt_forecast > 15 kW set mode time_based_control if battery tou is peak - forces discharging at 15 kW

The discharging control is a work around and other batteries have much better controls.

Can you tell us what type of battery you have and what automations you are using to control it?

For sonnen batteries, control is leveraged through POST command to switch between automatic and manual mode.

In manual mode POST commands to charge and discharge at whatever wattage and set backup buffer to whatever percentage of battery SOC. Backup buffer = Battery reserve.

I only use backup buffer to force standby state when in manual mode by switching to 100% standby as there is no stop charge/discharge command.

Automatic mode is self-consumption mode.

Would be good if EMHASS had a sensor for self-consumption as the battery follows house consumption instantly while EMHASS only adjusts to house consumption roughly depending on frequency of POST command to EMHASS.

Max charge or discharge is only 3300 Watts and I notice that the battery drifts from the requested wattage as the temperature rises and will sometimes stop charging altogether on very hot days after a few hours.

Currently I only use the p_batt_forecast level currently. This value will be send to my inverter. As you can see in the chart. The value looks not good enough for some reason.

Which battery do you have?

Maybe include a screen shot showing p_batt_forecast and your actual battery power, your earlier charts look like the battery is following the soc_forecast quite well.

Here is mine.

I have home build battery. LifePo4 16x280A. My inverter requires a -value for discharging and + value for charging. So the chart is the opposite. So the charge/discharge looks the same.

But the SOC is in my opinion not great.

Hi folks, something odd has just started happening with my EMHASS setup. Has anyone seen this?

2023-10-17 09:25:20,077 - web_server - INFO - Setting up needed data
2023-10-17 09:25:20,123 - web_server - INFO - Retrieve hass get data method initiated...
2023-10-17 09:25:20,326 - web_server - INFO - Retrieving weather forecast data using method = list
2023-10-17 09:25:20,330 - web_server - INFO -  >> Performing naive MPC optimization...
2023-10-17 09:25:20,330 - web_server - INFO - Performing naive MPC optimization
2023-10-17 09:25:20,335 - web_server - INFO - Perform an iteration of a naive MPC controller
2023-10-17 09:25:21,542 - web_server - ERROR - It was not possible to find a valid solver for Pulp package
2023-10-17 09:25:21,543 - web_server - INFO - Status: Not Solved
2023-10-17 09:25:21,543 - web_server - WARNING - Cost function cannot be evaluated, probably None
2023-10-17 09:25:21,546 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 204, in action_call
    opt_res = naive_mpc_optim(input_data_dict, app.logger)
  File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 275, in naive_mpc_optim
    opt_res_naive_mpc = input_data_dict['opt'].perform_naive_mpc_optim(
  File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 575, in perform_naive_mpc_optim
    self.opt_res = self.perform_optimization(df_input_data, P_PV.values.ravel(), P_load.values.ravel(),
  File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 418, in perform_optimization
    opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I]
  File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 418, in <listcomp>
    opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I]
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'

Here is the call to the MPC action:

2023-10-17 09:25:21.547 WARNING (MainThread) [homeassistant.components.rest_command] Error. Url: http://localhost:5000/action/naive-mpc-optim. Status code 500. Payload: b'{\n "load_power_forecast":\n [5901, 2206, 2093, 2428, 2246, 6518, 6430, 7149, 7360, 6744, 6559, 5878, 5942, 2502, 1947, 2536, 2013, 2187, 2731, 3890, 3416, 3042, 3175, 3009, 2452, 2029, 2288, 1622, 1276, 5147, 5428, 5503, 5485, 5584, 5446, 5574, 5482, 1200, 975, 1039, 935, 903, 1006, 1269, 1514, 5860, 6162, 6186], \n 
"load_cost_forecast":\n [0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.24915, 0.24915, 0.24915, 0.24915, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.24915, 0.24915, 0.24915, 0.24915], \n 
"pv_power_forecast":\n [4075, 12982, 17000, 17000, 17000, 17000, 16941, 15385, 14479, 14099, 13692, 13145, 12344, 11260, 9803, 8082, 5981, 3706, 1389, 285, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 30, 500, 2118, 4359, 6440, 8178, 9801, 11289, 12653, 13960, 14948, 15643, 16047, 16385, 16389, 16075, 15700, 15153, 14322, 13266, 11637, 9413, 6971, 4469, 1917, 375, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], \n 
"prediction_horizon":\n 48, \n 
"alpha":\n 0.2, \n "beta":\n 0.8, \n "soc_init":\n 0.15, \n "soc_final":\n 0.15, \n 
"set_def_constant":\n ["false","true","true"], \n 
"def_total_hours":\n [ 0, \n 7,\n 3]\n}'

All the newlines come from template formatting. I assume that is not an issue.

When I test the above between the {} at this json test site jsonlint.com I get this:

Invalid JSON!
Error: Parse error on line 1:
{\n "load_power_forec
-^
Expecting 'STRING', '}', got 'undefined'

If I take all the new lines out like this:

{ "load_power_forecast": [5901, 2206, 2093, 2428, 2246, 6518, 6430, 7149, 7360, 6744, 6559, 5878, 5942, 2502, 1947, 2536, 2013, 2187, 2731, 3890, 3416, 3042, 3175, 3009, 2452, 2029, 2288, 1622, 1276, 5147, 5428, 5503, 5485, 5584, 5446, 5574, 5482, 1200, 975, 1039, 935, 903, 1006, 1269, 1514, 5860, 6162, 6186],  
"load_cost_forecast": [0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.24915, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.46178, 0.24915, 0.24915, 0.24915, 0.24915, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.19239, 0.24915, 0.24915, 0.24915, 0.24915],  
"pv_power_forecast": [4075, 12982, 17000, 17000, 17000, 17000, 16941, 15385, 14479, 14099, 13692, 13145, 12344, 11260, 9803, 8082, 5981, 3706, 1389, 285, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 30, 500, 2118, 4359, 6440, 8178, 9801, 11289, 12653, 13960, 14948, 15643, 16047, 16385, 16389, 16075, 15700, 15153, 14322, 13266, 11637, 9413, 6971, 4469, 1917, 375, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],  
"prediction_horizon": 48,  
"alpha": 0.2,  "beta": 0.8,  "soc_init": 0.15,  "soc_final": 0.15,  
"set_def_constant": ["false","true","true"],  
"def_total_hours": [ 0,  7, 3]}

I get this:

JSON is valid!

It seems I just need to restart the EMHASS containers. Now I am back to battling infeasible solutions from the optimizer. I assume it is because I am trying to schedule pool pumps that are semi_continuous (ie 0 or nominal power) and I want minimal starts and stops so am setting def_constant to true. I’ll play around with set_def_constant to see if that helps.

After some initial problems with the (build-in) configuration, I have EMHASS up and running. From here I will start reading the docs and follow this forum to get myself comfortable with this extensive add-on.

I’m still a bit confused about the sensor_power_load_no_var_loads sensor. This is supposed to be the house consumption minus the deferrable loads. That’s clear to me:

sensor_power_load_no_var_loads = Phouse - Pdeferrable_loads

But I am confused about the house consumption sensor, should this be all power coming into the house minus everything going out:

Phouse = Pfromgrid + Psolar + Pfrombattery - Ptogrid - Ptobattery

Or should it just be all power consumed from the grid:

Phouse = Pfromgrid

Otherwise you miss your household load covered by the PV production

Hello,

A question for those using/with experience with the ML forecaster.
Once you have a trained and tuned model, then what are the next steps to use it to forecast your load?
Are you supposed to:

  • just update load_forecast_method: mlforecaster in config and when you run the dayahead/MPC optim it will take care of predicting the load for the right future timeframe
  • launch the forecast-model-predict every time before running the optimization and still change load_forecast_method: mlforecaster in config
  • launch the forecast-model-predict every time before running the optimization, publish the results and pass them to the optim as a list, hence change load_forecast_method: list in config
  • anything else

?

On top of this question I’m trying to understand if the ML method is good enough for me, as I have limited hardware and the max R2 I can achieve after tuning (when it doesn’t return an error - I fear is because of the hardware - for those who are curious this is the error I get from time to time) is about 0.5, so if it’s really worth using this approach.

Thank you!

Thanks for your reaction!

Just this and none of the other options you listed

1 Like

We cannot answer this for you.
Take the data history of the forecast with the default method and using the history of your load data compute the R2 of that. If its better than 0.5 then stick to that, otherwise switch to the ML model. Try to use enough representative data.

1 Like

Now that I am bringing pool heating and pool cleaning into EMHASS, I need to ensure that a minimum number of pool cleaning hours happens each day. I am setting def_hours for the pool cleaner based on how much time is has been running already today. I want to make sure that I get X hours min per day and the scheduling doesn’t keep moving the load to the next day.

What are my control levers here? Instead of having def_hours gradually reduce (ie receding horizon behaviour) over the day, should I keep it at X hours until the actual run time (for the day) is close to X and then reduce def_hours sharply?