EMHASS: An Energy Management for Home Assistant

Thanks for sharing. Yes that’s way easier :wink:

This sounds exciting and thank you once again! I am trying to figure exactly the application but my understanding is now I can use the outside temperature and historical load data for my heating system to create an hourly forecast for next 24 hours? Currently the ML forecaster can only use one sensor’s data to train the model. Do I understand this correctly or what do I miss?

Thanks!

1 Like

Helped me as well. Rolled back to ‘restore point’ and then updated to v0.9.0
Works fine now.


There appear to be glitch in the EMHASS: When you play with settings profit/cost/self-consumption then EMHASS can get locked in one of the modes.

I have found by actually defining both default values the verbose logging goes away.

Can somebody show how to have a longer history for the load sensor and maybe other sensors, and having all other sensors for only 10 days?

Currently I have:

recorder:
  purge_keep_days: 30

I don’t get the point how to have a different setting for a specific sensor but keep the defaults for all other.

so the answer is no? or didn’t I get the point?

what is your configuration?

No, don’t think you can. My config:

recorder:
  purge_keep_days: 10
  include:
    domains:
      - alarm_control_panel
      - switch
      - group
      - lock
      - light
      - update
    entity_globs:
      - sensor.sonnenbatterie_*
      - sensor.p_*
      - sensor.pb*
      - sensor.pm*
      - sensor.pp*
      - sensor.cecil_st*
      - sensor.solcast*
    entities:
      - input_text.fifo_buffer
      - sensor.optim_status
      - binary_sensor.cecil_st_price_spike
      - binary_sensor.back_bedroom_side_window_contact
      - sensor.amber_count
      - sensor.amber_feedin_price_negative
      - sensor.bathroom_livingroom_humi_diff
      - sensor.batteryinput
      - sensor.batteryoutput
      - sensor.bedroom_humidity
      - sensor.bedroom_temperature
      - sensor.car_charge_cost
      - sensor.car_charge_costs
      - sensor.dining_room_device_temperature
      - sensor.dining_room_illuminance_lux
      - sensor.dw_power
      - sensor.garage_humidity
      - sensor.garage_lux
      - sensor.garage_temperature
      - sensor.garage_power_point_power
      - sensor.garage_power_point_energy
      - sensor.gridinput
      - sensor.gridoutput
      - sensor.house_power_consumption_less_deferrables
      - sensor.humidity
      - sensor.kitchen_motion_device_temperature
      - sensor.kitchen_motion_illuminance_lux
      - sensor.kitchen_smoke_alarm_device_temperature
      - sensor.laundry_powerpoint_power
      - sensor.laundry_powerpoint_energy
      - sensor.living_room_humidity
      - sensor.living_room_temperature
      - sensor.livingroom_motion_device_temperature
      - sensor.livingroom_motion_illuminance_lux
      - sensor.livingroom2_humidity
      - sensor.livingroom2_lux
      - sensor.livingroom2_temperature
      - sensor.medion_plug_power
      - sensor.openweathermap_humidity
      - sensor.openweathermap_temperature
      - sensor.optim_status
      - sensor.powerconsumption
      - sensor.powerproduction
      - sensor.soc_batt_forecast
      - sensor.sonnen_operatingmode
      - sensor.staircase_device_temperature
      - sensor.temperature
      - sensor.total_cost_fun_value
      - sensor.under_floor_humidity
      - sensor.under_floor_temperature
      - sensor.underfloor_outdoor_humi_diff
      - sensor.unit_load_cost
      - sensor.unit_prod_price
      - sensor.upstairs_bathroom_humidity
      - sensor.upstairs_bathroom_lux
      - sensor.upstairs_bathroom_temperature
      - sensor.upstairs_living_room_motion_device_temperature
      - sensor.upstairs_living_room_motion_illuminance_lux
      - sensor.weatherstation_absolute_humidity
      - sensor.weatherstation_sea_level_pressure
      - sensor.weatherstation_w_station_humi
      - sensor.weatherstation_w_station_lux
      - sensor.weatherstation_w_station_pres
      - sensor.weatherstation_w_station_temp
      - sensor.weatherstation2_w_station2_humi
      - sensor.weatherstation2_w_station2_lux
      - sensor.weatherstation2_w_station2_pres
      - sensor.weatherstation2_w_station2_temp
      - sensor.ynot_home_charge_energy
      - binary_sensor.dining_area
      - binary_sensor.dining_room_occupancy
      - binary_sensor.doorbell_person
      - binary_sensor.doorbell_visitor
      - binary_sensor.front_door_open
      - binary_sensor.front_gate
      - binary_sensor.kitchen_smoke_alarm_smoke
      - binary_sensor.side_room
      - binary_sensor.staircase_occupancy
      - binary_sensor.verandagarage
      - binary_sensor.back_bedroom
      - binary_sensor.back_bedroom_side_window_contact
      - binary_sensor.cecil_st_price_spike
      - binary_sensor.doorbell_motion
      - binary_sensor.family_room_motion
      - binary_sensor.garage_motion
      - binary_sensor.hall_motion_occupancy
      - binary_sensor.kitchen_motion_occupancy
      - binary_sensor.laundry_motion
      - binary_sensor.left_dining_room_window
      - binary_sensor.left_living_room_window
      - binary_sensor.lills_room_motion
      - binary_sensor.livingroom_motion_occupancy
      - binary_sensor.livingroom2_motion
      - binary_sensor.master_bedroom_motion_occupancy
      - binary_sensor.right_dining_room_window
      - binary_sensor.right_living_room_window
      - binary_sensor.side_yard_light_motion
      - binary_sensor.stephs_room_motion
      - binary_sensor.stephs_window
      - binary_sensor.tv_area
      - binary_sensor.upstairs_bathroom_motion
      - binary_sensor.upstairs_living_room_motion_occupancy
      - binary_sensor.verandah_motion
      - cover.gate_control

So I only retain 10 days.

1 Like

I tried to use forecast.solar option and I run into this error now:

2024-05-16 11:05:00,214 - web_server - INFO - Passed runtime parameters: {'load_cost_forecast': [14.087, 13.5443, 13.6566, 13.8305, 16.1317, 19.2142, 20.7035, 22.8564, 25.1905, 23.6503, 22.2076, 20.907], 'prod_price_forecast': [-0.257, -0.769, -0.663, -0.499, 1.672, 4.58, 5.985, 8.016, 10.218, 8.765, 7.404, 6.177], 'load_power_forecast': [279, 200, 200, 300, 200, 200, 300, 300, 200, 200, 300, 200, 1400, 300, 400, 400, 300, 400, 200, 300, 400, 300, 600, 400, 500, 500, 400, 500, 500, 200, 500, 600, 300, 200, 200, 500, 500, 400, 300, 300, 200, 200, 200, 300, 300, 300, 200, 400], 'prediction_horizon': 12, 'alpha': 1, 'beta': 0, 'soc_init': 0.44, 'soc_final': 0.12}
2024-05-16 11:05:00,216 - web_server - INFO -  >> Setting input data dict
2024-05-16 11:05:00,216 - web_server - INFO - Setting up needed data
2024-05-16 11:05:00,228 - web_server - INFO - Retrieve hass get data method initiated...
2024-05-16 11:05:00,651 - web_server - INFO - Retrieving weather forecast data using method = solar.forecast
2024-05-16 11:05:00,831 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 109, in action_call
    input_data_dict = set_input_data_dict(emhass_conf, costfun,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 142, in set_input_data_dict
    df_weather = fcst.get_weather_forecast(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 281, in get_weather_forecast
    data_dict = {'ts':list(data_raw['result']['watts'].keys()), 'yhat':list(data_raw['result']['watts'].values())}
                           ~~~~~~~~~~~~~~~~~~^^^^^^^^^
TypeError: string indices must be integers, not 'str'
2024-05-16 11:05:00,853 - web_server - INFO - Passed runtime parameters: {}
2024-05-16 11:05:00,853 - web_server - INFO -  >> Setting input data dict
2024-05-16 11:05:00,853 - web_server - INFO - Setting up needed data
2024-05-16 11:05:00,854 - web_server - INFO -  >> Publishing data...
2024-05-16 11:05:00,855 - web_server - INFO - Publishing data to HASS instance
2024-05-16 11:05:00,866 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 5104.0
2024-05-16 11:05:00,870 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 200
2024-05-16 11:05:00,873 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 2000.0
2024-05-16 11:05:00,877 - web_server - INFO - Successfully posted to sensor.p_deferrable1 = 2500.0
2024-05-16 11:05:00,881 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = -404.0
2024-05-16 11:05:00,885 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 53.55
2024-05-16 11:05:00,888 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 0.0
2024-05-16 11:05:00,891 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = 204.53
2024-05-16 11:05:00,893 - web_server - INFO - Successfully posted to sensor.optim_status = Optimal
2024-05-16 11:05:00,896 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 14.087
2024-05-16 11:05:00,899 - web_server - INFO - Successfully posted to sensor.unit_prod_price = -0.257

I run latest version of EMHASS

Could you please open a github issue for this one. It is possible that the API was updated with a break change for us.

Hi, I raised a bug report for this.

Hiya team,
Firstly a big thanks to everything that’s been written in this thread, I’ve been able to achieve so much. My EMHASS config is now starting to make sense and works pretty well - I’m discovering other ways I automated deferable loads and making sure they use EMHASS only.

My Batteries are DC Coupled with my SolarEdge inverter so I have an odd-ish situation where the max rate I can charge at differs through the day (and from what I can see is impacted by how I configure P_from_grid_max) For example overnight the max that can charge the battery is 5kw as my inverter is only 5kw, whilst through the day i’ve seen the charge rate get up to 9000kw as it’s using both the solar panels (DC side of the inverter, which can get up to 6300kw) plus the grid to charge.

My conundrum is if I set the max charge rate (P_from_grid_max) to 5000, during the day if I use the EMHASS Battery Forecast value in my automations it can restrict my solar charge to battery. Whilst if I set the max charge rate to 9000, during the evening EMHASS thinks it’s going to be able to get more from the grid than the inverter allows

For context I’m running MPC optimise and update every 2 mins, as I’m on Ausgrid / Amber in Australia.

Maybe I’m overthinking things, but I have a couple of ideas on what could try but seeing if anyone else has dealt with this already.

  1. Set the max charge rate to 5000, but add some logic during daylight hours to modify the battery_forecast value before it’s applied to the inverter to make it much higher - EMHASS
  2. See if I can include the P_from_grid_max in an API call to EMHASS, and vary it based on time of day - I’m not sure if this is possible through the MPC endpoint or if I’d need to send to another endpoint (and then not sure on whether changes to that setting required EMHASS to restart)
  3. Set it to a middle figure and let EMHASS work it out through the day when things don’t run to plan.

For context I’m running an MPC optimise and update every 2 mins, as I’m on Ausgrid / Amber in Australia. Any thoughts would be appreciated.

(The final option is to replace my inverter, to say a 10kw one and then know that it can probably charge my batteries from the grid, solar or combined at a higher rate).

Grid buy cost of -3c/ kWh.

EMHASS is doing everything it can to keep up…

index P_PV P_Load P_deferrable0 P_deferrable1 P_deferrable2 P_deferrable3 P_deferrable4 P_deferrable5 P_grid_pos P_grid_neg P_grid P_batt SOC_opt unit_load_cost unit_prod_price cost_profit cost_fun_profit
2024-05-18 12:30:00+10:00 0 400 0 6333 11520 0 600 11520 40000 0 40000 -9627 0.715 -0.03 -0.09 0.6 0.6
2024-05-18 13:00:00+10:00 9332 600 0 6333 11520 0 600 11520 36241 0 36241 -15000 0.879 -0.03 -0.09 0.544 0.544
2024-05-18 13:30:00+10:00 8684 800 0 6333 11520 0 600 11520 31391 0 31391 -9302 0.98 0.01 -0.05 -0.157 -0.157
2024-05-18 14:00:00+10:00 7782 500 0 0 0 0 0 5479 0 0 0 -1802 1 0.02 -0.05 0 0
2024-05-18 14:30:00+10:00 6470 300 0 0 11520 0 0 1707 4839 0 4839 2217 0.975 0.02 -0.05 -0.048 -0.048
2024-05-18 15:00:00+10:00 5033 700 0 0 0 0 0 4333 0 0 0 0 0.975 0.02 -0.05 0 0
2024-05-18 15:30:00+10:00 3534 600 0 0 0 0 600 0 0 0 0 -2334 1 0.03 -0.04 0 0
2024-05-18 16:00:00+10:00 1939 400 0 0 0 0 0 0 0 -1539 -1539 0 1 0.25 0.03 0.023 0.023
2024-05-18 16:30:00+10:00 546 300 0 0 0 0 0 0 0 -246 -246 0 1 0.26 0.04 0.005 0.005
2024-05-18 17:00:00+10:00 0 2000 0 0 0 0 0 0 0 -5500 -5500 7500 0.914 0.33 0.1 0.275 0.275

Similar pattern here, everytime there is a bank holiday weekend the prices drop below -/-150 euro per megawatt. Which, including taxes, results is negative net prices.

I manually curtail my solar panels - so far it happend 3 times this calendar year so no need to automate. However, as wind and solar continues to expand the frequency will increase. Working on a script to shutdown my Enphase, growatt is more complicated, i can limit export but not shutdown without also shutting down the battery.

It would be great to feedback somehow the PV curtailment to EMHASS so that the scenarios can take it into account upfront.

1 Like

Correct if I’m wrong but your problem seems similar to:
https://github.com/davidusb-geek/emhass/issues/220
Which is just the fact that hybrid inverters are not supported yet.
But they will in the near future.

No, we cannot pass multiple features to the ML forecaster (just yet). Hopefully we will add this functionality in the future. Like I was explaining in my post, these are just regression models. The timestamp of the regression prediction are the same as the input data to the regressor. This is different to the ML forecaster where the prediction is provided for future timestamps. Forecasting concerns to out of sample observations, whereas prediction using a regression model concerns to in sample observations. In order to achieve this the ML forecaster uses an auto-regression approach.

A great ressource describing this can be found here: https://www.sktime.net/en/stable/examples/01a_forecasting_sklearn.html

There are many interesting uses cases for this new regression module.
In our case for energy management I think that a good use case for now is what I was saying that @gieljnssns is already doing with this tool: using it to predict the def_total_hours based on other variables.

This is exactly my current situation.

1 Like

why does my P Deferrable 2 load move all the time It started to be scheduled last night 00:00 but after a few hours it has moved forward until tomorrow morning. it applies to my electric car so I want it to charge as soon as possible

Every time you rerun the optimisation EMHASS will recalculate the best time to charge your car. If your prices change dynamically this can move the scheduled time around.

You can specify additional constraints for when the load should be scheduled.

I have defined a input helper that can constrain my EV charging to be ready before I depart in the morning, e.g. be completed by 8 am.

That’s really interesting, I’m trying to work on something like this for other loads, however I’m struggling with the 30 mins steps and how they differ from specific times when sending to the MPC endpoint through the day.
If you’re doing MPC optimisations every minute what’s the logic you use to tell emhass the finish time for that deferrable load?