EMHASS: An Energy Management for Home Assistant

vdiogo · April 1, 2023, 9:05am

I have installed EMHASS since some weeks, and my setup is:

PV 4kw
1x deferrable load (water resistance)
regular price/ kw (no difference between night/ day/ weekend)
no selling of eletricitcity to the grid

I have the following results for a typical day from EMHASS:

And have a “dumb” system where, if PV > house consumption for > 2000w, turn the resistance ON

I am assessing replacing my dumb system with EMHASS, so I created a virtual switch “EMHASS” which I’m comparing with my dumb “SHELLY” which I am also comparing:

Snag with EMHASS is that it is turning ON in times of the day where I don’t have PV, meaning it is not an wise suggestion to turn the resistance ON based on EMHASS.

Am I missing any important variable in my setup to get to better results?

davidusb · April 1, 2023, 4:07pm

You are using the default value of prod price of 0.065€/kWh.

Just to be clear, you are injecting your excess PV for free? If that is the case then set that prod price to zero.

vdiogo · April 1, 2023, 6:33pm

Indeed, well spotted! Thanks for checking this

haraldov · April 5, 2023, 7:25pm

days_to_retrieve: the total days to retrieve from Home Assistant for model training. Defined this in order to retrieve as much history data as possible.

If I want to store the sensordata from “sensor.power_load_no_var_loads” longer than 15 days. Can I use Home Assistant long-term statistics for this (Long- and short-term statistics | Home Assistant Data Science Portal)?

I use for the moment purge_keep_days: 40 in the recorder setting and the home-assistant_v2.db size is now 1,211.36 MB which is to big a file for me.

recorder:
  purge_keep_days: 40

Here is the template I use for getting long-term statistics.

      - name: "Power load no var loads"
        unique_id: fbfeef21-1aa3-4a54-b781-426f46fef597
        unit_of_measurement: kWh
        device_class: energy
        state_class: total_increasing
        state: >
          {% set powerload = states('sensor.power_adresse') | float(default=0) %}
          {% set vvb = states('sensor.bryter_varmvannsbereder_electric_consumption_w') | float(default=0) %}
          {% set varmekabler = states('sensor.power_varmekabler') | float(default=0) %}
          {% set varmepumpe = states('sensor.strombryter_varmepumpe_electric_consumption_w') | float(default=0) %}
          {% set value = ( powerload - vvb - varmekabler - varmepumpe) | round(1,default=0) %}
          {{ value }}

davidusb · April 5, 2023, 7:50pm

No, that won’t work. We need the history values of the exact same sensor without any transformation.

The only solution that I see is to increase the purge_keep_days on the recorder. In my case I fixed it to 365 and the database doesn’t grows bigger than 4Gb, which is ok for me since I’m running HA on a mini PC with 256Gb hdd.

In the recorder settings you can state which sensors are to be stored, that’s what I do to keep the database from growing too much.

haraldov · April 5, 2023, 10:41pm

Thanks you for the reply. I going to follow your advice.

I have filtered the entiteties following this guide: How to keep your recorder database size under control, which helped my a lot to get the database smaller.

With Home Assistant 2023.4.0 we get a new database format which has the benefit to have* Smaller (deduplication), less disk usage. Maybe this will help me.

haraldov · April 6, 2023, 10:35am

Here is my updated template for the “sensor.power_load_no_var_loads” without any Home Assistant long-term statistics transformation.

      - name: "Power load no var loads"
        unique_id: fbfeef21-1aa3-4a54-b781-426f46fef597
        unit_of_measurement: W
        device_class:  power
        state: >
          {% set powerload = states('sensor.power_adresse') | float(default=0) %}
          {% set vvb = states('sensor.bryter_varmvannsbereder_electric_consumption_w') | float(default=0) %}
          {% set varmekabler = states('sensor.power_varmekabler') | float(default=0) %}
          {% set varmepumpe = states('sensor.strombryter_varmepumpe_electric_consumption_w') | float(default=0) %}
          {% set value = ( powerload - vvb - varmekabler - varmepumpe) | round(1,default=0) %}
          {{ value }}

Project-bachelor · April 6, 2023, 10:41am

Hi, I have run into some issues with the current setup of EMHASS. Weirdly enough the automated forecast works and shows valid results but after 24 hours when it should do the same thing again in the morning there is a failure to compute. Here are my mistake protocols.

2023-04-05 09:44:26,211 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 170, in action_call
    input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
  File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 91, in set_input_data_dict
    P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'])
  File "/usr/local/lib/python3.9/dist-packages/emhass/forecast.py", line 597, in get_load_forecast
    forecast_out.index = self.forecast_dates
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/generic.py", line 5596, in __setattr__
    return object.__setattr__(self, name, value)
  File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/generic.py", line 769, in _set_axis
    self._mgr.set_axis(axis, labels)
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/internals/managers.py", line 214, in set_axis
    self._validate_set_axis(axis, new_labels)
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/internals/base.py", line 69, in _validate_set_axis
    raise ValueError(
ValueError: Length mismatch: Expected axis has 8 elements, new values have 24 elements

Within a couple minutes I switched the length of the time slots to 30 minutes and this mistake result showed up.

2023-04-05 10:03:53,977 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 170, in action_call
    input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
  File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 91, in set_input_data_dict
    P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'])
  File "/usr/local/lib/python3.9/dist-packages/emhass/forecast.py", line 597, in get_load_forecast
    forecast_out.index = self.forecast_dates
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/generic.py", line 5596, in __setattr__
    return object.__setattr__(self, name, value)
  File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/generic.py", line 769, in _set_axis
    self._mgr.set_axis(axis, labels)
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/internals/managers.py", line 214, in set_axis
    self._validate_set_axis(axis, new_labels)
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/internals/base.py", line 69, in _validate_set_axis
    raise ValueError(
ValueError: Length mismatch: Expected axis has 5 elements, new values have 48 elements

I changed the sensors in the setup but no change. If I restart HA everything works fine, I can optimize, publish and automate. Just the next day there will be this error. The error also persists when I only restart EMHASS. Here is also my config file, if it is of any use.

hass_url: empty
long_lived_token: empty
costfun: self-consumption
optimization_time_step: 30
historic_days_to_retrieve: 2
method_ts_round: nearest
set_total_pv_sell: false
lp_solver: COIN_CMD
lp_solver_path: /usr/bin/cbc
set_nocharge_from_grid: false
load_forecast_method: naive
sensor_power_photovoltaics: input_number.nullsensor
sensor_power_load_no_var_loads: input_number.nullsensor_2
number_of_deferrable_loads: 3
list_nominal_power_of_deferrable_loads:
  - nominal_power_of_deferrable_loads: 450
  - nominal_power_of_deferrable_loads: 60
  - nominal_power_of_deferrable_loads: 50
list_operating_hours_of_each_deferrable_load:
  - operating_hours_of_each_deferrable_load: 3
  - operating_hours_of_each_deferrable_load: 8
  - operating_hours_of_each_deferrable_load: 4
list_peak_hours_periods_start_hours:
  - peak_hours_periods_start_hours: "02:54"
  - peak_hours_periods_start_hours: "17:24"
list_peak_hours_periods_end_hours:
  - peak_hours_periods_end_hours: "15:24"
  - peak_hours_periods_end_hours: "20:24"
list_treat_deferrable_load_as_semi_cont:
  - treat_deferrable_load_as_semi_cont: true
  - treat_deferrable_load_as_semi_cont: true
  - treat_deferrable_load_as_semi_cont: true
load_peak_hours_cost: 4
load_offpeak_hours_cost: 4
photovoltaic_production_sell_price: 0
maximum_power_from_grid: 9000
list_pv_module_model:
  - pv_module_model: Avancis_PowerMax_STRONG_130
  - pv_module_model: Antaris_Solar_AS_P_230
list_pv_inverter_model:
  - pv_inverter_model: SolarEdge_Technologies_Ltd___SE3000__240V_
  - pv_inverter_model: ABB__MICRO_0_3_I_OUTD_US_240__240V_
list_surface_tilt:
  - surface_tilt: 90
  - surface_tilt: 40
list_surface_azimuth:
  - surface_azimuth: 180
  - surface_azimuth: 135
list_modules_per_string:
  - modules_per_string: 7
  - modules_per_string: 1
list_strings_per_inverter:
  - strings_per_inverter: 1
  - strings_per_inverter: 1
set_use_battery: false
battery_discharge_power_max: 1000
battery_charge_power_max: 1000
battery_discharge_efficiency: 0.95
battery_charge_efficiency: 0.95
battery_nominal_energy_capacity: 5000
battery_minimum_state_of_charge: 0.3
battery_maximum_state_of_charge: 0.9
battery_target_state_of_charge: 0.6

davidusb · April 6, 2023, 12:18pm

And what is the shell command that you are using to launch the dayahead-optim?

Project-bachelor · April 6, 2023, 12:24pm

this is it

shell_command:
  dayahead_optim: "curl -i -H \"Content-Type:application/json\" -X POST -d '{}' http://192.168.1.105:5000/action/dayahead-optim"
  publish_data: "curl -i -H \"Content-Type:application/json\" -X POST -d '{}' http://192.168.1.105:5000/action/publish-data"

davidusb · April 6, 2023, 12:45pm

Ok thanks.
Well the problem seems to be with sensor sensor_power_load_no_var_loads:input_number.nullsensor_2.
What you need to do is double check how are you building that sensor. In that part of the code the retrieved data for your case does not have the correct expected length. Any possibilities that your are resetting the sensor at midnight?

xxgmxx · April 9, 2023, 2:01pm

First of all, I would like to thank you for the addon! In spite of trying to set up Emhass for about half of a year, I am still struggling to understand a few things that prevents me from using it to its full potential. Is there a way to create a deferrable load that doesn’t have an operating_hours_of_each_deferrable_load but instead, if there is excess PV power production, it will always run. During the day, my water heater keeps a lower temperature, and before my family showers, it preheats water to a higher temperature. When there is excess energy, that my house can’t use with its own consumption, I wanted to use EMHASS to switch the deferrable load and turn on the water heater. If I specify the operating_hours_of_each_deferrable_load it does turn on while the PV production, but if I set the time for 1 hour, it just turns on for 1 hour. But I am unable to estimate how many hours of sun there will be, and preset the time in the morning. Is there a way to make deferrable_loads that will turn on for no predefined time, to use excess PV energy? If there is a sunny day, I would like to be able to preheat a pool, charge non PV batteries and other stuff that will not turn on when there is a super cloudy day. It is unclear to me whether I missed this function or if Emhass works mostly within predefined working hours and mostly based on peak/off-peak electricity. For me, the peak electricity cost is the same as the off-peak, so I am not using this function. Thank you for any suggestion!

markpurcell · April 9, 2023, 9:42pm

I do something similar.

For my hot water system, I just scheduled it to run for 6 hours and the optimisation almost always selects the solar production time to run.

You could get a little more sophisticated and set the hours to run based on the hours of sunlight expected. Which is what I do for my pool filter pump so it runs for more hours in the summer (when the days are longer) than in the winter. You can see the optimisation almost matches the water heater.

I only run my pool heater for the number of hours that I have very cheap electricity (< 5¢/ kWh), but you could also setup the run hours based on the forecast solar production. I e. 0 hours is solar forecast is below a threshold and then maybe an additional hour for each additional increment in production. My pool heater is 5 kW, so I would run for an additional hour for each 5 kWh of solar production over say a 30 kWh threshold.

I would recommend you start with just one system, hot water and get that working correctly before you try and get multiple systems working.

Project-bachelor · April 10, 2023, 7:20am

Thanks for the reply, the sensor is just a helper putting out a constant zero value. I changed it to a random other sensor since I don’t use the perfect optimization, but it still does not work properly. But anyway, I’m about to submit the thesis for that project so it won’t matter much anymore.
But thanks anyway

langestefan · April 13, 2023, 6:38pm

Thanks for all your detailed responses.

Regarding the list of things to do, I could help with adding support for heatpump modelling?

Specifically I would like to use this paper: Modeling a Domestic All-Electric Air-Water Heat-Pump System for Discrete-Time Simulations

I prefer this paper because it’s very recent, I know the authors personally (and I can ask them questions if needed ) and I have access to the full python simulation code, which I previously used in one of my ‘research’ projects. Unfortunately IEEE asks money to download the paper but I can send it to you for free.

davidusb · April 13, 2023, 7:31pm

No problem. Like I said we can discuss it on github and probably a proof of concept on simple Python scripts is a good idea before adding to the final code.

I’ll have a look to that paper. Don’t worry I’ve access to IEEE. At a first glance it doesn’t treat the LP problem. But 'll have a better look.
The paper from Langer et al is not old (2020) and it provides the complete linear programming model formulation for heat pumps.

markpurcell · April 13, 2023, 11:30pm

I have two air to water heat pumps for domestic hot water (600 W load) and pool heating (three phase 5 kW load) that I am optimising with EMHASS.

I find both of them work fairly well with EMHASS.

Domestic hot water

My 300l system needs to run for about 2 hours per day consuming about 2 kWh to maintain our device. By default it heats to 61°C, but I also have an SG Ready relay (controlled by Shelly) that superheats to 65°C.

Ideally this would run twice, once at the start of the solar day and once the end of the solar day to get me through the night, I have this programmed in EMHASS to run over a non continuous 6 hour window:

Three phase pool heater

I really just run this on an pure low cost basis. I set the def_hours based on the number of hours that the price forecast is extremely_low (< 5¢/ kWh), EMHASS then schedules this along with my other loads and it works quite well.

I don’t think there is much to be improved for air to water heatpumps optimisation, maybe hydronic heating could be a use case that would benefit from modelling.

HVAC

My biggest challenge with heat pump optimisation is my HVAC system. Currently we are still having warm days so there is reasonable alignment between the hottest part of the day and the highest solar production.

I schedule def_hours based on the number of hours the weather forecast is over a user defined set point, I have this set as a continuous load that can take values between 0-4 kW, whilst I cannot control power directly my automation adjusts the set point up and down to increase and decrease power consumption:

This works reasonably well as you can see from the scheduling although there could be some improvements.

You can see the scheduling of the HVAC is start / stop switching the system from 0 to 4000W in adjacent time slots, I would much prefer an average approach of maybe 2.5 kW over three adjacent time slots.

I would like the ability to ‘supercool’ when cheap energy is bountiful, covering periods just before a price spike. I’m my graphic above supercooling from 3-4pm would be ideal utilisation of cheap energy.

Of course the actual power consumption of my HVAC isn’t always the same as the deferrable loads forecast, as it is a function of external temp and setpoint.

Finally in the late afternoon window, after 4pm, I generally have a ongoing need for cooling, but if course EMHASS doesn’t schedule over that expensive period. I find I generally just manually switch on my HVAC during these periods.

Winter scheduling for heating doesn’t align with Solar production, so I generally turn off EMHASS HVAC scheduling over winter and just have that appear in my household load forecast for subsequent days.

langestefan · April 14, 2023, 9:41am

Very cool. I think the general advantage of modelling the heatpump would be the ability to look ahead, temperature wise. Together with temperature forecasts this could help you plan the cycles more efficiently. But ofcourse, we need some practical evidence first before we can make any conclusions.

markpurcell · April 14, 2023, 11:04am

Possibly something like a user defined power forecast for individual loads that could be injected into the optimisation model could be useful.

I know what the forecast external temperature profile is for tomorrow.

A simple power model would be 1kW per degree over say 24°C to a maximum of 4kW and maybe an additional 1kW at 3pm, as my buy price increases at 4pm. So with a one hour time slot the desired power profile for tomorrow would be:

Temp: [22,24,25,26,27,26,26,26,25,24,22,21,20,20]
Power:[ 0, 0, 1, 2, 3, 2, 2, 3, 2, 1, 0, 0, 0, 0]

EMHASS could then use this as another fixed constraint to optimise against.

davidusb · April 14, 2023, 12:19pm

Providing this and using it as an additional constraint can be an option. But this means that you know that model of Temp vs Power consumption.
Another option is to model that based on the Temp and some parameters that are characteristic of the thermal behavior of your house, and to do all this inside EMHASS.
The other goal that I have in mind is that this should be useful to simplify the control of the heat pump. So you will need to provide the Temp forecast and EMHASS will be able to give the optimal temperature setting for your heat pump. Mark, do you think that this approach can be useful? I currently don’t have a heat pump so it is not very clear to me what will be the easiest variables to interface with.