EMHASS: An Energy Management for Home Assistant

Jojo1 · March 4, 2024, 9:05am

I don’t know what i am missing here. Whay are the battery charging when the prises are high. Battery forecast -2000 ( - xxxx => charging) but the soc are the same. The Emhass is configured for profit.

alias: EMHASS Naive MPC optim call
mode: single
variables:
  json_string: >
      "prediction_horizon":13, "soc_init": 0.1, "soc_final": 0.0, "num_def_loads": 7,  "def_total_hours":
    [1,1,2,2,2,2,1],  "P_deferrable_nom": [500,1100,1000,1000,1000,1000,1000],
    "treat_def_as_semi_cont": [1,1,1,1,1,1,1],
    "def_start_timestep":[0,0,0,0,0,0,0], "def_end_timestep":[0,0,0,0,0,0,0],
    "set_def_constant": [false,false,false,false,false,false,false],
    "load_cost_forecast":[0.2011, 0.2011, 0.1911, 0.1841, 0.1801, 0.1931, 0.1951, 0.1971, 0.2061, 0.1991, 0.1761, 0.1801, 0.2021],  "prod_price_forecast":
    [0.0963, 0.0963, 0.0882, 0.0826, 0.0794, 0.0899, 0.0915, 0.0931, 0.1003, 0.0947, 0.0761, 0.0794, 0.0971], "pv_power_forecast":[192, 151, 172, 163, 98, 44, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 64, 170, 279], "alfa": 0.45, 
    "beta": 0.55

davidusb · March 4, 2024, 9:30am

This: optim_status = Infeasible.
When you have this status don’t try to overthink about the results, the solver for some reason did not converge to a solution and most of the time the numerical values of that solution don’t make any sense.
In this situation we need to try to find why it is Infeasible.
This status means that mathematically you are adding too much constraint to the system. One fast solution to try could to extend to a longer prediction_horizon.
A second quick solution could be to test with a lower number of deferrables. You have 7. Try with two, if it works then go for three, and so on…

davidusb · March 4, 2024, 10:08am

Yes, it is the drawback.

Jojo1 · March 4, 2024, 12:10pm

Hello David

Any more ides? I have tryed to do what you rekommended but I still get Infeasible.

davidusb · March 4, 2024, 12:42pm

Try "soc_init": 1.0.
Also it’s alpha not alfa.
And try "treat_def_as_semi_cont": [true,true,true,true,true,true,true]

rcruikshank · March 4, 2024, 9:54pm

I’m using this infeasible/optimal sensor to switch the battery to self consumption mode at the end of the day. I set the battery_minimum_state_of_charge to 24% and when the battery drops to 23% I use this infeasible state to disengage EMHASS control over the battery and set the battery to self consumption mode. The sonnen battery can then use its own management logic to use the tail end of the battery capacity overnight.

This is because my 5 year old battery does not perform very well at this lower state of charge which happens to occur at the end of the 2PM to 8PM window where the feed-in tariff is peaking in the two way tariff plans we have in Australia.

The system remains infeasible overnight and the battery discharges as best it can under its own self consumption logic. Then in the morning self consumption will start to charge the battery as the PV starts to exceed the household consumption and the system returns to optimal mode on it’s own. If the day is not sunny sometimes I have to trigger charging from the grid to return to optimal.

heindlalexander · March 6, 2024, 6:59am

Hi,

I have a strange error. I run mpc optimization every day at 7:00 am and 7:00 pm, publishing the data every hour a few seconds later (I use 1 hour as intervall). This worked for me until upgrading to 0.8.1 from the latest before. The one at 7:00 am fails for some reason. when I run it just a few minutes later it works. Does anyone know what the reason could be?

here is the output:

2024-03-06 07:00:00,819 - web_server - INFO -  >> Setting input data dict
2024-03-06 07:00:00,820 - web_server - INFO - Setting up needed data
2024-03-06 07:00:00,841 - web_server - INFO - Retrieve hass get data method initiated...
2024-03-06 07:00:03,879 - web_server - INFO - Retrieving weather forecast data using method = list
2024-03-06 07:00:03,883 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2024-03-06 07:00:03,884 - web_server - INFO - Retrieve hass get data method initiated...
2024-03-06 07:00:43,998 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 103, in action_call
    input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 127, in set_input_data_dict
    P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'], set_mix_forecast=True, df_now=df_input_data)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 603, in get_load_forecast
    forecast_out.index = self.forecast_dates
    ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/generic.py", line 6002, in __setattr__
    return object.__setattr__(self, name, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pandas/_libs/properties.pyx", line 69, in pandas._libs.properties.AxisProperty.__set__
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/generic.py", line 730, in _set_axis
    self._mgr.set_axis(axis, labels)
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/managers.py", line 225, in set_axis
    self._validate_set_axis(axis, new_labels)
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/base.py", line 70, in _validate_set_axis
    raise ValueError(
ValueError: Length mismatch: Expected axis has 0 elements, new values have 12 elements

bert1111 · March 6, 2024, 9:56am

I had something similar after updating to 0.8.1. See my post above. I went back to 0.8.0 because there was no answer to my post and I would try again when there will be a new update.

erikh · March 6, 2024, 12:42pm

@sladey

I just had a quick question on the new def load start/end config parameters. They are a step number in the current prediction horizon. So if you regularly call the MPC optimiser and you want your def loads to start and stop at the same “real time” each day, then you need to recalculate you start/end timesteps to reflect the current time when the MPC optimiser is run? That is a long question, so let me try again. If I want my pool pumps to only be scheduled between 7:00 and 17:00 then I need to keep recalculating def_start_timestep and def_end_timestep such that their timestep values equate to 7:00 and 17:00 in real time?

I’m passing all variables for the deferrable loads dynamically to EMHASS (via a MPC call every 10 minutes, with a timestep of 30 minutes).

For the deferrable timesteps I made several helpers:

input_boolean.emhass_p_deferrable_0_timestep input_datetime.emhass_p_deferrable_0_start_time input_datetime.emhass_p_deferrable_0_end_time

The 2 input_datetime helpers (with time only, no date) contain the actual start /end time during the day. Say, every evening between 20:00 and 23:00.

Also I made 2 sensors to convert the fixed start/end hours to start/end timesteps:

  - name: def_0_start_timestep
    state: "{{ is_state('input_boolean.emhass_p_deferrable_0_timestep','on')|int * ((today_at(states('input_datetime.emhass_p_deferrable_0_start_time'))|as_timestamp - now()|as_timestamp)/3600)|int }}"
 
  - name: def_0_end_timestep
    state: >
      {% if states('input_datetime.emhass_p_deferrable_0_start_time') > states('input_datetime.emhass_p_deferrable_0_end_time') %}
        {{ is_state('input_boolean.emhass_p_deferrable_0_timestep','on')|int * ((today_at(states('input_datetime.emhass_p_deferrable_0_end_time'))|as_timestamp - now()|as_timestamp)/3600)|int + 24 }}
      {% else %}
        {{ is_state('input_boolean.emhass_p_deferrable_0_timestep','on')|int * ((today_at(states('input_datetime.emhass_p_deferrable_0_end_time'))|as_timestamp - now()|as_timestamp)/3600)|int }}
      {% endif %}

If the end_time < start_time it means the end time is the next day, in which case I add 24 hours to that end timestep.

These sensors I pass to my mpc-optim call:

      \"def_start_timestep\": {{ [states('sensor.def_0_start_timestep')|int(0)*2, states('sensor.def_1_start_timestep')|int(0)*2, states('sensor.def_2_start_timestep')|int(0)*2] }},
      \"def_end_timestep\": {{ [states('sensor.def_0_end_timestep')|int(0)*2,states('sensor.def_1_end_timestep')|int(0)*2,states('sensor.def_2_end_timestep')|int(0)*2] }}

As you can see I’m doubling the timesteps, ....|int(0)*2, to meet my 30 minutes EMHASS timestep.

Everything dynamically controlled with a Lovelace card:

All working fine for me on a rolling EMHASS system.

heindlalexander · March 6, 2024, 1:20pm

I’ve seen your post, but I think it is different.
In my case, I assume there is something wrong with the load forcast. For what I “understand”, it has 0 elements, but it should have been 12. That’s the number of values I passed for the other data, but I didn’t pass load data (only cost)

here is my command, which would have 12 values in the morning (now only 5):

curl -i -H "Content-Type: application/json" -X POST -d '{"set_def_constant":[true],"pv_power_forecast":[12211.6, 8387.9, 6232.599999999999, 1024.0, 0.0],"def_total_hours":[2],"prediction_horizon":5,"load_cost_forecast":[15.130704000000001, 15.412512, 16.38648, 18.575436, 21.131484],"prod_price_forecast":[9.21, 9.22, 9.23, 9.24, 9.25]}' http://localhost:5000/action/naive-mpc-optim

heindlalexander · March 7, 2024, 7:07am

today the same problem.

I now captured the exact command. It’s the same in both cases. The first try fails, the second succeeds:

curl -i -H "Content-Type: application/json" -X POST -d '{
"set_def_constant":[true],
"pv_power_forecast":[1746.6999999999998, 4592.7, 6298.7, 5569.6, 5535.700000000001, 5293.400000000001, 4908.8, 4581.5, 3989.7, 1616.6000000000001, 162.70000000000002, 0.0],
"def_total_hours":[4],
"prediction_horizon":12,
"load_cost_forecast":[19.15512, 18.35172, 16.175124, 15.213516000000002, 14.576975999999998, 14.003472, 13.829196, 14.207412000000001, 14.997216000000002, 15.68196, 17.547084, 19.16748],
"prod_price_forecast":[9.26, 9.25, 9.24, 9.23, 9.22, 9.21, 9.2, 9.21, 9.22, 9.23, 9.24, 9.25]
}' http://localhost:5000/action/naive-mpc-optim

GeoDerp · March 9, 2024, 1:16am

@heindlalexander I have modified your example and tested, using a version that may be released soon. (testing in Docker standalone)


curl -i -H "Content-Type: application/json" -X POST -d '{

"set_def_constant":[true,false],

"pv_power_forecast":[1746.6999999999998, 4592.7, 6298.7, 5569.6, 5535.700000000001, 5293.400000000001, 4908.8, 4581.5, 3989.7, 1616.6000000000001, 162.70000000000002, 0.0],

"def_total_hours":[4,0],

"prediction_horizon":12,

"load_cost_forecast":[19.15512, 18.35172, 16.175124, 15.213516000000002, 14.576975999999998, 14.003472, 13.829196, 14.207412000000001, 14.997216000000002, 15.68196, 17.547084, 19.16748],

"prod_price_forecast":[9.26, 9.25, 9.24, 9.23, 9.22, 9.21, 9.2, 9.21, 9.22, 9.23, 9.24, 9.25]

}' http://MYIP:5000/action/naive-mpc-optim

Everything seem to work first try.
This could be because its fixed by the changes made. Or, some sort of issue with the EMHASS-Add-on in particular (E.x. setting the data dir to /data for precipitant storage, introduced in 0.8.1)

If I don’t test this before the release of PR: Docker, requirements and amhf support by GeoDerp · Pull Request #216 · davidusb-geek/emhass · GitHub, try updating and testing again. If the issue persists we may need to look into the two changes made in 0.8.1 (/data and default ip on addon empty)

erikh · March 10, 2024, 11:06am

Update on my earlier post on translating “realtime” time-window for deferrables to rolling start/end_timesteps.

I was having a problem with the earlier published sensors: when a deferrable load was scheduled during the evening/night, say run between 20:00 and 08:00 the following morning, the whole time-window would shift to the next evening as soon as the time passed midnight. So I had to build in some extra constraints to only shift the complete time-window to the next evening àfter the present time passed the planned endtime.

Now() > deferrable_x_end_time

  - name: def_0_start_timestep
    state: >
      {% set ts = (((today_at(states('input_datetime.emhass_p_deferrable_0_start_time'))|as_timestamp - now()|as_timestamp)+900)/1800) | round(0) |  int(0) %}
      {% if states('input_datetime.emhass_p_deferrable_0_start_time') > states('input_datetime.emhass_p_deferrable_0_end_time') %}
        {% if now() < today_at(states('input_datetime.emhass_p_deferrable_0_end_time')) %}
          {% set ts = ts - 48 %}
        {% endif %}
      {% endif %} 
      {{ ts * is_state('input_boolean.emhass_p_deferrable_0_timestep','on') }} 

  - name: def_0_end_timestep
    state: >
      {% set ts = (((today_at(states('input_datetime.emhass_p_deferrable_0_end_time'))|as_timestamp - now()|as_timestamp) +900)/1800) | round(0) | int(0) %}
      {% if states('input_datetime.emhass_p_deferrable_0_start_time') > states('input_datetime.emhass_p_deferrable_0_end_time') %}
        {% if now() > today_at(states('input_datetime.emhass_p_deferrable_0_end_time')) %}
          {% set ts = ts + 48 %}
        {% endif %}
      {% endif %} 
      {{ ts * is_state('input_boolean.emhass_p_deferrable_0_timestep','on') }}

Also, I changed the shell / rest command payload for the mpc-optim:

      "def_start_timestep": {{ [states('sensor.def_0_start_timestep')|int(0), states('sensor.def_1_start_timestep')|int(0), states('sensor.def_2_start_timestep')|int(0)] }},
      "def_end_timestep": {{ [states('sensor.def_0_end_timestep')|int(0),states('sensor.def_1_end_timestep')|int(0),states('sensor.def_2_end_timestep')|int(0)] }}

(Last update: added 900 (=15minutes) to time difference equation to make the timesteps decrease by 1 on every full or half hour. )
Everything seems to be running fine now…

erikh · March 11, 2024, 8:31am

EMHASS update from 0.8.1 to 0.8.2 went very smoothly.

New option to show the payload in the log is very nice!

2024-03-11 09:20:02,447 - web_server - INFO - Using core emhass version: 0.8.2
waitress   INFO  Serving on http://0.0.0.0:5000
2024-03-11 09:20:10,306 - web_server - INFO - Passed runtime parameters: {
'load_cost_forecast': [0.25749, 0.25749, 0.25227, 0.25227, 0.24346, 0.24346, 0.24236, 0.24236, 0.23582, 0.23582, 0.23389, 0.23389, 0.24176, 0.24176, 0.24863, 0.24863, 0.26161, 0.26161, 0.27382, 0.27382, 0.2835, 0.2835, 0.25422, 0.25422, 0.24563, 0.24563, 0.24361, 0.24361, 0.23399, 0.23399, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532], 
'prod_price_forecast': [0.25749, 0.25749, 0.25227, 0.25227, 0.24346, 0.24346, 0.24236, 0.24236, 0.23582, 0.23582, 0.23389, 0.23389, 0.24176, 0.24176, 0.24863, 0.24863, 0.26161, 0.26161, 0.27382, 0.27382, 0.2835, 0.2835, 0.25422, 0.25422, 0.24563, 0.24563, 0.24361, 0.24361, 0.23399, 0.23399, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532, 0.26532],
 'pv_power_forecast': [289, 387, 438, 338, 325, 547, 745, 843, 823, 769, 703, 620, 537, 457, 397, 303, 169, 84, 23, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 15, 84, 184, 364], 
'prediction_horizon': 48, 'soc_init': 0.94, 'soc_final': 0.1, 'alpha': 0.25, 'beta': 0.75, 'num_def_loads': 3, 'P_deferrable_nom': [1000, 500, 200], 'treat_def_as_semi_cont': [1, 1, 0], 'set_def_constant': [1, 0, 0], 'def_total_hours': [2, 2, 1], 'def_start_timestep': [22, -2, 26], 'def_end_timestep': [46, 22, 30]}
2024-03-11 09:20:10,306 - web_server - INFO -  >> Setting input data dict
2024-03-11 09:20:10,307 - web_server - INFO - Setting up needed data
2024-03-11 09:20:10,408 - web_server - INFO - Retrieve hass get data method initiated...
2024-03-11 09:20:15,773 - web_server - INFO - Retrieving weather forecast data using method = list
2024-03-11 09:20:15,777 - web_server - INFO - Retrieving data from hass for load forecast using method = mlforecaster

GeoDerp · March 11, 2024, 12:21pm

Greetings Everyone, 0.8.2 introduces some changes:

armhf support (docker file improvements)
param in EMHASS-Add-On to select /data or /share as your data path
there is a new button on the website to switch from basic to advance
bumped some packages
docs has dark theme and better wider screen support

Standalone mode has a current bug (my bad) that will be fixed in the next patch.

heindlalexander · March 11, 2024, 12:25pm

Thank you!
I installed 0.8.2 today. So far everything seems fine. Let’s see what happens tomorrow morning.

By the way: it was mentioned, that it is possible to have a test environment in docker while the addon is still there, correct?
What I want to say: is there a way to test a newer version before upgrading the addon? Is there a description for doing so? Or would I have to use a separate Home Assistant instance to test it?
Moreover: How do you revert to an older version? Is there a way not to restore Home Assistant? Maybe just replacing the sources?

Sorry for these (maybe dumb) questions…

GeoDerp · March 11, 2024, 12:34pm

Na none of this is dumb at all. There are multiple ways of testing emhass. One method is by running EMHASS separately on a different docker environment: GitHub - davidusb-geek/emhass: emhass: Energy Management for Home Assistant, is a Python module designed to optimize your home energy interfacing with Home Assistant. (standalone mode)

Another method is to use something like the HA SSH add-on to git clone and install EMHASS-Add-On repo locally: emhass-add-on/Test.md at abad5c41cf28bcb3ccfb1db4b8c2aa9ab2a7cf27 · davidusb-geek/emhass-add-on · GitHub
This second method you should be able change the version in config.yaml before you install. I believe it would pull the corresponding DockerHub version image.
This method would also act as an seperate add-on on HA. This allows you to run the local version and the original version at the same time. (Just on different ports)

If you like to publish the data from EMHASS without messing with your current HA environment, it may be best to run another complete HA environment just for testing. You can do that inside of an VS Code Dev Container: emhass-add-on/Test.md at abad5c41cf28bcb3ccfb1db4b8c2aa9ab2a7cf27 · davidusb-geek/emhass-add-on · GitHub

I’m sure there are possibility easier ways to bump up or down the EMHASS-Add-on version.

GeoDerp · March 11, 2024, 1:15pm

Some info for the differences between EMHASS and EMHASS-Add-on can be found here: EMHASS & EMHASS-Add-on differences — emhass 0.8.2 documentation

bert1111 · March 11, 2024, 1:38pm

just updated to emhass version 0.8.2 from 0.8.0.
This time it looks like it went better.
Only thing now is that it says that sensor history is empty now.

2024-03-11 14:29:06,140 - web_server - ERROR - The retrieved JSON is empty, A sensor:sensor.power_load_no_var_loads may have 0 days of history or passed sensor may not be correct

Is it again just a case of waiting? Or is there something else wrong?

rickroetenberg · March 12, 2024, 8:43am

Can perhaps someone assist me what I can achieve with the parameters:

weight_battery_discharge: An additional weight applied in cost function to battery usage for discharge.
weight_battery_charge: An additional weight applied in cost function to battery usage for charge.

Does this mean, when I add `weight_battery_charge = 0.04 there should be at least 4 cent between the pricing required in order to start charging the battery?

Currently I’m looking for a solution to only charge/discharge when there is a difference of 4 cent between the pricing because of costs of charging/discharing. I’m using at this moment the efficiency option for this.