EMHASS: An Energy Management for Home Assistant

I use this template to calculate the hours for my dishwasher (afwasmachien)

    - name: "Afwasmachien uren"
      unique_id: 062928fa-46a0-407c-9947-085b5278ec96
      unit_of_measurement: "uur"
      state: >
        {% if states("input_boolean.afwasmachien_starten") == 'on'%}
          {% if states("switch.afwasmachien") == 'off'%}
            2.5
          {% elif states("input_boolean.emhass_afwasmachien") == 'on'%}
            {% if (2.5 - ((as_timestamp(now()) - states("sensor.afwasmachien_is_gestart_op") | float(0)) / 3600) | round(1)) | float(0) > 0  %}
              {{ 2.5 - ((as_timestamp(now()) - states("sensor.afwasmachien_is_gestart_op") | float(0)) / 3600) | round(1) }}
            {% else %}
              0
            {% endif %}
          {% else %}
            0
          {% endif %}
        {% else %}
          0
        {% endif %}

I use input_boolean.afwasmachien_starten to tell emhass my dishwasher can be started.
switch.afwasmachien is my plug for my dishwasher
input_boolean.emhass_afwasmachien is an input_boolean that is also switched on when emhass turns on my dishwasher
As last I have a triggered template

- trigger:
    - platform: state
      entity_id: input_boolean.emhass_afwasmachien
      from: "off"
      to: "on"
  sensor:
    - name: "Afwasmachien is gestart op…"
      unique_id: c02023fb-05c4-45c3-a82c-9b69b9424ad7
      state: >
        {{ as_timestamp(states.input_boolean.emhass_afwasmachien.last_changed) }}

That sets the start hour of the dishwasher so you can count down from your needed hours.

1 Like

Greetings! Does anyone have any idea why this:

mpc: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"load_cost_forecast\":[0.0973, 0.0972, 0.0967, 0.0956, 0.0948, 0.0957, 0.0952, 0.0944, 0.0938, 0.0941, 0.0619, 0.0605], \"prod_price_forecast\":[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], \"prediction_horizon\":8,\"soc_init\":1,\"soc_final\":1,\"def_total_hours\":[1, 1]}' http://localhost:5000/action/naive-mpc-optim"

results in the deferrable loads being on during the first and second hour? Those are the most expensive ones after all. Also, can I just delete socs as I don’t have a battery?

You might need to specify def_start_timestep and def_end_timestep, see the configuration documentation. Could be that there are defaults for this in your config file or even in the software. If you specify [0,0] and [0,0] I think they aren’t used.

This specific mpc has worked correctly (as far as I can tell) until this moment. So it’s not like they are always on during the first hours. The prediction horizon and total hours change throughout the day.

Thanks for the inspiration @Cavemonkey , @markpurcell and @gieljnssns.

I solved it like this:

Instead of using another helper I leverage the last_triggered attribute of the automation starting the dishwasher. I calculate the last trigger timestamp and compare this with the hours to run. Furthermore I check if the dishwasher should run or if it is already running (using a binary_sensor which I get from my KNX integration).

template:
    - sensor: 
      - name: p_deferrable1_def_total_hours
        unique_id: 7a126cd6-d49c-40a9-867a-5f7c1d08cbdb
        state: |-
          {%- set last_triggered_h = ((now() | as_timestamp - state_attr("automation.p_deferrable1_dishwasher", "last_triggered")|as_timestamp) / 3600) | round(1, 'half', 0) %}
          {%- set hours_to_run = 3 %}

          {%- if 
            is_state('automation.p_deferrable1_dishwasher','on')
            and 
            is_state('input_boolean.p_deferrable1_todo_start','on')
            and
            last_triggered_h > hours_to_run %}
            {#- not started  #}
            {{ hours_to_run }}
          {%- elif 
            is_state('automation.p_deferrable1_dishwasher','on')
            and 
            is_state('binary_sensor.switch_dishwasher_running','on')
            and
            is_state('input_boolean.p_deferrable1_todo_start','off')
            and
            last_triggered_h <= hours_to_run %}
            {#- runs currently #}
            {{ hours_to_run - last_triggered_h }}
          {%- else %}
            {#- not scheduled #}
            0
          {%- endif %}

Furthermore I leverage def_end_timestep to force it in the given timeframe (more or less def_total_hours * 2). I have additionally a input_datetime to force the dishwasher to run before a given time.

template:
    - sensor: 
      - name: p_deferrable1_def_end_timestep
        unique_id: 6bf53911-382b-4485-bdf6-76caff946830
        state: |-
          {%- set hours_to_run = states("sensor.p_deferrable1_def_total_hours") | float(0) %}

          {%- if 
            is_state('automation.p_deferrable1_dishwasher','on')
            and 
            is_state('input_boolean.p_deferrable1_todo_start','on') %}
            {#- not started  #}
              {{ 
                (
                  (
                    ( states("input_datetime.p_deferrable1_end_time") | as_timestamp )
                    - now() | as_timestamp
                  ) / 1800
                ) | int(0)
              }}
          {%- elif 
            is_state('automation.p_deferrable1_dishwasher','on')
            and 
            is_state('binary_sensor.switch_dishwasher_running','on')
            and
            is_state('input_boolean.p_deferrable1_todo_start','off') %}
            {#- runs currently #}
            {{ (hours_to_run * 2) | int(0) }}
          {%- else %}
            {#- not scheduled #}
            0
          {%- endif %}

In my testing I found out that there might be still a bug in the boundray condition which should be fixed in some of the last releases. I filled a issue on GitHub where this problem is explained further.

5 Likes

Suddenly get this error today.
Nothing was changed.

-18 08:54:20,468 - web_server - INFO - Status: Optimal
2024-02-18 08:54:20,469 - web_server - INFO - Total value of the Cost function = -2.23
2024-02-18 08:54:20,495 - web_server - INFO - Solving for day: 17-2-2024
2024-02-18 08:54:21,254 - web_server - INFO - Status: Optimal
2024-02-18 08:54:21,254 - web_server - INFO - Total value of the Cost function = -1.80
2024-02-18 08:54:32,004 - web_server - INFO - Setting up needed data
2024-02-18 08:54:32,015 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2024-02-18 08:54:36,360 - web_server - INFO - Retrieving data from hass for load forecast using method = mlforecaster
2024-02-18 08:54:36,362 - web_server - INFO - Retrieve hass get data method initiated...
2024-02-18 08:54:40,029 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 49, in action_call
    input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 91, in set_input_data_dict
    P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'])
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 624, in get_load_forecast
    data = pd.DataFrame.from_dict(data_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/frame.py", line 1760, in from_dict
    return cls(data, index=index, columns=columns, dtype=dtype)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/frame.py", line 709, in __init__
    mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/construction.py", line 481, in dict_to_mgr
    return arrays_to_mgr(arrays, columns, index, dtype=dtype, typ=typ, consolidate=copy)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/construction.py", line 115, in arrays_to_mgr
    index = _extract_index(arrays)
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pandas/core/internals/construction.py", line 655, in _extract_index
    raise ValueError("All arrays must be of the same length")
ValueError: All arrays must be of the same length

Did you check your load power sensor. There is probably some problem with it. Check that it is complete. This is a HA issue

Cleared itself after I did a new Model Fit.

HI Mark!

I have followed you and you have quite impressive setup. Could you please share your configuration - shell scripts, automations, apex chart etc? Would be great to analyse what you have done and implement thing myself.

Edit: Figured out my first issue, so I’ll just update this with the next one.

So I’m playing around with EMHASS to see if I can make ti work for me. Now I’ve run into an issue: Solar production forecast from solcast is for 30 minute periods, while electricity price from entso-e is 60 minute periods. Whether i set optimization_time_step to 30 or 60 one of them causes trouble…

Is there a way to fix this? I was thinking a template that repeats all the numbers in the price list twice to make it half hours, but i have no idea how to make one or if its even possible…

The solcast sensors have several attributes. If you use “DetailedHourly” then you should be fine:

pv_power_forecast\":{{([states('sensor.solcast_pv_forecast_power_now')|int(0)] + state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedHourly')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list + state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedHourly')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list)| tojson}}

Note, i took part of my code. Please double check syntax in developer tools.

1 Like

Thanks, but i set up solcast in the addon settings so I’d rather not have to mess with setting it up another way. I’d also rather have the half hour optimization if given the option. But thanks anyways, I’ll have a look at it if no-one comes up with other options.

You could get the 60 minute solcast list and just double each value so that the list becomes twice as long. So e.g. [1200, 1300] → [1200, 1200, 1300, 1300]. No need to halve the values, since it will be watts/half hour I think.

That’s what i was thinking for the entso-e list (to make it half hour like solcast), but i have no clue ho i would do that…

This shell command used to work, but now always returns an error.

Also I have never been able to push the ‘ML Forecast Model Fit’ button in the GUI

  forecast_model_fit: 'curl -i -H ''Content-Type:application/json'' -X POST -d ''{"var_model": "sensor.power_consumption", "sklearn_model": "KNeighborsRegressor", "num_lags": 48, "split_date_delta": "48h", "model_type": "load_forecast", "days_to_retrieve": 28}'' http://localhost:5000/action/forecast-model-fit'
2024-02-23 14:01:41,517 - web_server - ERROR - Exception on /action/forecast-model-fit [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 49, in action_call
    input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 146, in set_input_data_dict
    rh.get_data(days_list, var_list)
  File "/usr/local/lib/python3.11/dist-packages/emhass/retrieve_hass.py", line 150, in get_data
    self.df_final = pd.concat([self.df_final, df_day], axis=0)
                                              ^^^^^^
UnboundLocalError: cannot access local variable 'df_day' where it is not associated with a value

@ThorAlex
I am working on a solution for this right now. I will come back to you in the next few days with a sensor that doubles up entsoe prices to a format that can be used in a 30 minutes timestep EMHASS. This is now in the final testing phase.

1 Like

That’s fantastic!

I made a sensor that reads all prices from the Entsoe integration, doubles them to 2 per hour and extends the list to 48 elements by adding the “average_electricity_price_today” as many times as necessary. This way the sensor attribute list always has a length of 48, making it possible to use a 24h rolling MPC-optim with a stepsize of 30 minutes together with a 30 minutes Solcast forecast.
This means that during the day EMHASS will start using the “average price today” for "tomorrow "instead of the actual “hour price” until around 1400LT, when the new dayahead prices (for tomorrow) are being published. If you don’t like this behavior you can leave out this part of the code. Only then the length of the list will go down from 48 to 20 during the day until the new dayahead prices update.

The original Entsoe “average electricity price” sensor has 3 different lists in the attributes:
Prices today / Prices tomorrow / Prices;
I choose to use the last one, Prices, because I’ve experienced strange behavior with the content of the other two while the Prices list has always been reliable.

  • name: entsoe_prices_forecast_30
    unique_id: entsoe_prices_forecast_30
    state: >
    {{ ‘30 minute entsoe forecast prices’}}
    attributes:
    prices: >
    {% set today = now().strftime(‘%Y-%m-%d’) %}
    {% set tomorrow = (now().timestamp()+86400) | timestamp_custom(‘%Y-%m-%d’) %}
    {% set x = (state_attr(‘sensor.entsoe_average_electricity_price_today’, ‘prices’) | selectattr(‘time’, ‘match’, today ) | map(attribute=‘price’) | list)[now().hour:]
    + state_attr(‘sensor.entsoe_average_electricity_price_today’, ‘prices’) | selectattr(‘time’, ‘match’, tomorrow ) | map(attribute=‘price’) | list %}
    {% set ns = namespace(z=) %}
    {% for i in x %}
    {% if loop.index == 1 and now().minute >29 %}
    {% set ns.z = ns.z + [i] %}
    {% else %}
    {% set ns.z = ns.z + [i] + [i] %}
    {% endif %}
    {% endfor %}
    {% set l = ns.z|length %}
    {% set y = states(‘sensor.entsoe_average_electricity_price_today’)|float %}
    {% if l < 48 %}
    {% for i in range(0,48-l) %}
    {% set ns.z = ns.z + [y] %}
    {% endfor %}
    {% endif %}
    {% set ns.z = ns.z[:48] %}
    {{ ns.z }}

Just make sure you replace the name of the sensor (‘sensor.entsoe_average_electricity_price_today’) with your own Entsoe sensor name.

You can use the sensor in your EMHASS mpc-optim configuration like this:

"load_cost_forecast": {{ state_attr(‘sensor.entsoe_prices_forecast_30’, ‘prices’) |tojson }},
"prod_price_forecast": {{ state_attr(‘sensor.entsoe_prices_forecast_30’, ‘prices’) |tojson }},
"prediction_horizon": {{ min(48, state_attr(‘sensor.entsoe_prices_forecast_30’, ‘prices’)|length |int) |tojson }},

1 Like

Just updated EMHASS addon from 0.6.6 to 0.8.0.
Only had to do a new forecast-model-fit, to make it running again.
No other issues…

1 Like

Thanks for reporting this :+1: