EMHASS: An Energy Management for Home Assistant

[markpurcell]

@RT1080

Question, why do you add 1 to the length? Should the prediction horizon not be equal to the number of elements in the array? In my case I have hourly prices which are published at 14:00 until 24:00 the next day, its now 10 am, so I have 14 prices left, if I add 1 it would assume a prediction horizon of 15 right?

I have the current cost, plus between 33- 48 x30 minute forecasts hence the +1.

I also lock my timestamp rounding to first, so at 10:17 I am injecting current/ now values for PV, load, price and cost for 10:00-10:30 + however many forecasts for 10:30 onward. The advantage of first approach is the MPC optimisation can then calculate exactly to the Watt what is allocated to deferrable loads/ battery. If I turn on the oven that will be reflected in the now load and my EV charging, for example, will reduce by that exact amount.

I understand nordpool publishes the forecast prices once per day. Do your actual prices then change during the day?

My forecasts and actuals prices/ costs are updated every 5 minutes which is why I adopt this approach.

[RT1080 (Ronald)]

Ah ok, in the Netherlands we have day ahead prices only for consumers. So our prices are fixed for the following day at 2 pm and do not change. Makes it a bit easier I guess.

[markpurcell]

@RT1080

Ah ok, in the Netherlands we have day ahead prices only for consumers. So our prices are fixed for the following day at 2 pm and do not change. Makes it a bit easier I guess

I’m on a wholesale tariff with the associated highs and lows. Default cost is 31¢/ kWh.

But in the same day we get this crazyness! I was buying from the grid at midday for 3¢/ kWh and selling tonight for $4/ kWh! Needless to say EMHASS just loves this dynamic environment.

[RT1080 (Ronald)]

You can almost make a living out of it with these delta’s. Most we see is 40-50 cents difference across the day but typically only 7-9cents

[rickroetenberg]

Hi everyone,

Can someone explain how I can solve the infeasible message?

2023-08-02 10:00:00,503 - web_server - INFO - Publishing data to HASS instance
2023-08-02 10:00:00,689 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 1514
2023-08-02 10:00:00,796 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 425.42
2023-08-02 10:00:00,808 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2023-08-02 10:00:00,820 - web_server - INFO - Successfully posted to sensor.p_deferrable1 = 0.0
2023-08-02 10:00:00,888 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 0.0
2023-08-02 10:00:00,926 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 30.0
2023-08-02 10:00:00,958 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = -1088.58
2023-08-02 10:00:00,979 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = 4.71
2023-08-02 10:00:00,990 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 0.269
2023-08-02 10:00:01,022 - web_server - INFO - Successfully posted to sensor.unit_prod_price = 0.269
2023-08-02 10:00:01,210 - web_server - INFO - Retrieving weather forecast data using method = list
2023-08-02 10:00:01,211 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-08-02 10:00:01,211 - web_server - INFO - Retrieve hass get data method initiated...
2023-08-02 10:00:04,161 - web_server - INFO -  >> Performing naive MPC optimization...
2023-08-02 10:00:04,161 - web_server - INFO - Performing naive MPC optimization
2023-08-02 10:00:04,166 - web_server - INFO - Perform an iteration of a naive MPC controller
2023-08-02 10:00:04,197 - web_server - INFO - Status: Infeasible
2023-08-02 10:00:04,198 - web_server - INFO - Total value of the Cost function = 5.3

my mpc:

naive_mpc_optim:
  url: http://localhost:5000/action/naive-mpc-optim
  method: POST
  content_type: 'application/json'
  payload: >-
    {
      "prod_price_forecast": {{
      ((state_attr('sensor.nordpool_kwh_nl_eur_3_09_0', 'raw_today') | map(attribute='value') | list  + state_attr('sensor.nordpool_kwh_nl_eur_3_09_0', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24]
        | tojson 
      }},
      "load_cost_forecast": {{
      ((state_attr('sensor.nordpool_kwh_nl_eur_3_09_0', 'raw_today') | map(attribute='value') | list + state_attr('sensor.nordpool_kwh_nl_eur_3_09_0', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24]
        | tojson 
      }},
      "pv_power_forecast": {{([states('sensor.solaredge_ac_power')|int(0)] +
        state_attr('sensor.forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list +
        state_attr('sensor.forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list
        )| tojson
        }},
      "prediction_horizon": {{
          min(12, (((state_attr('sensor.nordpool_kwh_nl_eur_3_09_0', 'raw_today')|map(attribute='value')|list + state_attr('sensor.nordpool_kwh_nl_eur_3_09_0', 'raw_tomorrow') | map(attribute='value')| list))|length)+1)
        }},
      "alpha": 1,
      "beta": 0,
      "num_def_loads": 2,
      "def_total_hours": [{{states('input_number.def_total_hours_ev')}},
        {{states('input_number.def_total_hours_ww')}}],
      "treat_def_as_semi_cont": [1,1],
      "set_def_constant": [0,0],
      "soc_init": {{ (states('sensor.sofarcontroller_battery_soc')|int(0))/100 }},
      "soc_final": 0.05
    }```

[davidusb]

@rickroetenberg

prediction_horizon

Your template leads to what prediction_horizon?
First check that all the passed data on those list templates are correct (no missing data, stranted string, etc).
If all your passed data are correct then this infeasible status can be caused by you asking for something on your system that is just impossible. For example (this may not be your case) if you ask to completely discharge your 5 kWh battery in just one hour but your maximum battery discharge power is 1 kW then the optimization will throw a infeasible status. Check your battery limits settings, the prediction_horizon, the result of that soc_init template, etc. And why are you setting soc_final = 0.05? You are asking the system to completely discharge your battery on one single MPC iteration, that can be a source of problems.

[rickroetenberg]

Found the issue, Thanks! Turns out I configured “The minimum allowed battery state of charge” higher than the current SOC.

[rickroetenberg]

What is a good prediction_horizon value? I have 2 deferrable value at the moment. 1 EV and 1 Boiler. The boiler needs to run once a day of course, but the EV sometimes 8 hours at, but the prediction_horizon might be less then normal because I need to car the next day.

Selecting a prediction_horizon of 24 hours (because sample is 60 min due Nordpool integration) the car will charge sometimes the next day

[jmcollin (Jean-Marc COLLIN)]

For those who are looking for a very simple integration which only starts / stop differable loads depending on the solar production, there is a new integration: GitHub - jmcollin78/solar_optimizer: The Solar Optimizer integration for Home Assistant starts and stops your equipments depending on the Solar net production

As a very interesting feature, it is able to control the power of charging a Tesla Vehicle depending on the solar production.

This integration is quiet root and new but it works like a charm for my needs.

[markpurcell]

Breaking changes - SolCast 3.1.1

Just a heads up with some breaking changes if you upgrade the SolCast custom component.

The detailed 30 minute forecasts have changed from the detailedForecast attribute to forecast attribute.

I am now using the following for my pv_power_forecast in EMHASS:

        "pv_power_forecast": {{
          ([states('sensor.APF_Generation_Entity')|int(0)] +
          state_attr('sensor.solcast_forecast_today', 'forecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list +
          state_attr('sensor.solcast_forecast_tomorrow', 'forecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list
          )| tojson

Updated:

solcast 3.1.4 has reverted back to detailedForecast:

        "pv_power_forecast": {{
          ([states('sensor.APF_Generation_Entity')|int(0)] +
          state_attr('sensor.solcast_forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list +
          state_attr('sensor.solcast_forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list
          )| tojson
        }},

[RT1080 (Ronald)]

@davidusb would it be possible to add an additional constraint in relation to the max total return to grid? There is a “max power from grid” but that doesn’t do the trick. Perhaps there is a setting to do this already but I haven’t found it so far.

Background, one of my inverters is a hybrid one. The max power is 8kva, this is both PV and battery discharge combined. Meaning that, if for example, PV generates 2kw the discharge is capped at 6kw. During day time this means my discharge is limited by the PV generated by the hybrid inverter.

Capping the return to grid at an absolute number is not perfect as I have a second array of panels on a separate inverter (and it would not take into account any load usage) but it at least moves in the right direction. Ideally the PV production sensor/forecast tied to the hybrid inverter is linked to the battery but that’s probably a whole lot more complicated

[gieljnssns (Giel)]

EMHASS: An Energy Management for Home Assistant

I have a question about the sensor_power_load_no_var_loads. I use this template for that sensor - name: Huidig verbruik zonder wp unique_id: 664ab9ff-3875-4a45-b53a-685b7a8f7f96 unit_of_measurement: "W" device_class: power availability: >- {{ has_value("sensor.huidig_verbruik") and has_value("sensor.wasmachien_vermogen") and has_value("sensor.droogkast_vermogen") and has_value("sensor.warmtepompboiler_huidig_verbruik") and …

Does someone can give me an answer on this question?

[markpurcell]

@gieljnssns

Sometimes my heatpump for hot water sensor.warmtepompboiler_huidig_verbruik goes on because the water in the boiler is to cold (before emhass schedules it). Do I have to leave it out the sensor_power_load_no_var_loads on that moment?
The same for other deferrable loads when they are turned on without emhass?

It is a good question and there are multiple answers, none of which will be exact.

For dayahead optimisation you probably don’t need to leave it out and the optimisation will still be a good model.

For high frequency MPC, I am trying to get the model as close to the required W as possible. I was finding that whilst the p_nominal for devices is fixed my hot water heatpump varies over the cycle between 400-650W, so I set p_nominal to the highest value. Then I had the case of my EV charging or HVAC being out of EMHASS cycle.

So I have been experimenting with a different approach using the scheduled deferrable load forecast to setup no_var_loads:

    - name: power_load_no_var_loads
      unit_of_measurement: W
      device_class: power
      state_class: measurement 
      unique_id: 9e2d02cb-2ac4-4eb2-9f57-c4c158c145e7
      state: >-
        {{states('sensor.home_energy_gateway_load_power')|int(0)
        - states('sensor.p_deferrable0')|int(0) 
        - states('sensor.p_deferrable1')|int(0)
        - states('sensor.p_deferrable2')|int(0) 
        - states('sensor.p_deferrable3')|int(0)
        - states('sensor.p_deferrable4')|int(0)
        - states('sensor.p_deferrable5')|int(0)
        }}

Which seems to do a better job of allocating power by taking into account my overs and unders for each device.

Your mileage may vary so consider this an experiment and try with other methods.

[gieljnssns (Giel)]

I don’t use the dayahead optimisation, I use MPC optim every 5min.
When Emhass turns on a device an input_boolean is also turned on, so I’m going to use that in the template.

    - name: Huidig verbruik zonder wp
      unique_id: 664ab9ff-3875-4a45-b53a-685b7a8f7f96
      unit_of_measurement: "W"
      device_class: power
      availability: >-
        {{ has_value("sensor.huidig_verbruik") and 
          has_value("sensor.wasmachien_vermogen") and
          has_value("sensor.droogkast_vermogen") and
          has_value("sensor.warmtepompboiler_huidig_verbruik") and
          has_value("sensor.warmtepomp_huidig_verbruik") and
          has_value("sensor.afwasmachien_vermogen") }}
      state: >
        {% set wasmachien = states("input_boolean.emhass_wasmachien") == "on" -%}
        {%- set droogkast = states("input_boolean.emhass_droogkast") == "on" -%}
        {%- set afwasmachien = states("input_boolean.emhass_afwasmachien") == "on" -%}
        {%- set warmtepompboiler = states("input_boolean.emhass_warmtepompboiler") == "on" -%}
        {%- set warmtepomp = states("input_boolean.emhass_warmtepomp") == "on" -%}
        {%- set deferrable = namespace(all=[]) -%}
        {%- if wasmachien -%}
          {%- set deferrable.all = deferrable.all + [states("sensor.wasmachien_vermogen") | float(0)] -%}
        {% endif %}
        {%- if droogkast %}
          {%- set deferrable.all = deferrable.all + [states("sensor.droogkast_vermogen") | float(0)] -%}
        {% endif %}
        {%- if afwasmachien %}
          {%- set deferrable.all = deferrable.all + [states("sensor.afwasmachien_vermogen") | float(0)] -%}
        {% endif %}
        {%- if warmtepompboiler %}
          {%- set deferrable.all = deferrable.all + [states("sensor.warmtepompboiler_huidig_verbruik") | float(0)] -%}
        {% endif %}
        {%- if warmtepomp %}
          {%- set deferrable.all = deferrable.all + [states("sensor.warmtepomp_huidig_verbruik") | float(0)] -%}
        {% endif %}
        {%- if states("sensor.huidig_verbruik") | float(0) - ((deferrable.all) | sum ) | round(1) > 0 -%}
            {{ (states("sensor.huidig_verbruik") | float(0) - ((deferrable.all) | sum )) | round(1) }}
        {% else %}
            {{ 0 | float(0) }}
        {% endif %}

[bjornstark]

Hi!

Thanks @davidusb for an interesting and useful project, and to other contributors in the thread!

I’ve just started using Home Assistant and I’d like to use EMHASS to control a heat pump (IVT Premiumline EQ C8) for warming the house and warm water. I’ll be using the Husdata H66 card to control the heat pump. Also have a PV setup and a Solis inverter integrated in the HA instance.

I’d like to estimate the heat need for the house using the temperature forecast and the home set temp and historic data read from heat pump on energy used for heat production and use this for the needed power and the time needed for the deferred load using MPC optimizations.

I’m struggling a little using this add-on and I’d appreciate some advice. I’ve been able to do day-ahead optimization, but I’m wondering:

- Is it true that EMHASS does not generate plots and tables? (Seems that people get them from EMHASS but I cannot find where I could get them)

• Would it be OK to have the var_load = 0 when I export power, ignoring the domestic power use?
  • I currently resolve the domestic power use needed in the house when I export to the grid using
    • [PV (inverter) output] - [grid export] + [power consumption from grid]
      Would ideally > 0 but due to poor measurement data of the inverter, I often get negative values.
• Do I need to save the published data in order to plot them using “History”? I can plot them using ApexCharts but I’d also like to log the parameters and to plot them in “History”. Saved data from EMHASS seems to be constant until a run a new optimization and publish it to HA.
• How would I automate the triggering of the curl-command? I tried copying EMHASS: An Energy Management for Home Assistant - #292 by per.takman into a script but it seems I get the error “Message malformed: extra keys not allowed @ data[‘trigger’]”, I suppose it has to do with the time definition.

Thanks a lot in advance!

[rcruikshank (Robert Cruikshank)]

@bjornstark This is how I use EMHASS.

EMHASS: An Energy Management for Home Assistant

EMHASS config for sonnen users

[bjornstark]

Thanks @rcruikshank - what a good documentation! Will have to be digested for a bit. But I’m not sure I’d also like to try Node Red, will try to keep it simple as simple as I can at the moment and then try to extend…

[rcruikshank (Robert Cruikshank)]

@bjornstark I find Node-Red greatly simplifies automation scripts and actually adds greater power to automations. For instance the documented way to execure curl commands with EMHASS is to call shell commands from an automation script which makes returned error handling impossible. With Node-Red you simply use an http request node and error handling is at your fingertips. Give it a go. Bit of a learning curve but worth it.

In the link where that document is stored I’ve included the export of my Node-Red “flow” for EMHASS (EMHASS.json). It won’t work for other environment as is. Its meant for a sonnen battery so has all the API calls that work with sonnen. But give you a starting point at least up to where I’ve manage to get to with EMHASS (not that far but better than cold start).

[andress]

Hi,

@davidusb, a big thank you for the excellent project !

Can anyone provide insight into whether and how to configure a situation in EMHASS where the maximum consumable energy from the electrical grid is different from the maximum allowable amount of energy to be export to the grid?

For example, if I have an inverter with a capacity of 10kW and the battery’s maximum charging power is also 10kW, then I can consume a maximum of 10kW from the grid, but I am only allowed to sell back a maximum of 1kW to the grid.
If I set P_grid_max=1000, the algorithm might not be the most optimal. If I set P_grid_max to 10000 and then, at a moment when the algorithm suggests exporting 10000, if I restrict the export amount, the algorithm will likely become confused as well.
Is there a better solution?

[hendrer (Hendre)]

Hi folks,

Wondering if someone can help with mlforecast. It’s automatically being optimized with 6 lags and the predict is only returning 6 hours in the future. This is limiting my MPC predictions horizon only to 6 hours.

Is anyone else experiencing this lately? If I run the fit and tune multiple times, at some point it will settle on 72 lags as optimal. I’m using 95 days worth of historical data to generate the model.

Any tips appreciated! I saw someone had a similar issue, logged a bug on GitHub and subsequently closed it.

Cheers,
H