EMHASS: An Energy Management for Home Assistant

@marst444 You need to call the publish API repreatedly. This is the standard way to use the dayahead method. Call the dayahead API once per day but call the pub API every 5 minutes. This is what passes the changes in the daily plan to the Home Assistant sensors.

Example automation from EMHASS documentation:

- alias: EMHASS day-ahead optimization
  trigger:
    platform: time
    at: '05:30:00'
  action:
  - service: shell_command.dayahead_optim
- alias: EMHASS publish data
  trigger:
  - minutes: /5
    platform: time_pattern
  action:
  - service: shell_command.publish_data

Note the -minutes: /5 ! This runs the publish API call every 5 minutes but only runs the day-ahead once per day.

But why call the API repeatedly, if there are now changes but every 24 hours? I call/run from restcommand, not shell_command. Remember we discussed before, and that restcommand was easier to debug? But see most/many use the shell_command. Why? Despite that the function seem to work now. (passing parameters to EMHASS, and getting them published to HA, depsite only running once/24 hours :slight_smile: I get a “timeout” when i run my restcommand for the dayahead optimization, but the optimization does start, run and publish. Any ideas?

Because the forecast plan for NOW changes constantly with time but the entity value is only updated on publishing. You don’t call dayahead repeatedly, just the publish so that the data for time NOW is published to the state of the sensor.

The days forecast plan calculated by EMHASS can only be represented in the Attributes of the sensor.

The State of the sensor can only represent one value and that’s the value at time of publishing.

This should be updated constantly so that it’s updated constantly to the current value. If you run it only once then it will be stuck at the value when published and probably go unavailable eventually.

So call publish API every 5 minutes like the documentations states and use the sensor STATE as the kW rate your battery should try to achieve.

You can use the attribute data to graph what is coming. The attribute data will also be updated as you publish and the values will progress through the day. I use apex-charts HACS addon to do the charting. Look for markpurcell post on apexchart in this topic.

You can alternativly use the SOC forecast (sensor.soc_bat_forecast) which states the expected SOC at end of current period and calculate what rate of charge you thing you need to achieve that SOC.

However, using sensor.p_bat_forecast is simpeler to use.

You can use something like this to calculate that rate of charge.

{%- set third_row = state_attr('sensor.soc_batt_forecast', 'battery_scheduled_soc')[0] -%}
{%- set soc_value = third_row['soc_batt_forecast']|float(0) -%}
{%- set raw_power = ((states('sensor.sonnenbatterie_84324_state_charge_user')|float(0) - soc_value) / 100 * 15000 / 0.5 / 0.95)|round(0) -%}
{%- set limited_power = min(max(raw_power, -3300), 3300) -%}
{{ limited_power }}
1 Like

Oh you can use either shell_command or rest_command or node-red flow or whatever. Up to you. But the day-ahead should be called once in the morning and the publish called every 5 minutes.

Here’s a rest_command example.

- alias: EMHASS day-ahead optimization
  trigger:
    platform: time
    at: '05:30:00'
  action:
  - service: rest_command.dayahead_optim
- alias: EMHASS publish data
  trigger:
  - minutes: /5
    platform: time_pattern
  action:
  - service: rest_command.publish_data

I think that’s how the automation is configured. I don’t use automations so dan’t take my word for it. Have a look through this topic for examples.

What kind of resolution do I need on forecast data in CSV files?
The source of my forecasts have different resolutions:

  • Electricity price: 15 minutes
  • Solar forecast: 30 minutes
  • Temperature forecast: 60 minutes

If I provide the forecasts when calling e.g. the naive-mpc-optim command I understand that I need to have all forecasts with the same resolution which means I need to prepare the forecast data.

What are the requirements if I provide the forecasts in CSV files. From the documentation I understand that I provide the forecast using rows with timestamp and value. Does this mean that I can have different resolutions for the different forecasts for price, solar and temp forecast? This would simplify it for me.

I would choose 15 minute, at least overhere there can be quite a different price each different 15 minute slot. It’s not super difficult to change a resolution, you could use something like this as an example:

  - name: solcast_48hrs_forecast_15minutes_interval_org
    state: none
    attributes:
      solar_power: >-
        {%- set power = state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast')| map(attribute='pv_estimate') | list + state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedForecast')| map(attribute='pv_estimate') | list %}
        {%- set values_all = namespace(all=[]) %}
        {% for i in range(power | length) %}
          {%- set v = (power[i] | float |multiply(1000) ) | int(0) %}
          {%- set values_all.all = values_all.all + [ v ] %}
          {%- set values_all.all = values_all.all + [ v ] %}
        {%- endfor %} {{ (values_all.all)[:192] }} 

It takes half hour solcast values and simply doubles them to make quarter values.
Note this does not generate csv, but you can pass this as a list of values,

That is how I do it today.
However, when I provide the forecast to EMHASS I thought I need to have the first item in the list the one that is the current forecast. Or, have I misunderstood this?

I thought that if I provide the forecast in a css file I can just dump the forecast together with the timestamp and forget about removing the already passed forecast values.

I just fount this chapter in the documentation. I must have missed that earlier. Providing time stamped forecast seems so much easier after reading this.
It looks like I can provide timestamped forecast during runtime and can skip csv.

Ha I did not know that either, that’s super easy!

Here is an Apexcart configuration that uses the attributes data from the various sensors updated by EMHASS publishing API.

type: custom:apexcharts-card
experimental:
  color_threshold: true
graph_span: 16h
span:
  start: minute
now:
  show: true
  label: now
  color: red
yaxis:
  - id: first
    decimals: 2
    apex_config:
      forceNiceScale: true
  - id: second
    opposite: true
    min: -4000
    max: 7000
    decimals: 0
    apex_config:
      forceNiceScale: true
header:
  show: true
  title: Energy flow
  show_states: true
  colorize_states: true
series:
  - entity: sensor.amber_30min_forecasts_general_price
    yaxis_id: first
    curve: stepline
    float_precision: 2
    show:
      in_header: after_now
    stroke_width: 1
    name: price kWh
    data_generator: |
      return entity.attributes.Forecasts.map((entry) => {
        return [new Date(entry.start_time), entry.advanced_price_predicted];
      });
  - entity: sensor.amber_30min_forecasts_feed_in_price
    yaxis_id: first
    curve: stepline
    float_precision: 2
    show:
      in_header: after_now
    stroke_width: 1
    name: FiT kWh
    data_generator: |
      return entity.attributes.Forecasts.map((entry) => {
        return [new Date(entry.start_time), -entry.advanced_price_predicted];
      });
  - entity: sensor.p_batt_forecast
    yaxis_id: second
    curve: stepline
    type: area
    show:
      in_header: before_now
    stroke_width: 1
    opacity: 0.2
    color: rgb(0,255,0)
    name: Batt Forecast
    data_generator: |
      return entity.attributes.battery_scheduled_power.map((entry) => {
        return [new Date(entry.date), entry.p_batt_forecast];
      });
  - entity: sensor.p_deferrable0
    yaxis_id: second
    curve: stepline
    stroke_width: 1
    name: Pool
    show:
      in_header: before_now
    data_generator: |
      return entity.attributes.deferrables_schedule.map((entry) => {
        return [new Date(entry.date), entry.p_deferrable0];
      });
  - entity: sensor.p_deferrable1
    yaxis_id: second
    curve: stepline
    stroke_width: 1
    name: Car
    color: white
    show:
      in_header: before_now
    data_generator: |
      return entity.attributes.deferrables_schedule.map((entry) => {
        return [new Date(entry.date), entry.p_deferrable1];
      });
  - entity: sensor.p_pv_forecast
    yaxis_id: second
    curve: stepline
    show:
      in_header: before_now
    stroke_width: 1
    name: PV Forecast
    color: rgb(255,215,0)
    data_generator: |
      return entity.attributes.forecasts.map((entry) => {
        return [new Date(entry.date), entry.p_pv_forecast];
      });
  - entity: sensor.p_load_forecast
    yaxis_id: second
    curve: stepline
    show:
      in_header: before_now
    stroke_width: 1
    name: Load Forecast
    color: rgb(255,0,0)
    data_generator: |
      return entity.attributes.forecasts.map((entry) => {
        return [new Date(entry.date), entry.p_load_forecast];
      });
  - entity: sensor.p_grid_forecast
    yaxis_id: second
    curve: stepline
    opacity: 0.1
    color: rgb(0,255,255)
    type: area
    show:
      in_header: before_now
    stroke_width: 1
    name: Grid Forecast
    data_generator: |
      return entity.attributes.forecasts.map((entry) => {
        return [new Date(entry.date), entry.p_grid_forecast];
      });

And here is it’s output:

You can see that the chart starts at time NOW which is updated every 60 seconds in my case with the publish_data API call. It then plots the attribute entities (e.g p_batt_forecast:) along the X axis using the “Date:” entity in attribute. Its also publishing my tariff forecast sensors which will be different to yours.

You can see that my EV (white line) needs a bit of charging today as we drove down to Gerringong yesterday (4 hour drive there and back from Sydney).

Also it looks like cloud is predicted as the yellow line is sinking after 10:00 (although its sunny right now).

But the price for electricity (orange line) remains low throughout the day at below 5 cents per kWh due to the Australian NEM being powered by 1/3 of all homes having solar panels up and down the 5,000 km coast line from Port Douglas in Queensland to Port Lincoln in South Australia and extending to Tasmania (NEM National Energy Market). Western Australian grid is isolated from the east.

But also note that the feed-in tariff is negative at up to -3 cents per kWh for most of the day as well so I don’t want any of my solar production to leak out onto the grid therefore the pool pump (purple line) will stay on most of the day and the battery charging (green line and area) will soak up the limited solar energy (limited due to predicted cloud).

Useful to get all that from one graph.

A second graph I use is this:


Forecasr battery SOC compared to tariffs.

Clearly shows the profit mode of the battery.

But not much money to be made with maximum feed-in tariff tonight at just over 14 cents.

Its the spikes we look forward to when the temperature hits 40 deg and everybody turns their HVAC on and the feed-in tariff hits $18 for 3 hours (due to dirty coal generators and gas peaker stations cutting in). We make some coin then.

But I suspect $18 feed-in tariffs will become more uncommon as more and more people install home batteries ( over 271,000 home batteries installed in Australia now) and big commercial battery installations continue to grow.

1 Like

Hi and thanx again Robert for all your help! I now have set the publish to /5min as you suggest. I also increased the timeout for the rest_command to 60s. Now no more timeout errors. This could be a potential reason for using shell-commands (according to chat GPT :)). The GPT says the shell-commands are “fire and forget”. Cannot verify that statement, but again, increasing timeout did the trick!

1 Like

Thanx for excellent explanation! Good reason for the recurrent POST, to have an actual p_batt_forecast state to use. Have played with the automation and used the GP, using the ACTUAL SOC of battery compared to the mean of next three predicted p_batt_soc (using 15min timesteps) to smoothen the curve. Will see how it runs, but atleast seem to do more or less whats expected :slight_smile:

To my experience all timesteps have to be the same when passing at runtime (At least for day ahead optimization). I have created custom sensors for electricity prices (different for import and export because of grid taxes and extras on import aso) using the Nordpool 15min timesteps. And divided the Solcast (similar to solarforcast) also in 15min timesteps. Have not passed custom(runtime) temperature data.

How do you handle the “edge case” when EMHASS wants to charge the battery, but there is currently not enough sun and therefor it would discharge the battery. Do you have some logic for example when high price then discharge battery and when low price then import or something else?

@soco There are basically two different configurations for a home PV/Battery setup, 1 Self-Consumption and 2 Time of Use or manual mode.

  • Self-consumption – The typical setup delivered by an installer.
    Depending on DC or AC coupled battery the solar PV energy supplies the house first battery second and grid last. So, the house consumption is satisfied before anything else. If there is spare solar energy that goes into the battery as fast as the battery will take it. Any spare energy after that goes to the grid. The revers of that when the sun goes down. Battery energy first then grid when battery runs out or can’t keep up.

  • Time of Use or manual mode -This is a more complex configuration usually managed by an EMS.
    This is where the value of energy is not flat. It’s influenced by price or tariff, and the battery is manually controlled to take advantage of the changing dollar value of energy.
    So in this case the EMS has to decide when to store energy in the battery when its cheap, free or negative in value and when to supply the house and when to dump to the grid.

  1. Cheap means low cost from the grid compared to expected prices later in the day.
  2. Free means free from the sun or sometimes free from the grid and even netgative supply tariff from the grid so you get paid to use grid energy (this happens a lot in Australia in summer)
  3. Negative means grid feed-in tariff is negative and solar energy sent to the grid will cost you money so store it, use it in the home or curtail it.

So, when you implement an EMS like EMHASS you decouple the battery from the automatic mode of self-consumption and start to manage charging and discharging the battery depending on the forecast expectations of an AI calculation.

The edge case you are talking about doesn’t occur when you decouple the battery from the solar PV array and start to charge and discharge it in accordance with the forecast. You will see the battery charging at any time from the grid or from solar not because there is available solar energy but because the price is right to take advantage of future forecasts. That’s why it is imperative that you have an accurate forecast of what supply and feed-in tariffs will be in the foreseeable “horizon” period.

This may be a simple Time of Use tariff where you always know what the tariffs are. e.g. Supply Sholder, Peak and Off Peak and perhaps a flat feed-in tariff.

In my case both the supply tariff and the feed-in tariff can change every 5 minutes and fluctuate between -10cents and +$20 per kWh.

So if there is a forecast of $18 feed-in tariff for two hours later in the evening, you don’t care how the battery gets charged, by spare solar energy or from the grid. As long as you can sell it for a profit later on.

So the first thing you have to do is figure out how to decouple your battery from available solar energy aka manual mode. Then you have to figure out how to control the battery state from Home Assistant. Typically via REST API or Modbus over TCP/IP or something like that. Highly dependant on what type of battery you have.

Hope that makes sense?

1 Like

That makes sense, thanks a lot.

I was referring to some other cases, I should have added more context.

My case is, when EMHASS thinks there is enough PV to charge the battery. In my case this happens currently in the morning and evening hours (heading towards winter) and during cloudy or foggy days. I already use adjusted pv ( R2: 0,83).

@soco My system will charge the battery in the middle of the night if it calculates that there is an advantage in buying energy at that time for use later in the day.

There might be a supply tariff of 15 cents at 2am and forecast feed-in tariff of 25 cents at 8am so the battery will charge enough to cover the estimated home usage and to discharge to the grid at that time to make some money.

So I suspect that data you are feeding into it is causing the system to think that there is a financial advantage in charging at that time.

You would have to look at the data and the forecast state being calculated.

If someone is interested. Here is my charging logic so far for a solar-edge battery-PV system I use both the forecast from emhass (soc, batt, grid forecast), actual state of charge, PV production. A dynamic charge and discharge limit to follow the EMHASS suggested SOC-target. And a security “no battery discharge when EV is charging”, to not drain the battery. There is also a “keep charging if not reached forecasted SOC and prices have not increased”. I use the 24hour day ahead optimization with solcast and Nordpool 15min prices:

Operation Condition / Trigger Dynamic Sensors Used Action / Notes
----------- ------------------- -------------------- ----------------
Charge – Clipped Solar PV > AC limit AND SOC < max SOC Batt Forecast Smooth, Dynamic Storage Charge Limit Charge using excess PV; set dynamic charge limit; call charge_from_clipped_solar_script
Charge – SOC-driven (Solar > Load) SOC < target AND PV > house load SOC Batt Forecast Smooth, Dynamic Storage Charge Limit Maximize self-consumption; set storage charge limit to 5000W; call maximize_self_consumption_script
Charge – SOC-driven (Solar ≀ Load) SOC < target AND PV ≀ house load OR PV = 0 but low grid price SOC Batt Forecast Smooth, Dynamic Storage Charge Limit, Average Last Charging Price Charge from PV + grid; set dynamic charge limit; call charge_from_solar_power_and_grid_script
Discharge – Max Export SOC > target AND EV off AND grid_fc < 0 AND batt_fc < 0 SOC Batt Forecast Smooth, Dynamic Storage Discharge Limit Discharge to maximize grid export; call discharge_to_maximize_export_script
Discharge – Min Import / Max Self-Consumption SOC > target AND EV off AND grid_fc ≄ 0 AND batt_fc ≀ 0 SOC Batt Forecast Smooth, Dynamic Storage Discharge Limit Discharge to reduce grid import; maximize self-consumption
Maintain Zone – PV > Load SOC within target ± deadband AND PV > house load SOC Batt Forecast Smooth, Dynamic Storage Charge & Discharge Limit Allow max charge & dynamic discharge; maximize self-consumption
Maintain Zone – PV ≀ Load SOC within target ± deadband AND (PV ≀ house load OR batt_fc < 0) SOC Batt Forecast Smooth, Dynamic Storage Charge & Discharge Limit Dynamic charge & discharge; maximize self-consumption
Fallback – Low PV / SOC below target SOC < target AND PV = 0 AND batt_fc = 0 AND grid_fc > 0 Dynamic Storage Charge Limit Charge using grid if needed; call solar_power_only_script
EV Charging Override EV charging = ON Dynamic Storage Discharge Limit Set discharge limit to 100% to prevent battery discharge during EV charging

Hi!

Now have a (for what it seems) working 24h EMHASS restcommand update once/day, with runtime injection of nordpool15min prices and Solcast forecast data.

Run a ML-fit and predict once a day.

Now am trying to set up the MPC. Please confirm my understanding;

First i run my 24hour forcast. I can then decrease the prediction horizon from the 96timesteps until 0 to have runtime parameters (like actual SOC, load and PVproduction) added to the model. using the 24hour day ahead as a “frame”?

So i have created a “shrinking” time window and the arrays for respective loads and forecast to adhere to the shrinking window, having same length as the shrinking prediction horizon.

But EMHASS claims it wants 96timesteps, eventhough i pass another number!

These are my payload in the restCOMMAND call:
What am i missing? Is it a misunderstanding to “shrink” the prediction horizon from 96 to0 as “time goes by” closing in on the next day ahead?

      {# --- Build payload --- #}
      {
        "weather_forecast_cache_only": true,
        "load_power_forecast": {{ load_forecast_window | tojson }},
        "load_cost_forecast": {{ load_cost_window | tojson }},
        "prod_price_forecast": {{ prod_price_window | tojson }},
        "pv_power_forecast": {{ pv_forecast_window | tojson }},
        "prediction_horizon": {{ horizon }},
        "soc_init": {{ soc_init }},
        "soc_final": {{ soc_final }},
        "battery_minimum_state_of_charge": {{ states('input_number.minimum_state_of_charge') | float(20) / 100 }},
        "battery_maximum_state_of_charge": {{ states('input_number.maximum_state_of_charge') | float(90) / 100 }}
      }

I’ll tell you what Amber Electric users do in Australia. We get a rotating prediction from Amber (the electricity supplier) that works like this:

  • Amber publishes a 24-hour Forecast starting at 12:30 each day with 48 by 1/2 hour forecasts until 03:30 the next norming. After 03:30 the forecasts stop.
  • Between 03:30 and 12:30 you have a decreasing number of 1/2 hour forecasts. Decreasing from 48 to 31.
  • At 12:00 you have the smallest number of forecasts 12:00-03:30 which is 31 by 30 minute forecasts.
  • This decreasing horizon is passed to EMHASS in the API call by testing for the shortest number of elements in all of the forecast arrays and pass that to EMHASS in the API post as the “prediction_horizon”.
  {#-- Calculate prediction horizon as the shortest list --#}
  "prediction_horizon": {{
    [prod_price_forecast | length,
     load_cost_forecast | length,
     pv_power_forecast | length,
     load_power_forecast | length] | min -1
  }},

Recently our forecasts have changed to 5 min for then next hour and 30 min as well. So we now use the 5 min actual tariffs (not forecast) as the lead value in the arrays but stick the 30 min for the rest of the forecast.

I only call the MPC API and directly after a successful MPC call, the publish API every 60 seconds. I don’t use the dayahead API at all.

How you would make use of your data I don’t know. I don’t see mentioned of nordpool15min anywhere in this thread. They all seem to talk about 24 x 60 min, not 15 min?