EMHASS: An Energy Management for Home Assistant

No problem, as soc_final is so far away it has little impact on the initial forecasts, so I set it to zero to see the maximum that EMHASS could return over the full 24 hours.

Warning. My full mpc_optimisation call is quite complex and I don’t recommend people start here with this level of complexity. I actually have five different shell commands that I can set from the straight forward day-ahead to the complex MPC, so if things go wrong I can easily roll back to a working configuration.

  post_mpc_optim: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"load_cost_forecast\":{{(
          ([states('sensor.amber_general_price')|float(0)] +
          state_attr('sensor.amber_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)[:48])
          }}, \"load_power_forecast\":{{
          [states('sensor.power_load_no_var_loads')|int] +(states('input_text.fi_fo_buffer').split(', ')|map('multiply',1000)|map('int')|list)[1:]
          }}, \"prod_price_forecast\":{{(
          ([states('sensor.amber_feed_in_price')|float(0)] +
          (state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)))
          }}, \"pv_power_forecast\":{{[min(15000,(states('sensor.APF_Generation_Entity')|int(0)
                                      /(states('sensor.solaredge_i1_active_power_limit')|int(0)
                                        +states('sensor.solaredge_i2_active_power_limit')|int(0))*200)|int(0))]
            + state_attr('sensor.solcast_forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow())
              | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list 
            + state_attr('sensor.solcast_forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow())
              | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list
          }}, \"prediction_horizon\":{{min(48,
          (state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1)
          }}, \"alpha\":1, \"beta\":0, \"soc_init\":{{(states('sensor.filtered_powerwall_soc')|int(0))/100
          }}, \"soc_final\":0.0, \"def_total_hours\":[{{states('sensor.def_total_hours_pool_filter')
          }},{{states('sensor.def_total_hours_pool_heatpump')
          }},{{states('sensor.def_total_hours_ev')
          }},{{states('sensor.def_total_hours_hvac')
          }},{{states('sensor.def_total_hours_hws')
          }}]}' http://localhost:5000/action/naive-mpc-optim"

Here’s a breakdown of each part of the JSON payload:

  1. load_cost_forecast: This is a list of the current electricity price from the ‘amber_general_price’ sensor, plus a list of future prices extracted from the ‘amber_general_forecast’ sensor. The list is trimmed or extended to have exactly 48 elements.
  2. load_power_forecast: This is a list starting with the current load from the ‘power_load_no_var_loads’ sensor, followed by values extracted from the ‘fi_fo_buffer’ input text, which are multiplied by 1000 and converted to integers.
  3. prod_price_forecast: This is a list of the current feed-in electricity price from the ‘amber_feed_in_price’ sensor, plus a list of future prices extracted from the ‘amber_feed_in_forecast’ sensor.
  4. pv_power_forecast: This starts with the current generation from the ‘APF_Generation_Entity’ sensor, normalized to a maximum of 15000 and a minimum of 0. This is followed by the solar forecasts for today and tomorrow extracted from the ‘solcast_forecast_today’ and ‘solcast_forecast_tomorrow’ sensors, which are multiplied by 2000 and converted to integers.
  5. prediction_horizon: This is the minimum of 48 and one more than the number of forecasts in the ‘amber_feed_in_forecast’ sensor.
  6. alpha: This is a fixed value of 1.
  7. beta: This is a fixed value of 0.
  8. soc_init: This is the current state of charge of the Powerwall, obtained from the ‘filtered_powerwall_soc’ sensor and divided by 100.
  9. soc_final: This is a fixed value of 0.0.
  10. def_total_hours: This is a list of values obtained from the ‘def_total_hours_pool_filter’, ‘def_total_hours_pool_heatpump’, ‘def_total_hours_ev’, ‘def_total_hours_hvac’, and ‘def_total_hours_hws’ sensors.

:+1: Thanks again

1 Like

Minimal configuration to inject Amber Electric prices:

shell_command:
  dayahead_optim: "curl -i -H \"Content-Type: application/json\" -X POST -d '{}' http://localhost:5000/action/dayahead-optim"
  publish_data: "curl -i -H \"Content-Type: application/json\" -X POST -d '{}' http://localhost:5000/action/publish-data "

  post_amber_forecast: "curl -i -H 'Content-Type: application/json' -X POST -d '{\"prod_price_forecast\":{{(
          state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)
          }},\"load_cost_forecast\":{{(
          state_attr('sensor.amber_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)
          }},\"prediction_horizon\":33}' http://localhost:5000/action/dayahead-optim"

Which should return something like:

  post_amber_forecast: "curl -i -H 'Content-Type: application/json' -X POST -d '{\"prod_price_forecast\":[-0.04, 0.01, 0.02, 0.04, 0.03, 0.07, 0.14, 0.16, 0.21, 0.4, 0.34, 0.33, 0.19, 0.19, 0.19, 0.17, 0.16, 0.1, 0.12, 0.14, 0.14, 0.12, 0.1, 0.1, 0.1, 0.07, 0.07, 0.07, 0.07, 0.07, 0.05, 0.06, 0.08, 0.1, 0.13, 0.44, 0.37, 0.29, 0.09, 0.07, 0.07, 0.04, 0.02, 0.02, 0.0, -0.01, -0.01, 0.01],\"load_cost_forecast\":[0.03, 0.08, 0.1, 0.12, 0.11, 0.15, 0.23, 0.25, 0.31, 0.51, 0.45, 0.44, 0.28, 0.29, 0.29, 0.26, 0.25, 0.18, 0.22, 0.24, 0.24, 0.22, 0.19, 0.2, 0.19, 0.16, 0.16, 0.16, 0.16, 0.16, 0.14, 0.15, 0.18, 0.19, 0.22, 0.57, 0.48, 0.39, 0.18, 0.15, 0.15, 0.12, 0.1, 0.1, 0.08, 0.07, 0.07, 0.08],\"prediction_horizon\":33}' http://localhost:5000/action/dayahead-optim"

Try this on your command line to see if it is working correctly.

Bit of a weird one today… I’m assuming this is a bug.
The max charge rate of my battery is set to 3300W. However the model today seems to be suggesting a charge rate of 5.3KW

I’m passing in a SoC of 0.11 (which is what it is), but the graph is refusing to believe me and is showing 0.3 instead:

My guess is that it doesn’t expect the battery to every be below 30% (my settings). However in reality today it discharged beyond 30%, even though I don’t want EMHASS to allow this as part of the prediction

Battery config is:

set_use_battery: true
battery_discharge_power_max: 3000
battery_charge_power_max: 3300
battery_discharge_efficiency: 0.95
battery_charge_efficiency: 0.95
battery_nominal_energy_capacity: 12000
battery_minimum_state_of_charge: 0.1
battery_maximum_state_of_charge: 0.9
battery_target_state_of_charge: 0.35

And my new shell command:

curl -i -H "Content-Type: application/json" -X POST -d ''{"prediction_horizon":{{ state_attr("sensor.populate_solcast_forecast", "steps_left") }},"pv_power_forecast":{{ states("sensor.solcast_24hrs_forecast") }}, "soc_init":{{ (states("sensor.battery_soc")|float(0))/100 }}}'' http://localhost:5000/action/naive-mpc-optim

Check the logs for the optimization status. Maybe you are falling under an infeasible condition due to that low initial SOC? Check that status in the logs. If the optimization is feasible then all your constraints should be respected, including those SOC min/max constraints.

Yep, I think it did say infeasible. What’s the suggested way to handle this?

tried to activate no_discharge_to_grid:

Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 331, in <module>
    params = build_params(params, options, args.addon)
  File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 125, in build_params
    params['optim_conf'][19]['set_nodischarge_to_grid'] = options['set_nodischarge_to_grid']
IndexError: list index out of range

Not able to open web gui after this, not able to revert config to get it working again.

this is after upgrading to 11

Yes there were some issues on the latest release. They are solved now. Search for updates and install the latest version.

I had problem with starting the service since a couple of days. So I started from clean, only changed the long live token with a correct one and tried to run the service with the standard values in the configuration file. But still receive these kinds of errors:

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun emhass (no readiness notification)
s6-rc: info: service legacy-services successfully started
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.9/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 4 (char 3)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 314, in <module>
    config_hass = response.json()
  File "/usr/local/lib/python3.9/dist-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Extra data: line 1 column 4 (char 3)

Any idea what I am doing wrong? Used the standard model and inverter in the standard config file and didn’t changed anything.

The first step is to understand why it is not converging. Sometimes it is just one of the input data that didn’t have correct values. On occasions with the passed input values we are able to conclude that the problem is truly infeasible. For example if you have not much PV production and you are demanding your battery to store energy but in fact there is not enough energy to store. You can maybe do these check ups and try to understand the root cause. Most of the time we are able to explain infeasible conditions.

Are you using the add-on? If this is the case there is no need for the long lived access token. Or are you trying to set the docker standalone mode ?

I’m getting so stuck when trying to find my inverter and panel modules in PVLib. The config docs link to a document from 2019… I’m feeling rather stupid right now.
I’ve got JA Solar JAM66S30 modules that are 500W each.
The inverter is a Sungrow SH10RT with the small SBR096 (9.6kWh) battery.
Should I just grab whatever and then use SolCast or just pass the PV estimates myself to the service? I find it a bit problematic that the service relies on a list of available hardware in such a rapidly changing marked as home PV systems is today. I’d rather just code the required parameters of my installation.

How do I get out of this trouble?

EHMASS does not “relies” on this list of outdated hardware.
That’s why there are many option to provide your PV estimates:

  • You have the PVLib option with the “scrapping” method which is great because the PV modelling is very detailed using PVLib. But I agree the database is a bit outdated. If you can’t find a suitable hardware for you then just simply don’t use this option. Move on to the next option.
  • Pass your estimates as lists of values. You will need to format your list using templates. See the docs.
  • There is also the option to use Solcast and Forecas.Solar directly from inside EMHASS without passing any list of values. These services are fully implemented. The problem with these will be that if you are already using them on your HA instance (for example for the energy dasboard), then you will have problems with the limited number of API calls per day. But if this is not a problem then use these.
1 Like

Piece of art!

What EMHASS was forecasting for my afternoon optimisation:

What my devices reported as actual power consumption:

4 Likes

Very impressive!
These machine learning algorithms are magic. And it is good to note that we are not using neural netwroks or anything more complicated than that, just some classic machine learning algorithms :slight_smile:

@davidusb I have been following this thread for a while. Great stuff and thanks for bringing this work to the community!

My question to you would be if using Emhass is useful in my situation:

  • Only 1000w in solar (3 panels) connected to grid
  • Daily gas prices. Gas heating but potentially could deploy my AC’s (not all rooms) to do a bit more heating ins special cases (eg gas price so high using
  • No battery
  • Fully variable (1 hour slots) energy pricing which also can go negative (so there are sometimes hours where I get paid to consume power, happens once every few weeks)
  • Getting an EV in 2 months (hybrid with 33kwh bat pack)
  • I have almost no deferrable loads. The EV when it arrives & a hottub

Currently I have several automations that for instance shut of solar when prices are negative, my hottub heats only when prices are lowest in the day. I avoid doing dishwasher, dryer and washer when prices are high (as they are not connected) and actually have created a nice work around for that

Do you think I could benefit from Emhass?

Hi yes of course you can use it.
It may be useful for charging your EV on the best optimal conditions.
You can even use it for your manual management. I saw your ESP device, looks neat!
For example for each of your manually controlled devices (dishwasher, dryer, etc.) you could set then in EMHASS and use the optimization results to show on your ESP device screen when is a good moment to turn on those devices.
I think that EMHASS is helpful for anyone that has variable energy pricing.

1 Like

I would stress that optimization can prove extremely beneficial, particularly when resources are limited. It allows you to maximize the output for your household under given constraints.

If you’re actively monitoring and enhancing your usage patterns, you can discover valuable insights which can spur changes in your behavior.

A key understanding I’ve gained from using EMHASS is related to EV charging:

  • At first, I would plug in and begin charging as soon as I arrived home in the evening, due to a flat rate tariff.
  • Upon installing solar panels, I transitioned to charging only with surplus solar energy.
  • Through EMHASS, I’ve realized that at times it’s more advantageous to export power instead of charging (when prices are exceptionally high), and in other situations, it’s better to charge both from the grid and solar simultaneously (when costs are extremely low).

While the advantages of optimization and these insights may seem intuitive, there’s truly nothing more enlightening than observing the tangible impact firsthand. Seeing the effects of your decisions, guided by a tool like EMHASS, enables a deeper understanding of your energy usage. This not only reinforces the value of informed decision-making but also provides a strong motivation to continue pursuing efficient energy practices.

3 Likes

Hi Guys,

Great work david…and also thanks mark for sharing your knowledge/experience.

My first question woould be… do I need a subscription to solcast to run mpi_optimization?
I saw some of the codes that the data injected into EMHASS would be from current time to 24 hours ahead. And I saw mark is running it every minute. With limit number of API calls a day (every 4 hours max) you would be feeding solcast data at some point old data due to 4 hourly solcast sensor update.

I’m still learning the ropes on EMHASS but I think I get the daily optimization done already

No Amber data yet

1 Like

Hi folks,

I am interested to take this step, there’s so much value in this add-on, I’m surprised it’s free :slight_smile: huge thanks!

I would like to start small since I have PV, battery, solar collectors, heat pump and EV. I am in NL with Tibber hourly pricing and currently using their ML capabilities to optimize deferrable loads.

Simply put, can I use EMHASS without deferrable loads and secondly simply use all the forecasted and actuals I already have in HA to push to EMHASS API? Based on the readings, I understand this should be possible but it isn’t exactly clear to me which configs are mandatory and not.

Long write up for a short question but wanted to give you all context. Thanks in advance!

1 Like