Thanks for sharing. Yes that’s way easier 
This sounds exciting and thank you once again! I am trying to figure exactly the application but my understanding is now I can use the outside temperature and historical load data for my heating system to create an hourly forecast for next 24 hours? Currently the ML forecaster can only use one sensor’s data to train the model. Do I understand this correctly or what do I miss?
Thanks!
1 Like
Helped me as well. Rolled back to ‘restore point’ and then updated to v0.9.0
Works fine now.
There appear to be glitch in the EMHASS: When you play with settings profit/cost/self-consumption then EMHASS can get locked in one of the modes.
Can somebody show how to have a longer history for the load sensor and maybe other sensors, and having all other sensors for only 10 days?
Currently I have:
recorder:
purge_keep_days: 30
I don’t get the point how to have a different setting for a specific sensor but keep the defaults for all other.
so the answer is no? or didn’t I get the point?
what is your configuration?
No, don’t think you can. My config:
recorder:
purge_keep_days: 10
include:
domains:
- alarm_control_panel
- switch
- group
- lock
- light
- update
entity_globs:
- sensor.sonnenbatterie_*
- sensor.p_*
- sensor.pb*
- sensor.pm*
- sensor.pp*
- sensor.cecil_st*
- sensor.solcast*
entities:
- input_text.fifo_buffer
- sensor.optim_status
- binary_sensor.cecil_st_price_spike
- binary_sensor.back_bedroom_side_window_contact
- sensor.amber_count
- sensor.amber_feedin_price_negative
- sensor.bathroom_livingroom_humi_diff
- sensor.batteryinput
- sensor.batteryoutput
- sensor.bedroom_humidity
- sensor.bedroom_temperature
- sensor.car_charge_cost
- sensor.car_charge_costs
- sensor.dining_room_device_temperature
- sensor.dining_room_illuminance_lux
- sensor.dw_power
- sensor.garage_humidity
- sensor.garage_lux
- sensor.garage_temperature
- sensor.garage_power_point_power
- sensor.garage_power_point_energy
- sensor.gridinput
- sensor.gridoutput
- sensor.house_power_consumption_less_deferrables
- sensor.humidity
- sensor.kitchen_motion_device_temperature
- sensor.kitchen_motion_illuminance_lux
- sensor.kitchen_smoke_alarm_device_temperature
- sensor.laundry_powerpoint_power
- sensor.laundry_powerpoint_energy
- sensor.living_room_humidity
- sensor.living_room_temperature
- sensor.livingroom_motion_device_temperature
- sensor.livingroom_motion_illuminance_lux
- sensor.livingroom2_humidity
- sensor.livingroom2_lux
- sensor.livingroom2_temperature
- sensor.medion_plug_power
- sensor.openweathermap_humidity
- sensor.openweathermap_temperature
- sensor.optim_status
- sensor.powerconsumption
- sensor.powerproduction
- sensor.soc_batt_forecast
- sensor.sonnen_operatingmode
- sensor.staircase_device_temperature
- sensor.temperature
- sensor.total_cost_fun_value
- sensor.under_floor_humidity
- sensor.under_floor_temperature
- sensor.underfloor_outdoor_humi_diff
- sensor.unit_load_cost
- sensor.unit_prod_price
- sensor.upstairs_bathroom_humidity
- sensor.upstairs_bathroom_lux
- sensor.upstairs_bathroom_temperature
- sensor.upstairs_living_room_motion_device_temperature
- sensor.upstairs_living_room_motion_illuminance_lux
- sensor.weatherstation_absolute_humidity
- sensor.weatherstation_sea_level_pressure
- sensor.weatherstation_w_station_humi
- sensor.weatherstation_w_station_lux
- sensor.weatherstation_w_station_pres
- sensor.weatherstation_w_station_temp
- sensor.weatherstation2_w_station2_humi
- sensor.weatherstation2_w_station2_lux
- sensor.weatherstation2_w_station2_pres
- sensor.weatherstation2_w_station2_temp
- sensor.ynot_home_charge_energy
- binary_sensor.dining_area
- binary_sensor.dining_room_occupancy
- binary_sensor.doorbell_person
- binary_sensor.doorbell_visitor
- binary_sensor.front_door_open
- binary_sensor.front_gate
- binary_sensor.kitchen_smoke_alarm_smoke
- binary_sensor.side_room
- binary_sensor.staircase_occupancy
- binary_sensor.verandagarage
- binary_sensor.back_bedroom
- binary_sensor.back_bedroom_side_window_contact
- binary_sensor.cecil_st_price_spike
- binary_sensor.doorbell_motion
- binary_sensor.family_room_motion
- binary_sensor.garage_motion
- binary_sensor.hall_motion_occupancy
- binary_sensor.kitchen_motion_occupancy
- binary_sensor.laundry_motion
- binary_sensor.left_dining_room_window
- binary_sensor.left_living_room_window
- binary_sensor.lills_room_motion
- binary_sensor.livingroom_motion_occupancy
- binary_sensor.livingroom2_motion
- binary_sensor.master_bedroom_motion_occupancy
- binary_sensor.right_dining_room_window
- binary_sensor.right_living_room_window
- binary_sensor.side_yard_light_motion
- binary_sensor.stephs_room_motion
- binary_sensor.stephs_window
- binary_sensor.tv_area
- binary_sensor.upstairs_bathroom_motion
- binary_sensor.upstairs_living_room_motion_occupancy
- binary_sensor.verandah_motion
- cover.gate_control
So I only retain 10 days.
1 Like
I tried to use forecast.solar option and I run into this error now:
2024-05-16 11:05:00,214 - web_server - INFO - Passed runtime parameters: {'load_cost_forecast': [14.087, 13.5443, 13.6566, 13.8305, 16.1317, 19.2142, 20.7035, 22.8564, 25.1905, 23.6503, 22.2076, 20.907], 'prod_price_forecast': [-0.257, -0.769, -0.663, -0.499, 1.672, 4.58, 5.985, 8.016, 10.218, 8.765, 7.404, 6.177], 'load_power_forecast': [279, 200, 200, 300, 200, 200, 300, 300, 200, 200, 300, 200, 1400, 300, 400, 400, 300, 400, 200, 300, 400, 300, 600, 400, 500, 500, 400, 500, 500, 200, 500, 600, 300, 200, 200, 500, 500, 400, 300, 300, 200, 200, 200, 300, 300, 300, 200, 400], 'prediction_horizon': 12, 'alpha': 1, 'beta': 0, 'soc_init': 0.44, 'soc_final': 0.12}
2024-05-16 11:05:00,216 - web_server - INFO - >> Setting input data dict
2024-05-16 11:05:00,216 - web_server - INFO - Setting up needed data
2024-05-16 11:05:00,228 - web_server - INFO - Retrieve hass get data method initiated...
2024-05-16 11:05:00,651 - web_server - INFO - Retrieving weather forecast data using method = solar.forecast
2024-05-16 11:05:00,831 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1473, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 882, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 880, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 109, in action_call
input_data_dict = set_input_data_dict(emhass_conf, costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 142, in set_input_data_dict
df_weather = fcst.get_weather_forecast(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 281, in get_weather_forecast
data_dict = {'ts':list(data_raw['result']['watts'].keys()), 'yhat':list(data_raw['result']['watts'].values())}
~~~~~~~~~~~~~~~~~~^^^^^^^^^
TypeError: string indices must be integers, not 'str'
2024-05-16 11:05:00,853 - web_server - INFO - Passed runtime parameters: {}
2024-05-16 11:05:00,853 - web_server - INFO - >> Setting input data dict
2024-05-16 11:05:00,853 - web_server - INFO - Setting up needed data
2024-05-16 11:05:00,854 - web_server - INFO - >> Publishing data...
2024-05-16 11:05:00,855 - web_server - INFO - Publishing data to HASS instance
2024-05-16 11:05:00,866 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 5104.0
2024-05-16 11:05:00,870 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 200
2024-05-16 11:05:00,873 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 2000.0
2024-05-16 11:05:00,877 - web_server - INFO - Successfully posted to sensor.p_deferrable1 = 2500.0
2024-05-16 11:05:00,881 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = -404.0
2024-05-16 11:05:00,885 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 53.55
2024-05-16 11:05:00,888 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 0.0
2024-05-16 11:05:00,891 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = 204.53
2024-05-16 11:05:00,893 - web_server - INFO - Successfully posted to sensor.optim_status = Optimal
2024-05-16 11:05:00,896 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 14.087
2024-05-16 11:05:00,899 - web_server - INFO - Successfully posted to sensor.unit_prod_price = -0.257
I run latest version of EMHASS
Could you please open a github issue for this one. It is possible that the API was updated with a break change for us.
Hi, I raised a bug report for this.
Hiya team,
Firstly a big thanks to everything that’s been written in this thread, I’ve been able to achieve so much. My EMHASS config is now starting to make sense and works pretty well - I’m discovering other ways I automated deferable loads and making sure they use EMHASS only.
My Batteries are DC Coupled with my SolarEdge inverter so I have an odd-ish situation where the max rate I can charge at differs through the day (and from what I can see is impacted by how I configure P_from_grid_max) For example overnight the max that can charge the battery is 5kw as my inverter is only 5kw, whilst through the day i’ve seen the charge rate get up to 9000kw as it’s using both the solar panels (DC side of the inverter, which can get up to 6300kw) plus the grid to charge.
My conundrum is if I set the max charge rate (P_from_grid_max) to 5000, during the day if I use the EMHASS Battery Forecast value in my automations it can restrict my solar charge to battery. Whilst if I set the max charge rate to 9000, during the evening EMHASS thinks it’s going to be able to get more from the grid than the inverter allows
For context I’m running MPC optimise and update every 2 mins, as I’m on Ausgrid / Amber in Australia.
Maybe I’m overthinking things, but I have a couple of ideas on what could try but seeing if anyone else has dealt with this already.
- Set the max charge rate to 5000, but add some logic during daylight hours to modify the battery_forecast value before it’s applied to the inverter to make it much higher - EMHASS
- See if I can include the P_from_grid_max in an API call to EMHASS, and vary it based on time of day - I’m not sure if this is possible through the MPC endpoint or if I’d need to send to another endpoint (and then not sure on whether changes to that setting required EMHASS to restart)
- Set it to a middle figure and let EMHASS work it out through the day when things don’t run to plan.
For context I’m running an MPC optimise and update every 2 mins, as I’m on Ausgrid / Amber in Australia. Any thoughts would be appreciated.
(The final option is to replace my inverter, to say a 10kw one and then know that it can probably charge my batteries from the grid, solar or combined at a higher rate).
Grid buy cost of -3c/ kWh.
EMHASS is doing everything it can to keep up…
| index | P_PV | P_Load | P_deferrable0 | P_deferrable1 | P_deferrable2 | P_deferrable3 | P_deferrable4 | P_deferrable5 | P_grid_pos | P_grid_neg | P_grid | P_batt | SOC_opt | unit_load_cost | unit_prod_price | cost_profit | cost_fun_profit |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2024-05-18 12:30:00+10:00 | 0 | 400 | 0 | 6333 | 11520 | 0 | 600 | 11520 | 40000 | 0 | 40000 | -9627 | 0.715 | -0.03 | -0.09 | 0.6 | 0.6 |
| 2024-05-18 13:00:00+10:00 | 9332 | 600 | 0 | 6333 | 11520 | 0 | 600 | 11520 | 36241 | 0 | 36241 | -15000 | 0.879 | -0.03 | -0.09 | 0.544 | 0.544 |
| 2024-05-18 13:30:00+10:00 | 8684 | 800 | 0 | 6333 | 11520 | 0 | 600 | 11520 | 31391 | 0 | 31391 | -9302 | 0.98 | 0.01 | -0.05 | -0.157 | -0.157 |
| 2024-05-18 14:00:00+10:00 | 7782 | 500 | 0 | 0 | 0 | 0 | 0 | 5479 | 0 | 0 | 0 | -1802 | 1 | 0.02 | -0.05 | 0 | 0 |
| 2024-05-18 14:30:00+10:00 | 6470 | 300 | 0 | 0 | 11520 | 0 | 0 | 1707 | 4839 | 0 | 4839 | 2217 | 0.975 | 0.02 | -0.05 | -0.048 | -0.048 |
| 2024-05-18 15:00:00+10:00 | 5033 | 700 | 0 | 0 | 0 | 0 | 0 | 4333 | 0 | 0 | 0 | 0 | 0.975 | 0.02 | -0.05 | 0 | 0 |
| 2024-05-18 15:30:00+10:00 | 3534 | 600 | 0 | 0 | 0 | 0 | 600 | 0 | 0 | 0 | 0 | -2334 | 1 | 0.03 | -0.04 | 0 | 0 |
| 2024-05-18 16:00:00+10:00 | 1939 | 400 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -1539 | -1539 | 0 | 1 | 0.25 | 0.03 | 0.023 | 0.023 |
| 2024-05-18 16:30:00+10:00 | 546 | 300 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -246 | -246 | 0 | 1 | 0.26 | 0.04 | 0.005 | 0.005 |
| 2024-05-18 17:00:00+10:00 | 0 | 2000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -5500 | -5500 | 7500 | 0.914 | 0.33 | 0.1 | 0.275 | 0.275 |
Similar pattern here, everytime there is a bank holiday weekend the prices drop below -/-150 euro per megawatt. Which, including taxes, results is negative net prices.
I manually curtail my solar panels - so far it happend 3 times this calendar year so no need to automate. However, as wind and solar continues to expand the frequency will increase. Working on a script to shutdown my Enphase, growatt is more complicated, i can limit export but not shutdown without also shutting down the battery.
It would be great to feedback somehow the PV curtailment to EMHASS so that the scenarios can take it into account upfront.
1 Like
Correct if I’m wrong but your problem seems similar to:
https://github.com/davidusb-geek/emhass/issues/220
Which is just the fact that hybrid inverters are not supported yet.
But they will in the near future.
No, we cannot pass multiple features to the ML forecaster (just yet). Hopefully we will add this functionality in the future. Like I was explaining in my post, these are just regression models. The timestamp of the regression prediction are the same as the input data to the regressor. This is different to the ML forecaster where the prediction is provided for future timestamps. Forecasting concerns to out of sample observations, whereas prediction using a regression model concerns to in sample observations. In order to achieve this the ML forecaster uses an auto-regression approach.
A great ressource describing this can be found here: https://www.sktime.net/en/stable/examples/01a_forecasting_sklearn.html
There are many interesting uses cases for this new regression module.
In our case for energy management I think that a good use case for now is what I was saying that @gieljnssns is already doing with this tool: using it to predict the def_total_hours based on other variables.
1 Like
This is exactly my current situation.
1 Like
why does my P Deferrable 2 load move all the time It started to be scheduled last night 00:00 but after a few hours it has moved forward until tomorrow morning. it applies to my electric car so I want it to charge as soon as possible
Every time you rerun the optimisation EMHASS will recalculate the best time to charge your car. If your prices change dynamically this can move the scheduled time around.
You can specify additional constraints for when the load should be scheduled.
I have defined a input helper that can constrain my EV charging to be ready before I depart in the morning, e.g. be completed by 8 am.
That’s really interesting, I’m trying to work on something like this for other loads, however I’m struggling with the 30 mins steps and how they differ from specific times when sending to the MPC endpoint through the day.
If you’re doing MPC optimisations every minute what’s the logic you use to tell emhass the finish time for that deferrable load?