EMHASS add-on: An energy management optimization add-on for Home Assistant OS and supervised

110hs · August 2, 2023, 2:18pm

I tried to launch it from the UI (“perfect optimization”). But after two hours that was the log.
But of course no problem I can still try to shell command.

Edit:

Launching this from the HA terminal:

curl -i -H \"Content-Type:application/json\" -X POST -d '{}' http://localhost:5000/action/dayahead-optim

returns:

HTTP/1.0 400 Bad Request
Connection: close
Content-Length: 56
Content-Type: text/plain; charset=utf-8
Date: Wed, 02 Aug 2023 14:29:41 GMT
Server: waitress

Bad Request

Invalid header

(generated by waitress)

davidusb · August 2, 2023, 2:39pm

Those logs doesn’t show that you push any button. No optimization was launched. So that’s weird.
These optimization are really fast because we are solving relatively small LP problems. They should take one or two seconds for each optimized day.

Try this instead:

curl -i -H 'Content-Type:application/json' -X POST -d '{}' http://localhost:5000/action/dayahead-optim

110hs · August 2, 2023, 2:43pm

Now I tried to launch the day ahead optim form the UI and something is being logged.

2023-08-02 16:39:18,461 - web_server - INFO - Setting up needed data
2023-08-02 16:39:19,220 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-08-02 16:39:32,282 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-08-02 16:39:32,352 - web_server - INFO - Retrieve hass get data method initiated...
2023-08-02 16:40:31,486 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2023-08-02 16:40:31,487 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2023-08-02 16:40:31,488 - web_server - INFO - The data path is: /share
2023-08-02 16:40:31,514 - web_server - INFO - Using core emhass version: 0.4.14
waitress   INFO  Serving on http://0.0.0.0:5000

I let you know what happens, then I try with your updated command

110hs · August 2, 2023, 2:49pm

This command was accepted but the message I got from the terminal is

curl: (52) Empty reply from the server

and the log while running this was

2023-08-02 16:45:41,709 - web_server - INFO - Setting up needed data
2023-08-02 16:45:42,034 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-08-02 16:45:49,377 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-08-02 16:45:49,408 - web_server - INFO - Retrieve hass get data method initiated...
2023-08-02 16:46:49,311 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2023-08-02 16:46:49,312 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2023-08-02 16:46:49,312 - web_server - INFO - The data path is: /share
2023-08-02 16:46:49,330 - web_server - INFO - Using core emhass version: 0.4.14
waitress   INFO  Serving on http://0.0.0.0:5000

Edit:
I’m using COIN_CMD as solver

Edit2:
Debug logging doesn’t give more details.
I get the Empty reply in the terminal when the log shows:
2023-08-02 16:58:05,212 - web_server - INFO - Retrieve hass get data method initiated...

110hs · August 4, 2023, 12:06am

I tried to reinstall the component, use the defauls and just pass the PV and load variables, invoke the command playing a bit with https, localhost, 0.0.0.0:5000, homeassistat.local:5000, change the resolver to GLPK but no luck.
The best I can get is using

curl -i -H 'Content-Type:application/json' -X POST -d '{}' http://localhost:5000/action/dayahead-optim

and still the results is always the log I previously posted and an empty reply error in the Terminal.

PS, if useful:
when I use https in the curl command I get nothing in the log and an SSL error in the Terminal

rcruikshank · August 5, 2023, 10:51am

Running this curl command won’t return anything of any significance in your terminal session. It should update the graph in the UI though. What does the graph look like after running the day ahead optimisation?

110hs · August 5, 2023, 12:07pm

Hello Robert,
That’s the problem.
Nothing, no charts.
This is the log after starting the add-on, launching the day-ahead command and then checking the front-end for any charts. Also the log confirms the data container is still empty.

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun emhass (no readiness notification)
s6-rc: info: service legacy-services successfully started
2023-08-05 14:01:54,505 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2023-08-05 14:01:54,511 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2023-08-05 14:01:54,511 - web_server - INFO - The data path is: /share
2023-08-05 14:01:54,530 - web_server - INFO - Using core emhass version: 0.4.14
waitress   INFO  Serving on http://0.0.0.0:5000
2023-08-05 14:01:57,557 - web_server - INFO - Setting up needed data
2023-08-05 14:01:57,979 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-08-05 14:02:08,291 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-08-05 14:02:08,298 - web_server - INFO - Retrieve hass get data method initiated...
2023-08-05 14:03:54,921 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2023-08-05 14:03:54,922 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2023-08-05 14:03:54,922 - web_server - INFO - The data path is: /share
2023-08-05 14:03:54,942 - web_server - INFO - Using core emhass version: 0.4.14
waitress   INFO  Serving on http://0.0.0.0:5000
2023-08-05 14:03:55,599 - web_server - INFO - EMHASS server online, serving index.html...
2023-08-05 14:03:55,637 - web_server - WARNING - The data container dictionary is empty... Please launch an optimization task

rcruikshank · August 5, 2023, 12:36pm

What’s your configuration?

hass_url: empty
long_lived_token: empty
costfun: profit
logging_level: INFO
optimization_time_step: 30
historic_days_to_retrieve: 2
method_ts_round: first
set_total_pv_sell: false
lp_solver: COIN_CMD
lp_solver_path: /usr/bin/cbc
set_nocharge_from_grid: false
set_nodischarge_to_grid: false
set_battery_dynamic: false
battery_dynamic_max: 0.9
battery_dynamic_min: -0.9
load_forecast_method: naive
sensor_power_photovoltaics: sensor.sonnenbatterie_XXXXX_production_w
sensor_power_load_no_var_loads: sensor.house_power_consumption_less_deferrables
number_of_deferrable_loads: 1
list_nominal_power_of_deferrable_loads:
  - nominal_power_of_deferrable_loads: 1300
list_operating_hours_of_each_deferrable_load:
  - operating_hours_of_each_deferrable_load: 2
list_peak_hours_periods_start_hours:
  - peak_hours_periods_start_hours: "10:00"
list_peak_hours_periods_end_hours:
  - peak_hours_periods_end_hours: "16:00"
list_treat_deferrable_load_as_semi_cont:
  - treat_deferrable_load_as_semi_cont: true
load_peak_hours_cost: 0.1907
load_offpeak_hours_cost: 0.1419
photovoltaic_production_sell_price: 0.065
maximum_power_from_grid: 14490
list_pv_module_model:
  - pv_module_model: CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M
list_pv_inverter_model:
  - pv_inverter_model: Fronius_International_GmbH__Fronius_Primo_5_0_1_208_240__240V_
list_surface_tilt:
  - surface_tilt: 21
list_surface_azimuth:
  - surface_azimuth: 350
list_modules_per_string:
  - modules_per_string: 17
list_strings_per_inverter:
  - strings_per_inverter: 1
set_use_battery: true
battery_discharge_power_max: 3300
battery_charge_power_max: 3300
battery_discharge_efficiency: 0.95
battery_charge_efficiency: 0.95
battery_nominal_energy_capacity: 9300
battery_minimum_state_of_charge: 0.1
battery_maximum_state_of_charge: 1
battery_target_state_of_charge: 0.1

110hs · August 5, 2023, 12:59pm

I don’t think I have direct access to the config_emhass.yaml through the add-on (or at least I don’t know how to retrieve it), anyway this is my configuration (as I was experiencing troubles I let it to defaults and just changed the PV and load sensors; I wanted to see it running first and then make the changes little by little).
Thanks

110hs · August 5, 2023, 1:14pm

What I can add, if useful, is that the add-on seems not to crash during the execution (I disabled the watchdog and after the execution of the command it is still running).
But right after I get the Empty reply message in the Terminal, if I try to reload the UI this is unavailable. It works again some moments later (but still the data container is recognized as empty).

rcruikshank · August 5, 2023, 1:28pm

Just click the three dots at the top and select “Edit in YAML”

rcruikshank · August 5, 2023, 1:42pm

you have two sensors configured. sensor.production_pv_w and sensor.consumption_w.
If you enter these two sensors, one at a time, into the “Filter entities” field under STATES in the Developer Tools screen what is displayed?

The sensor I have configured for the first (sensor_power_photovoltaics: sensor.sonnenbatterie_XXXXX_production_w) shows 0 because there is no PV production at present:

Do you get something similar?

And for the second I have sensor_power_load_no_var_loads: sensor.house_power_consumption_less_deferrables which shows his:

rcruikshank · August 5, 2023, 1:54pm

I don’t think EMHASS is crashing, I just think you’re not passing any data to it. I’m not sure how to pass data to it the way you want to so I’m just hunting for misconfigurations. I could be wrong.

110hs · August 5, 2023, 2:03pm

Ah you are right. This was an easy one.

hass_url: empty
long_lived_token: empty
costfun: profit
logging_level: INFO
optimization_time_step: 30
historic_days_to_retrieve: 2
method_ts_round: nearest
set_total_pv_sell: false
lp_solver: COIN_CMD
lp_solver_path: /usr/bin/cbc
set_nocharge_from_grid: false
set_nodischarge_to_grid: false
set_battery_dynamic: false
battery_dynamic_max: 0.9
battery_dynamic_min: -0.9
load_forecast_method: naive
sensor_power_photovoltaics: sensor.production_pv_w
sensor_power_load_no_var_loads: sensor.consumption_w
number_of_deferrable_loads: 2
list_nominal_power_of_deferrable_loads:
  - nominal_power_of_deferrable_loads: 3000
  - nominal_power_of_deferrable_loads: 750
list_operating_hours_of_each_deferrable_load:
  - operating_hours_of_each_deferrable_load: 5
  - operating_hours_of_each_deferrable_load: 8
list_peak_hours_periods_start_hours:
  - peak_hours_periods_start_hours: "02:54"
  - peak_hours_periods_start_hours: "17:24"
list_peak_hours_periods_end_hours:
  - peak_hours_periods_end_hours: "15:24"
  - peak_hours_periods_end_hours: "20:24"
list_treat_deferrable_load_as_semi_cont:
  - treat_deferrable_load_as_semi_cont: true
  - treat_deferrable_load_as_semi_cont: true
load_peak_hours_cost: 0.1907
load_offpeak_hours_cost: 0.1419
photovoltaic_production_sell_price: 0.065
maximum_power_from_grid: 9000
list_pv_module_model:
  - pv_module_model: CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M
list_pv_inverter_model:
  - pv_inverter_model: Fronius_International_GmbH__Fronius_Primo_5_0_1_208_240__240V_
list_surface_tilt:
  - surface_tilt: 30
list_surface_azimuth:
  - surface_azimuth: 205
list_modules_per_string:
  - modules_per_string: 16
list_strings_per_inverter:
  - strings_per_inverter: 1
set_use_battery: false
battery_discharge_power_max: 1000
battery_charge_power_max: 1000
battery_discharge_efficiency: 0.95
battery_charge_efficiency: 0.95
battery_nominal_energy_capacity: 5000
battery_minimum_state_of_charge: 0.3
battery_maximum_state_of_charge: 0.9
battery_target_state_of_charge: 0.6

About the sensors:

My recorder settings are:

  purge_keep_days: 8
  commit_interval: 30

which is above the 2 days of data required for this configuration of the add-on.

Thanks for the time you are spending on my problem!

rcruikshank · August 6, 2023, 7:47am

Nope, no idea. I jumped over all those methods and implemented MPC. Perhaps try feeding some data manually to see if it responds to MPC optim?

110hs · August 6, 2023, 9:23am

Thanks anyway.
Just a question: on which platform are you running HA/the add-on?
I’m on a Pi3b+ but I could try on a Pi4.

rcruikshank · August 6, 2023, 9:37am

I’m on an intel NUC running proxmox. The NUC is running a couple of other servers as well like Teslamate.

110hs · August 6, 2023, 9:50am

In the meantime I tried with a Pi4 and it worked with some “fake” sensors.
I start to think the problem may be related to (my) Pi3b+.
Anybody else using a Pi3b+?

@davidusb any comments on this? Any additional minimum hardware requirements?
Thanks

110hs · August 6, 2023, 10:07am

Hold on!
I had an idea and tried to pass different fake sensors on my Pi3b+.
This time it worked. Maybe my problem is related to the amount of data passed to the solver.
My sensors have a 2 seconds resolutions (I know, it’s high… but I had some troubles with my battery system which is providing all the measurements I need and I was trying to be as close as possible to real time data) and maybe this is too much for the hardware/solver. @rcruikshank may I ask you what your sensors resolution is?
What do you think @davidusb ? What would be your suggestion on data data resolution? Did you perform any tests?

In the meantime I will try to create some lower resolution sensors and see what happens.

rcruikshank · August 6, 2023, 11:20am

Looks like its about 11 seconds.