EMHASS: An Energy Management for Home Assistant

nixtagangsta · March 13, 2023, 10:21am

and this with 30min:

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun emhass (no readiness notification)
s6-rc: info: service legacy-services successfully started
2023-03-13 11:19:19,639 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2023-03-13 11:19:19,640 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2023-03-13 11:19:19,641 - web_server - INFO - The data path is: /share
2023-03-13 11:19:19,650 - web_server - INFO - Using core emhass version: 0.4.5
waitress INFO Serving on http://0.0.0.0:5000
2023-03-13 11:20:00,486 - web_server - INFO - Setting up needed data
2023-03-13 11:20:00,646 - web_server - INFO - >> Publishing data…
2023-03-13 11:20:00,647 - web_server - INFO - Publishing data to HASS instance
2023-03-13 11:20:00,742 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 5637.14
2023-03-13 11:20:00,802 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 15568.98
2023-03-13 11:20:00,873 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2023-03-13 11:20:00,944 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 0.0
2023-03-13 11:20:01,034 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 60.0
2023-03-13 11:20:01,121 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 9000.0
2023-03-13 11:20:01,212 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = -170.7

but error when running day ahead optimization

2023-03-13 11:21:08,361 - web_server - INFO - Setting up needed data
2023-03-13 11:21:08,386 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-03-13 11:21:13,670 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-03-13 11:21:13,672 - web_server - INFO - Retrieve hass get data method initiated…
2023-03-13 11:21:14,264 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
File “/usr/local/lib/python3.9/dist-packages/flask/app.py”, line 2528, in wsgi_app
response = self.full_dispatch_request()
File “/usr/local/lib/python3.9/dist-packages/flask/app.py”, line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/usr/local/lib/python3.9/dist-packages/flask/app.py”, line 1823, in full_dispatch_request
rv = self.dispatch_request()
File “/usr/local/lib/python3.9/dist-packages/flask/app.py”, line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File “/usr/local/lib/python3.9/dist-packages/emhass/web_server.py”, line 170, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
File “/usr/local/lib/python3.9/dist-packages/emhass/command_line.py”, line 91, in set_input_data_dict
P_load_forecast = fcst.get_load_forecast(method=optim_conf[‘load_forecast_method’])
File “/usr/local/lib/python3.9/dist-packages/emhass/forecast.py”, line 585, in get_load_forecast
rh.get_data(days_list, var_list)
File “/usr/local/lib/python3.9/dist-packages/emhass/retrieve_hass.py”, line 140, in get_data
df_tp = df_raw.copy()[[‘state’]].replace(
File “/usr/local/lib/python3.9/dist-packages/pandas/core/generic.py”, line 5920, in astype
new_data = self._mgr.astype(dtype=dtype, copy=copy, errors=errors)
File “/usr/local/lib/python3.9/dist-packages/pandas/core/internals/managers.py”, line 419, in astype
return self.apply(“astype”, dtype=dtype, copy=copy, errors=errors)
File “/usr/local/lib/python3.9/dist-packages/pandas/core/internals/managers.py”, line 304, in apply
applied = getattr(b, f)(**kwargs)
File “/usr/local/lib/python3.9/dist-packages/pandas/core/internals/blocks.py”, line 580, in astype
new_values = astype_array_safe(values, dtype, copy=copy, errors=errors)
File “/usr/local/lib/python3.9/dist-packages/pandas/core/dtypes/cast.py”, line 1292, in astype_array_safe
new_values = astype_array(values, dtype, copy=copy)
File “/usr/local/lib/python3.9/dist-packages/pandas/core/dtypes/cast.py”, line 1237, in astype_array
values = astype_nansafe(values, dtype, copy=copy)
File “/usr/local/lib/python3.9/dist-packages/pandas/core/dtypes/cast.py”, line 1098, in astype_nansafe
result = astype_nansafe(flat, dtype, copy=copy, skipna=skipna)
File “/usr/local/lib/python3.9/dist-packages/pandas/core/dtypes/cast.py”, line 1181, in astype_nansafe
return arr.astype(dtype, copy=True)
ValueError: could not convert string to float: “16513.0 - states(‘sensor.battery_charge_discharge_power’)|float(0)}}”

rickroetenberg · March 13, 2023, 8:10pm

Anyone has seen this issue before?

requests.exceptions.SSLError: HTTPSConnectionPool(host=‘URL’, port=xxxx): Max retries exceeded with url: /api/states/sensor.p_pv_forecast (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)’)

nixtagangsta · March 15, 2023, 4:11pm

ok

so no errors in emhass logfile, finally
but i won’t get any nordpool prices in my list?

running this:
shell_command: trigger_nordpool_forecast: “curl -i -H “Content-Type: application/json” -X POST -d ‘{“load_cost_forecast”:{{((state_attr(‘sensor.nordpool’, ‘raw_today’) | map(attribute=‘value’) | list + state_attr(‘sensor.nordpool’, ‘raw_tomorrow’) | map(attribute=‘value’) | list))[now().hour:][:24] }},“prod_price_forecast”:{{((state_attr(‘sensor.nordpool’, ‘raw_today’) | map(attribute=‘value’) | list + state_attr(‘sensor.nordpool’, ‘raw_tomorrow’) | map(attribute=‘value’) | list))[now().hour:][:24]}}}’ http://localhost:5000/action/dayahead-optim”

When i run it under development tool /service i see prices, but the wont show in tabel below

index	P_PV	P_Load	P_deferrable0	P_deferrable1	P_grid_pos	P_grid_neg	P_grid	P_batt	SOC_opt	unit_load_cost	unit_prod_price	cost_profit	cost_fun_profit
2023-03-15 17:00:00+01:00	1276.679088	15.687079	0.0	0.0	0.0000	-3304.9901	-3304.9901	2.043998e+03	0.384842	0.1419	0.065	0.212598	0.212598
2023-03-15 18:00:00+01:00	-3.000000	15.404325	3000.0	0.0	8018.4043	0.0000	8018.4043	-5.000000e+03	0.859842	0.1419	0.065	-0.427886	-0.427886
2023-03-15 19:00:00+01:00	-3.000000	15.121571	0.0	0.0	0.0000	-4731.8784	-4731.8784	4.750000e+03	0.359842	0.1419	0.065	0.305426	0.305426
2023-03-15 20:00:00+01:00	-3.000000	14.838817	3000.0	0.0	8017.8388	0.0000	8017.8388	-5.000000e+03	0.834842	0.1419	0.065	-0.427806	-0.427806
2023-03-15 21:00:00+01:00	-3.000000	14.556063	0.0	0.0	0.0000	-4732.4439	-4732.4439	4.750000e+03	0.334842	0.1419	0.065	0.305543	0.305543
2023-03-15 22:00:00+01:00	-3.000000	14.273309	3000.0	5000.0	9000.0000	0.0000	9000.0000	-9.827267e+02	0.428201	0.1419	0.065	-1.137225	-1.137225
2023-03-15 23:00:00+01:00	-3.000000	13.990555	3000.0	5000.0	9000.0000	0.0000	9000.0000	-9.830094e+02	0.521587	0.1419	0.065	-1.137185	-1.137185
2023-03-16 00:00:00+01:00	-3.000000	13.707801	0.0	5000.0	9000.0000	0.0000	9000.0000	-3.983292e+03	0.900000	0.1419	0.065	-0.711445	-0.711445
2023-03-16 01:00:00+01:00	-3.000000	13.425047	0.0	0.0	0.0000	-3579.0065	-3579.0065	3.595432e+03	0.521534	0.1419	0.065	0.230730	0.230730
2023-03-16 02:00:00+01:00	-3.000000	13.142293	0.0	5000.0	9000.0000	0.0000	9000.0000	-3.983858e+03	0.900000	0.1419	0.065	-0.711365	-0.711365
2023-03-16

davidusb · March 16, 2023, 9:57pm

As shown in the documentation yaml is tricky with that syntax, try:

shell_command: 
  trigger_nordpool_forecast: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"load_cost_forecast\":{{((state_attr('sensor.nordpool', 'raw_today') | map(attribute='value') | list + state_attr('sensor.nordpool', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24] }},\"prod_price_forecast\":{{((state_attr('sensor.nordpool', 'raw_today') | map(attribute='value') | list + state_attr('sensor.nordpool', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24]}}}' http://localhost:5000/action/dayahead-optim"

markpurcell · March 17, 2023, 12:49am

One of the complexities that is hard to resolve is the management of double quoting, you have " inside ", and ’ inside ’ which are not usually interpreted correctly.

The other step I would recommend is to get the curl command running on the command line first, before you try and incorporate into a shell_command.

For example have a look inside your DeveloperTools:Template section to confirm the expansion of your calculations is correct and then see if it will execute on the command line, before you attempt to execute inside a shell_command: Indeed it would be helpful if you could post the expansion of your curl command here.,

$ curl -i -H “Content-Type: application/json” -X POST -d ‘{“load_cost_forecast”:{{((state_attr(‘sensor.nordpool’, ‘raw_today’) | map(attribute=‘value’) | list + state_attr(‘sensor.nordpool’, ‘raw_tomorrow’) | map(attribute=‘value’) | list))[now().hour:][:24] }},“prod_price_forecast”:{{((state_attr(‘sensor.nordpool’, ‘raw_today’) | map(attribute=‘value’) | list + state_attr(‘sensor.nordpool’, ‘raw_tomorrow’) | map(attribute=‘value’) | list))[now().hour:][:24]}}}’ http://localhost:5000/action/dayahead-optim

nixtagangsta · March 17, 2023, 6:22am

Thx

When i run this command in template section

“curl -i -H “Content-Type: application/json” -X POST -d ‘{“load_cost_forecast”:{{((state_attr(‘sensor.nordpool’, ‘raw_today’) | map(attribute=‘value’) | list + state_attr(‘sensor.nordpool’, ‘raw_tomorrow’) | map(attribute=‘value’) | list))[now().hour:][:24] }},“prod_price_forecast”:{{((state_attr(‘sensor.nordpool’, ‘raw_today’) | map(attribute=‘value’) | list + state_attr(‘sensor.nordpool’, ‘raw_tomorrow’) | map(attribute=‘value’) | list))[now().hour:][:24]}}}’ http://localhost:5000/action/dayahead-optim”

I’ll get

“curl -i -H “Content-Type: application/json” -X POST -d ‘{“load_cost_forecast”:[46.614, 46.992, 45.943, 46.013, 45.593, 45.523, 45.227, 45.284, 42.204, 41.7, 41.125, 40.495, 37.176, 33.873, 32.696, 31.295, 25.722],“prod_price_forecast”:[46.614, 46.992, 45.943, 46.013, 45.593, 45.523, 45.227, 45.284, 42.204, 41.7, 41.125, 40.495, 37.176, 33.873, 32.696, 31.295, 25.722]}’ http://localhost:5000/action/dayahead-optim”

davidusb · March 17, 2023, 6:41am

Now try it in a console but with the correct syntax.

nixtagangsta · March 17, 2023, 6:46am

thx davidusb but thats the same syntax i’m running except found to spacing before the + sign…
but no difference

I see Status: Infeasible in latest log. could it be some problem?

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun emhass (no readiness notification)
s6-rc: info: service legacy-services successfully started
2023-03-17 07:42:55,884 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2023-03-17 07:42:55,885 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2023-03-17 07:42:55,887 - web_server - INFO - The data path is: /share
2023-03-17 07:42:55,917 - web_server - INFO - Using core emhass version: 0.4.5
waitress INFO Serving on http://0.0.0.0:5000
2023-03-17 07:43:54,505 - web_server - INFO - Setting up needed data
2023-03-17 07:43:55,469 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-03-17 07:44:00,963 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-03-17 07:44:00,979 - web_server - INFO - Retrieve hass get data method initiated…
2023-03-17 07:44:17,759 - web_server - INFO - >> Performing dayahead optimization…
2023-03-17 07:44:17,759 - web_server - INFO - Performing day-ahead forecast optimization
2023-03-17 07:44:17,779 - web_server - INFO - Perform optimization for the day-ahead
2023-03-17 07:44:19,184 - web_server - INFO - Status: Infeasible
2023-03-17 07:44:19,186 - web_server - INFO - Total value of the Cost function = 14.0
2023-03-17 07:44:29,865 - web_server - INFO - Setting up needed data
2023-03-17 07:44:29,887 - web_server - INFO - >> Publishing data…
2023-03-17 07:44:29,888 - web_server - INFO - Publishing data to HASS instance
2023-03-17 07:44:30,103 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 642.53
2023-03-17 07:44:30,219 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 7.76
2023-03-17 07:44:30,310 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2023-03-17 07:44:30,428 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = -634.77
2023-03-17 07:44:30,531 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 21.03
2023-03-17 07:44:30,620 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 0.0
2023-03-17 07:44:30,712 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = 14.0
2023-03-17 07:44:52,197 - web_server - INFO - EMHASS server online, serving index.html…
2023-03-17 07:47:16,325 - web_server - INFO - EMHASS server online, serving index.html…

nixtagangsta · March 17, 2023, 7:37am

This is the log output
But nothing in emhass webpage

2023-03-17 08:34:05,524 - web_server - INFO - Setting up needed data
2023-03-17 08:34:05,728 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-03-17 08:34:11,375 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-03-17 08:34:11,378 - web_server - INFO - Retrieve hass get data method initiated…
2023-03-17 08:34:15,980 - web_server - INFO - >> Performing dayahead optimization…
2023-03-17 08:34:15,980 - web_server - INFO - Performing day-ahead forecast optimization
2023-03-17 08:34:16,001 - web_server - INFO - Perform optimization for the day-ahead
2023-03-17 08:34:16,260 - web_server - INFO - Status: Infeasible
2023-03-17 08:34:16,262 - web_server - INFO - Total value of the Cost function = 14.0
2023-03-17 08:34:43,959 - web_server - INFO - Setting up needed data
2023-03-17 08:34:43,979 - web_server - INFO - >> Publishing data…
2023-03-17 08:34:43,980 - web_server - INFO - Publishing data to HASS instance
2023-03-17 08:34:44,122 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 1213.67
2023-03-17 08:34:44,202 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 7.76
2023-03-17 08:34:44,272 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2023-03-17 08:34:44,334 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 1667.02
2023-03-17 08:34:44,430 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = -2.55
2023-03-17 08:34:44,533 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = -2872.93
2023-03-17 08:34:44,644 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = 14.0

smi · March 17, 2023, 8:45am

@davidusb any idea about the high P_gird_pos and P_grid_neg?

Also I really think it’s not taking the correct time to run the defferable load:

While the unit_load_cost is low, the unit_prod_price is very high while P_deferrable0 is running while plenty of solar is available.
Therefore I’d expect it to run the deferrable loads during the night where pv production is 0 and prices are still lower then unit_prod_price (which is the same every hour):

Thanks

davidusb · March 17, 2023, 9:52am

Check this:

davidusb · March 17, 2023, 9:55am

Not the same syntax with the \".
Anyways it seems to call the optimization task but if it is unfeasible is problematic.
This will depend on the options that you have activated on the add-on configuration and on your load and PV sensor data.

davidusb · March 17, 2023, 10:03am

Also pull the latest docker container version and check your configuration syntax. You have some missing elements and some of the parameters in plant_conf should be passed as lists.
Like shown here: emhass/config_emhass.yaml at 188f67ee203c850718ae3bfb56a103c6f128ef64 · davidusb-geek/emhass · GitHub

smi · March 17, 2023, 10:03am

Yes it is.

I decreased the forecast horizon but it still is infeasible.

Would rounding the numbers help?`

Thanks

nixtagangsta · March 17, 2023, 10:33am

Ahh changed to your but with no success

2023-03-17 11:29:56,232 - web_server - INFO - Status: Infeasible

smi · March 17, 2023, 10:36am

Changed them to list items and the only missing thing seems to be set_nocharge_from_grid which I don’t think has some influence.

nixtagangsta · March 17, 2023, 10:52am

Do you see any obvious config error?
for now i just want the battery management option

hass_url: empty
long_lived_token: empty
costfun: profit
optimization_time_step: 60
historic_days_to_retrieve: 2
method_ts_round: nearest
set_total_pv_sell: true
lp_solver: COIN_CMD
lp_solver_path: /usr/bin/cbc
set_nocharge_from_grid: false
load_forecast_method: naive
sensor_power_photovoltaics: sensor.power_photovoltaics
sensor_power_load_no_var_loads: sensor.accumulated
number_of_deferrable_loads: 1
list_nominal_power_of_deferrable_loads:
  - nominal_power_of_deferrable_loads: 3000
list_operating_hours_of_each_deferrable_load:
  - operating_hours_of_each_deferrable_load: 0
list_peak_hours_periods_start_hours:
  - peak_hours_periods_start_hours: "02:54"
list_peak_hours_periods_end_hours:
  - peak_hours_periods_end_hours: "15:24"
list_treat_deferrable_load_as_semi_cont:
  - treat_deferrable_load_as_semi_cont: false
load_peak_hours_cost: 0.1907
load_offpeak_hours_cost: 0.1419
photovoltaic_production_sell_price: 0.065
maximum_power_from_grid: 9000
list_pv_module_model:
  - pv_module_model: Phono_Solar_Technology_Co__Ltd__PS360MH_24_T
list_pv_inverter_model:
  - pv_inverter_model: Huawei_Technologies_Co___Ltd___SUN2000_10KTL_USL0__240V_
list_surface_tilt:
  - surface_tilt: 40
list_surface_azimuth:
  - surface_azimuth: 180
list_modules_per_string:
  - modules_per_string: 33
list_strings_per_inverter:
  - strings_per_inverter: 1
set_use_battery: true
battery_discharge_power_max: 5000
battery_charge_power_max: 5000
battery_discharge_efficiency: 0.95
battery_charge_efficiency: 0.95
battery_nominal_energy_capacity: 10000
battery_minimum_state_of_charge: 0.3
battery_maximum_state_of_charge: 0.9
battery_target_state_of_charge: 0.15

davidusb · March 17, 2023, 11:00am

Please edit your post putting your yaml configuration inside a “preformatted text” option. Otherwise we can’t check syntax and is quite unreadable.

davidusb · March 17, 2023, 12:42pm

Ok, I don’t see any problems with that configuration.
Check that you have correct values on sensor.accumulated and also now just provide a number of operating hours other than zero to see if it converges.

smi · March 20, 2023, 7:39am

@davidusb I changed the number of upcoming hours I put into the call and also rounded the values to two digits where both made no change.

Do you have any idea what I can do about EMHASS returning infeasible state?

Thanks