EMHASS: An Energy Management for Home Assistant

Same:

2023-11-29 11:00:08,310 - web_server - INFO - Setting up needed data
2023-11-29 11:00:08,312 - web_server - ERROR - ERROR: The passed data is either not a list or the length is not correct, length should be 24
2023-11-29 11:00:08,312 - web_server - ERROR - Passed type is <class 'list'> and length is 11
2023-11-29 11:00:08,312 - web_server - ERROR - ERROR: The passed data is either not a list or the length is not correct, length should be 24
2023-11-29 11:00:08,312 - web_server - ERROR - Passed type is <class 'list'> and length is 11
2023-11-29 11:00:08,315 - web_server - INFO - Retrieving weather forecast data using method = list
2023-11-29 11:00:08,316 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-11-29 11:00:08,317 - web_server - INFO - Retrieve hass get data method initiated...
2023-11-29 11:00:08,679 - web_server - ERROR - The retrieved JSON is empty, check that correct day or variable names are passed
2023-11-29 11:00:08,680 - web_server - ERROR - Either the names of the passed variables are not correct or days_to_retrieve is larger than the recorded history of your sensor (check your recorder settings)
2023-11-29 11:00:08,680 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1455, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 869, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 867, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 852, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "src/emhass/web_server.py", line 179, in action_call
    input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
  File "/usr/local/lib/python3.8/site-packages/emhass-0.5.1-py3.8.egg/emhass/command_line.py", line 91, in set_input_data_dict
    P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'])
  File "/usr/local/lib/python3.8/site-packages/emhass-0.5.1-py3.8.egg/emhass/forecast.py", line 585, in get_load_forecast
    rh.get_data(days_list, var_list)
  File "/usr/local/lib/python3.8/site-packages/emhass-0.5.1-py3.8.egg/emhass/retrieve_hass.py", line 147, in get_data
    self.df_final = pd.concat([self.df_final, df_day], axis=0)
UnboundLocalError: local variable 'df_day' referenced before assignment

tested the templates in home assitant.

{{min(24, ((state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_today') | map(attribute='value') | list  + state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_tomorrow') | map(attribute='value') | list)[now().hour:][:24]|list|length))}}

Result: 11
{{((state_attr('sensor.nordpool_kwh_ee_eur_3_05_02', 'raw_today') | map(attribute='value') | list  + state_attr('sensor.nordpool_kwh_ee_eur_3_05_02', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24] }}

Result:
[
  0.157,
  0.196,
  0.182,
  0.3,
  0.24,
  0.3,
  0.3,
  0.24,
  0.193,
  0.167,
  0.154
]
{{((state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_today') | map(attribute='value') | list  + state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24] }}

Result:
[
  0.131,
  0.164,
  0.152,
  0.25,
  0.2,
  0.25,
  0.25,
  0.2,
  0.161,
  0.139,
  0.129
]
{{([states('sensor.solcast_pv_forecast_power_now')|int(0)] + state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedHourly')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list + state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedHourly')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list)| tojson}}

Result:
[
  256,
  123,
  12,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  41,
  116,
  167,
  173,
  130,
  62,
  5,
  0,
  0,
  0,
  0,
  0,
  0,
  0,
  0
]

Maybe something to do with my config?

# Configuration file for EMHASS

retrieve_hass_conf:
  - freq: 60 # The time step to resample retrieved data from hass in minutes
  - days_to_retrieve: 2 # We will retrieve data from now and up to days_to_retrieve days
  - var_PV: 'sensor.inverter_active_power' # Photovoltaic produced power sensor in Watts
  - var_load: 'sensor.power_load_no_var_loads' # Household power consumption sensor in Watts (deferrable loads should be substracted)
  - load_negative: False # Set to True if the retrived load variable is negative by convention
  - set_zero_min: True # A special treatment for a minimum value saturation to zero. Values below zero are replaced by nans
  - var_replace_zero: # A list of retrived variables that we would want  to replace nans with zeros
    - 'sensor.inverter_active_power'
  - var_interp: # A list of retrived variables that we would want to interpolate nan values using linear interpolation
    - 'sensor.inverter_active_power'
    - 'sensor.power_load_no_var_load'
  - method_ts_round: 'nearest' # Set the method for timestamp rounding, options are: first, last and nearest

optim_conf:
  - set_use_battery: False # consider a battery storage
  - delta_forecast: 1 # days
  - num_def_loads: 2
  - P_deferrable_nom: # Watts
    - 2700.0
    - 4000.0
  - def_total_hours: # hours
    - 4
    - 10
  - treat_def_as_semi_cont: # treat this variable as semi continuous 
    - True
    - True
  - set_def_constant: # set as a constant fixed value variable with just one startup for each 24h
    - False
    - False
  - weather_forecast_method: 'scrapper' # options are 'scrapper' and 'csv'
  - load_forecast_method: 'naive' # options are 'csv' to load a custom load forecast from a CSV file or 'naive' for a persistance model
  - load_cost_forecast_method: 'hp_hc_periods' # options are 'hp_hc_periods' for peak and non-peak hours contracts and 'csv' to load custom cost from CSV file 
  - list_hp_periods: # list of different tariff periods (only needed if load_cost_forecast_method='hp_hc_periods')
    - period_hp_1:
      - start: '02:54'
      - end: '15:24'
    - period_hp_2:
      - start: '17:24'
      - end: '20:24'
  - load_cost_hp: 0.1907 # peak hours load cost in €/kWh (only needed if load_cost_forecast_method='hp_hc_periods')
  - load_cost_hc: 0.1419 # non-peak hours load cost in €/kWh (only needed if load_cost_forecast_method='hp_hc_periods')
  - prod_price_forecast_method: 'constant' # options are 'constant' for constant fixed value or 'csv' to load custom price forecast from a CSV file
  - prod_sell_price: 0.065 # power production selling price in €/kWh (only needed if prod_price_forecast_method='constant')
  - set_total_pv_sell: False # consider that all PV power is injected to the grid (self-consumption with total sell)
  - lp_solver: 'PULP_CBC_CMD' # set the name of the linear programming solver that will be used
  - lp_solver_path: 'empty' # set the path to the LP solver
  - set_nocharge_from_grid: False # avoid battery charging from the grid
  - set_nodischarge_to_grid: True # avoid battery discharging to the grid
  - set_battery_dynamic: False # add a constraint to limit the dynamic of the battery power in power per time unit
  - battery_dynamic_max: 0.9 # maximum dynamic positive power variation in percentage of battery maximum power
  - battery_dynamic_min: -0.9 # minimum dynamic negative power variation in percentage of battery maximum power

plant_conf:
  - P_grid_max: 10000 # The maximum power that can be supplied by the utility grid in Watts
  - module_model: # The PV module model
    - 'CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M'
  - inverter_model: # The PV inverter model
    - 'Fronius_International_GmbH__Fronius_Primo_5_0_1_208_240__240V_'
  - surface_tilt: # The tilt angle of your solar panels
    - 30
  - surface_azimuth: # The azimuth angle of your PV installation
    - 205
  - modules_per_string: # The number of modules per string
    - 16 
  - strings_per_inverter: # The number of used strings per inverter
    - 1
  - Pd_max: 1000 # If your system has a battery (set_use_battery=True), the maximum discharge power in Watts
  - Pc_max: 1000 # If your system has a battery (set_use_battery=True), the maximum charge power in Watts
  - eta_disch: 0.95 # If your system has a battery (set_use_battery=True), the discharge efficiency
  - eta_ch: 0.95 # If your system has a battery (set_use_battery=True), the charge efficiency
  - Enom: 5000 # If your system has a battery (set_use_battery=True), the total capacity of the battery stack in Wh
  - SOCmin: 0.3 # If your system has a battery (set_use_battery=True), the minimun allowable battery state of charge
  - SOCmax: 0.9 # If your system has a battery (set_use_battery=True), the minimun allowable battery state of charge
  - SOCtarget: 0.6 # If your system has a battery (set_use_battery=True), the desired battery state of charge at the end of each optimization cycle

Here’s my dayahead jinja code.

{
  "load_cost_forecast": {{
    ([states('sensor.cecil_st_general_price')|float(0)] + 
    state_attr('sensor.cecil_st_general_forecast', 'forecasts') |map(attribute='per_kwh')|list) 
    | tojson 
  }},
  "prod_price_forecast": {{
    ([states('sensor.cecil_st_feed_in_price')|float(0)] +
    (state_attr('sensor.cecil_st_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list))
    | tojson 
  }},
  "pv_power_forecast": {{
    ([states('sensor.sonnenbatterie_84324_production_w')|int(0)] +
    state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list +
    state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list
    )| tojson
  }},
  "num_def_loads": 2,
  "def_total_hours": [3, 3],
  "P_deferrable_nom":  [1300, 2300],
  "treat_def_as_semi_cont": [1, 0]  
}

This code produces the following when you test it in developers tools:

{
  "load_cost_forecast": [0.16, 0.15, 0.16, 0.16, 0.16, 0.16, 0.16, 0.16, 0.15, 0.15, 0.15, 0.15, 0.14, 0.15, 0.16, 0.16, 0.16, 0.15, 0.13, 0.1, 0.09, 0.09, 0.09, 0.06, 0.05, 0.07, 0.06, 0.06, 0.07, 0.07, 0.07, 0.1, 0.35, 0.36, 0.36, 0.36, 0.36, 0.37, 0.37, 0.38, 0.41, 0.41, 0.42, 0.42, 0.2, 0.19, 0.16, 0.16, 0.16],
  "prod_price_forecast": [0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.06, 0.06, 0.06, 0.05, 0.04, 0.01, -0.0, 0.0, 0.0, -0.02, -0.06, -0.04, -0.05, -0.05, -0.04, -0.04, -0.04, -0.01, 0.27, 0.28, 0.28, 0.28, 0.28, 0.28, 0.28, 0.3, 0.32, 0.32, 0.33, 0.33, 0.09, 0.09, 0.07, 0.07, 0.06],
  "pv_power_forecast": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12, 81, 177, 577, 1047, 1555, 2097, 2564, 2965, 3261, 3336, 3363, 3407, 3400, 3383, 3287, 3118, 2925, 2694, 2435, 2162, 1878, 1591, 1310, 1003, 654, 305, 82, 7, 0, 0, 0, 0, 0, 0, 0, 0],
  "num_def_loads": 2,
  "def_total_hours": [3, 3],
  "P_deferrable_nom":  [1300, 2300],
  "treat_def_as_semi_cont": [1, 0]  
}

I post this using node-red flow:

Here’s the JSON code for this flow:

[{"id":"962843add2ce0c1c","type":"debug","z":"65840aa926d9c567","name":"debug Day Ahead POST","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1130,"y":1160,"wires":[]},{"id":"b3613da7047487af","type":"http request","z":"65840aa926d9c567","name":"POST Day Ahead","method":"use","ret":"txt","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":770,"y":1100,"wires":[["962843add2ce0c1c","710a3b34e8103b99"]]},{"id":"6cfcb69f84018e45","type":"inject","z":"65840aa926d9c567","d":true,"name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"59 03 * * *","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"str","x":170,"y":1100,"wires":[["5e6c81ecba2fb539"]]},{"id":"61cffcb335f42876","type":"change","z":"65840aa926d9c567","name":"","rules":[{"t":"set","p":"url","pt":"msg","to":"http://192.168.99.17:5000/action/dayahead-optim","tot":"str"},{"t":"set","p":"method","pt":"msg","to":"POST","tot":"str"},{"t":"set","p":"headers","pt":"msg","to":"Content-Type: application/json","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":580,"y":1100,"wires":[["b3613da7047487af"]]},{"id":"aa71f4422db79ef8","type":"comment","z":"65840aa926d9c567","name":"Testing dayahead POST","info":"","x":210,"y":1060,"wires":[]},{"id":"7e925a1388464de2","type":"change","z":"65840aa926d9c567","name":"","rules":[{"t":"set","p":"url","pt":"msg","to":"http://192.168.99.17:5000/action/publish-data","tot":"str"},{"t":"set","p":"method","pt":"msg","to":"POST","tot":"str"},{"t":"set","p":"headers","pt":"msg","to":"Content-Type: application/json","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"{}","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":560,"y":1160,"wires":[["43cfe6b46818abdf"]]},{"id":"43cfe6b46818abdf","type":"http request","z":"65840aa926d9c567","name":"POST Publish data","method":"use","ret":"txt","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":770,"y":1160,"wires":[["710a3b34e8103b99","962843add2ce0c1c"]]},{"id":"710a3b34e8103b99","type":"function","z":"65840aa926d9c567","name":"POST return != 201","func":"var forecastData = flow.get('forecastData');\nvar status = msg.statusCode;\nif(status != 201)\n{\n    var timestamp = new Date().toISOString();\n    var newpayload = [timestamp, msg.statusCode, msg.url, msg.payload, forecastData];\n    msg.payload = newpayload;\n\n    return msg;\n}\nelse\nreturn;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1150,"y":1100,"wires":[["d1c6b5976c8967e6","3034893cf2f75e12"]]},{"id":"d1c6b5976c8967e6","type":"api-call-service","z":"65840aa926d9c567","name":"Notify Robert's iPhones","server":"afc27684.cf6ed8","version":5,"debugenabled":false,"domain":"notify","service":"mobile_app_robsiphone","areaId":[],"deviceId":[],"entityId":[],"data":"{\"title\":\"POST to EMHASS service failed\",\"message\":\"error: '{{statusCode}}' URL: '{{responseUrl}}'\"}","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":1470,"y":1060,"wires":[[]]},{"id":"3034893cf2f75e12","type":"file","z":"65840aa926d9c567","name":"write error","filename":"/share/EMHASSDayAheadPostError.log","filenameType":"str","appendNewline":true,"createDir":true,"overwriteFile":"false","encoding":"none","x":1430,"y":1140,"wires":[[]]},{"id":"25a81a7ad77094cb","type":"inject","z":"65840aa926d9c567","d":true,"name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"300","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":190,"y":1160,"wires":[["7e925a1388464de2"]]},{"id":"5e6c81ecba2fb539","type":"api-render-template","z":"65840aa926d9c567","name":"Day Ahead","server":"afc27684.cf6ed8","version":0,"template":"{\n  \"load_cost_forecast\": {{\n    ([states('sensor.cecil_st_general_price')|float(0)] + \n    state_attr('sensor.cecil_st_general_forecast', 'forecasts') |map(attribute='per_kwh')|list) \n    | tojson \n  }},\n  \"prod_price_forecast\": {{\n    ([states('sensor.cecil_st_feed_in_price')|float(0)] +\n    (state_attr('sensor.cecil_st_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list))\n    | tojson \n  }},\n  \"pv_power_forecast\": {{\n    ([states('sensor.sonnenbatterie_84324_production_w')|int(0)] +\n    state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list +\n    state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list\n    )| tojson\n  }},\n  \"num_def_loads\": 2,\n  \"def_total_hours\": [3, 3],\n  \"P_deferrable_nom\":  [1300, 2300],\n  \"treat_def_as_semi_cont\": [1, 0]  \n}","resultsLocation":"payload","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":390,"y":1100,"wires":[["61cffcb335f42876"]]},{"id":"afc27684.cf6ed8","type":"server","name":"Home Assistant","version":5,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":30,"areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m","enableGlobalContextStore":true}]

I’m using "load_cost_forecast" in my conf and you are using “load_cost_forecast”, could the different format have something to do with it?

My jinja:

{
    \"load_cost_forecast\":{{((state_attr('sensor.nordpool_kwh_ee_eur_3_05_02', 'raw_today') | map(attribute='value') | list  + state_attr('sensor.nordpool_kwh_ee_eur_3_05_02', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24] }},
    \"prod_price_forecast\":{{((state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_today') | map(attribute='value') | list  + state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_tomorrow') | map(attribute='value') | list))[now().hour:][:24] }}, 
    \"prediction_horizon\":{{min(24, (((state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_today')|map(attribute='value')|list + state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_tomorrow') | map(attribute='value')| list)[now().hour:][:24]|list|length)))}},    
    \"pv_power_forecast\":{{([states('sensor.solcast_pv_forecast_power_now')|int(0)] + state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedHourly')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list + state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedHourly')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list)| tojson}}
    }

And developer tools:

{
    \"load_cost_forecast\":[0.157, 0.196, 0.182, 0.3, 0.24, 0.3, 0.3, 0.24, 0.193, 0.167, 0.154],
    \"prod_price_forecast\":[0.131, 0.164, 0.152, 0.25, 0.2, 0.25, 0.25, 0.2, 0.161, 0.139, 0.129], 
    \"prediction_horizon\":11,    
    \"pv_power_forecast\":[214, 123, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 41, 116, 167, 173, 130, 62, 5, 0, 0, 0, 0, 0, 0, 0, 0]
    }

Looks quite ok

I don’t use prediction_horizon in dayahead, only in naive-mpc-optim which I run every 60 seconds.

{
  "prod_price_forecast": {{
    ([states('sensor.cecil_st_feed_in_price')|float(0)] +
    (state_attr('sensor.cecil_st_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list))
    | tojson 
  }},
  "load_cost_forecast": {{
    ([states('sensor.cecil_st_general_price')|float(0)] + 
    state_attr('sensor.cecil_st_general_forecast', 'forecasts') |map(attribute='per_kwh')|list) 
    | tojson 
  }},
  "pv_power_forecast": {{
    ([states('sensor.sonnenbatterie_84324_production_w')|int(0)] +
    state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list +
    state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list
    )| tojson
  }},
  "prediction_horizon": {{
    min(48, (state_attr('sensor.cecil_st_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1)
  }},
  "num_def_loads": 2,
  "def_total_hours": [
    {%- if is_state('sensor.openweathermap_forecast_condition', ['rainy']) -%}
      0
    {%- elif is_state('sensor.season', 'winter') -%}
      2
    {%- elif is_state('sensor.season', 'summer') -%}
      4
    {%- else -%}
      3
    {%- endif -%},
    {%- if is_state('device_tracker.ynot_location_tracker', ['home']) -%}
      {%- if is_state('cover.ynot_charger_door', ['open']) -%}
        {{ ((90-(states('sensor.ynot_battery')|int(0)))/30*3)|int(0) }}
      {%- else -%} 
        0
      {%- endif -%}
    {%- else -%} 
      0
    {%- endif -%}
    ],
  "P_deferrable_nom": [1300, 7360],
  "treat_def_as_semi_cont": [1, 0],
  "set_def_constant": [0, 0],
  "soc_init": {{ (states('sensor.sonnenbatterie_84324_state_charge_user')|int(0))/100 }},
  "soc_final": 0.03,
  "alpha": 0.25,
  "beta": 0.75
}

This produces the following, note this is where I include prediction_horizon:

{
  "prod_price_forecast": [0.07, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.05, 0.05, 0.05, 0.05, 0.05, 0.06, 0.06, 0.06, 0.05, 0.04, 0.02, -0.0, 0.01, 0.0, -0.02, -0.06, -0.04, -0.05, -0.04, -0.04, -0.04, -0.04, -0.04, 0.27, 0.29, 0.28, 0.28, 0.28, 0.28, 0.28, 0.3, 0.32, 0.32, 0.33, 0.33, 0.09, 0.09, 0.07, 0.07, 0.06],
  "load_cost_forecast": [0.16, 0.16, 0.16, 0.16, 0.16, 0.16, 0.16, 0.16, 0.16, 0.15, 0.15, 0.15, 0.15, 0.15, 0.16, 0.16, 0.16, 0.15, 0.13, 0.11, 0.09, 0.1, 0.09, 0.06, 0.05, 0.07, 0.06, 0.07, 0.07, 0.07, 0.07, 0.07, 0.35, 0.37, 0.36, 0.36, 0.36, 0.37, 0.37, 0.38, 0.41, 0.41, 0.42, 0.42, 0.2, 0.19, 0.16, 0.16, 0.16],
  "pv_power_forecast": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12, 81, 177, 577, 1047, 1555, 2097, 2564, 2965, 3261, 3336, 3363, 3407, 3400, 3383, 3287, 3118, 2925, 2694, 2435, 2162, 1878, 1591, 1310, 1003, 654, 305, 82, 7, 0, 0, 0, 0, 0, 0, 0, 0],
  "prediction_horizon": 48,
  "num_def_loads": 2,
  "def_total_hours": [0,0],
  "P_deferrable_nom": [1300, 7360],
  "treat_def_as_semi_cont": [1, 0],
  "set_def_constant": [0, 0],
  "soc_init": 0.1,
  "soc_final": 0.03,
  "alpha": 0.25,
  "beta": 0.75
}
1 Like

Dynamic prediction_horizon is configured, because Nordpool next day hourly prices will be given around 1PM, but it can take even longer for the HA integration to get them. So this template will check how many hours of data is available:

{{min(24, (((state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_today')|map(attribute='value')|list + state_attr('sensor.nordpool_kwh_ee_eur_3_10_0', 'raw_tomorrow') | map(attribute='value')| list)[now().hour:][:24]|list|length)))}}

And set the prediction_horizon accordingly. With this you can run the shell command and not be worried of errors.

But back to my case. I think the issue at the moment is that I don’t have enough data on the sensor:

var_load: 'sensor.power_load_no_var_loads'

I changed the calculation behind it and this zeroed the history. So to my understanding i just have to wait for 48 hours now. I can probably check that if i change it to some other power reading which has history in HA

1 Like

I’m with amber electric here in Australia. Amber publishes 24h Forecast after 12:30 each day with 48x30 minute forecasts until 03:30 after which the forecasts stop. So, between 03:30 and 12:30 you have incrementally less than 48 forecasts. At 12:00 you have the smallest number of forecasts 12:00-03:30 the next day which is 33x30 minute forecasts.

However I only run dayahead once a day at 03:59 (if I’m not using MPC for some reason) so I guess this doesn’t need a rediction horizon set.

My power_load_no_var_loads is taken from my sonnen battery using a custom integration.

# Calculate power consumption for cecil st less the deferrable appliances
  - platform: template
    sensors:
      house_power_consumption_less_deferrables:
        unique_id: house_power_consumption_less_deferrables
        unit_of_measurement: W
        device_class: power
        value_template: >-
          {% set consumption = states('sensor.sonnenbatterie_84324_consumption_w') | float(0) %}
          {% if is_state('switch.garage_power_point_l1', 'on') %}
            {# If the pool light is on subtract 11 watts #}
              {% set deferrable0 = states('sensor.garage_power_point_power') | float(0) - 11 %}
          {% else %}
              {% set deferrable0 = states('sensor.garage_power_point_power') | float(0) %}
          {% endif %}
          {% set deferrable1 = states('sensor.ynot_home_charge') | float(0) if is_state('device_tracker.ynot_location_tracker', 'home') else 0 %}
          {# below code stops resulte dropping below 0 when consumption hasn't cought up #}
          {% if (consumption - (deferrable0 + deferrable1)) | float(0) <= 0 %}
            {{ consumption | float(0) }}
          {% else %}
            {{ consumption - (deferrable0 + deferrable1) | float(0) }}
          {% endif %}

I still get some negative values:

You can use the max function to ensure that the final output is always zero or a positive number. Here’s the updated template:

- platform: template
  sensors:
    house_power_consumption_less_deferrables:
      unique_id: house_power_consumption_less_deferrables
      unit_of_measurement: "W"
      device_class: power
      value_template: >-
        {% set consumption = states('sensor.sonnenbatterie_84324_consumption_w') | float(0) %}
        {% if is_state('switch.garage_power_point_l1', 'on') %}
          {% set deferrable0 = states('sensor.garage_power_point_power') | float(0) - 11 %}
        {% else %}
          {% set deferrable0 = states('sensor.garage_power_point_power') | float(0) %}
        {% endif %}
        {% set deferrable1 = states('sensor.ynot_home_charge') | float(0) if is_state('device_tracker.ynot_location_tracker', 'home') else 0 %}
        {% set total_deferrables = deferrable0 + deferrable1 %}
        {% set net_consumption = consumption - total_deferrables %}
        {{ [net_consumption, 0] | max }}

2 Likes

I don’t know what happened, but it works :slight_smile:

Just one question. It looks like the UI is off by 2 hours. Estonia is GMT +2 and probably this is the reason.

It should be 15:00, but it’s 13:00 + 2:00 doens’t bother som much in the table

But in the graph would be nice to see correct time:

In secrets_emhass.yaml time_zone is set to Europe/Tallinn

time_zone: Europe/Tallinn

Try to play with the rounding parameter. Have a look here: https://community.home-assistant.io/t/emhass-an-energy-management-for-home-assistant/338126/1475?u=110hs.
If you are passing the right number of elements this should work and reflect your local time.
This is not a minor aspect as based on the time and energy price tariff EMHASS will tell you what to do.

EDIT:
Is the log showing the right time?

Nope, also 2h behind

Probably not rounding related as well, because its 2h off

Try last instead of nearest.

Maybe this is a question for those using EMHASS in a container (this is how you @arva use it, right?)… if there is any settings for the time, date and timezone.
Because 2h off seems too much… to just be a rounding problem.

Time zone was not defined in the container and it solved the -2h, but now its one hour ahead. ugh

EDIT: I put back “nearest” and now it’s in sync

Try to monitor it but in my experience “first” is the one that gives you the best flexibility (I started with “last” and then switched to “first” and changed my code as in this way till half past, regardless of how frequently you run the optim, it will not switch to the next slot until it’s time)

Would be nice if EMHASS could take in timestamps as well and match them. Also would be nice if you can send as many hours of data you have and then it will use as much coplete data it has for the prediction. At the moment i have set the prediction to 24h and as nordpool publishes next day data at 13:00 (in HA more likely 15:00) there is a time when I can upload 9 hours (24-15) and times when i can upload 33 hours of data. Solar forecast has data for 48 hours. Of course i can make automation that will update electricity prices around midnight to get consistent 24 hours but i loose the possibility to have a longer prediction between 15:00 00:00.

Can confirm - first is accurate in all times.

Maybe a little bit out or energy management scope, but would it be possible to load in my heat pump data:

  • outside temperature
  • room temperature
  • heat pump power sensor

I would like to use machine learning to calculate how long the heat pump needs to work, and what are the electricity consumption and power required to maintain room temperature in relation to the outside temperature. This information could be a good input for the defferable load and time prediction. Also heat pump power is not constant and can vary a bit. Here is a screenshot of the hesat pump power sensor:
Screenshot 2023-11-30 at 09.08.34
Heat pumo in constantly in a cycle of heating and defrosting and power is related to compressor frequency, which is related to degree minutes, which is related to several tings like outside and inside temperature and heat curve etc. At the moment I control the heat pump raising and lowering heat curve offset when electricity is cheap or expensive. Would be nice to know if this is the most optimum way, as heat pump consumes less energy when compressor is running in lower frequency. But to pinpoint most optimum running time and frequency to maintain the room temp is something that ML would probably do better.

At th moment i’m using NibePI to controll that. Creator of nibepi has done electricity, weather, and room temp oprtimizations, but would like to see more than a black box.

There are probably more data that machine learing model can take for that. I have several:


Screenshot 2023-11-30 at 09.11.58
Screenshot 2023-11-30 at 09.11.51

Screenshot 2023-11-30 at 09.11.33

I understand that it’s a totally different machine learing model, but to me it looks like EMHASS could be a good platform to do it, because integration and data flows with Home Assitant are already there.

And to make things even complicated, I have gas heater in the system too. So can choose between two heat sources or run them together. I have done the calculations when gas heating is cheaper than heat pump and have some results already. At the monent just collecting data.

More advanced thermodynamic models are on the future roadmap for EMHASS so your post is very much on topic.

I currently run a very simple model for my heat pump, which takes into consideration the temperature forecast for the coming 24 hours and allocated the requisite run hours based on a desirable set point.

image

image

It is summer here so the solar PV production and demand for my HVAC cooling is well aligned:


My automation takes the desired power level, sensor.p_deferrable3 and converts that into a set point for my HVAC using a simple linear mathematical formula:

0 W = 27 deg C
4000 W = 24 deg C

As you can see sometimes my HVAC can run away to 11 kW, but I use 4000W as an ‘average’ set point, I can adjust dynamically my p_nom_hvac using the slider, and if I set to 5000W the set point goes down to 23 deg and so on.

I also treat the sensor.power_load_no_var_loads slightly differently but I won’t go into that in this post.

A thermal model would be better, but this seems reasonable.

This approach doesn’t work for me during winter as the peak power demands for heating (overnight) and cheap solar PV energy (during the day) are out of sync. So I typically haven’t controlled my HVAC in winter via EMHASS and just catch consumption via sensor.power_load_no_var_loads

A couple of notes on the graphic, you can see when the p_deferrable3 goes to 5 kW (at 11am), the HVAC set point goes down to 23 deg C and the power consumption goes up to between 5-8 kW, you can also see at 3pm when the deferrable3 goes to 0W, the set point goes up to 27 deg C and power consumption goes down to 1-2 kW, so it is a reasonable approximation.

2 Likes

This would be a good step to start, but i’m totally clueless how to do this. I don’t know the correlation between outside temp and hours my heat pump needs to run.

I have made sensors that count’s how many hours heat pump has run heating and hot water today and i also have a sensors that calculate the daily average heating and hot water power (daily energy / running hours).
Screenshot 2023-12-01 at 09.20.32

So at the moment i can send def_total_hours and P_deferrable_nom to the prediction model by the end of day. But this is then based on yesterdays values and does not take account the outside temperature. Just model next day from previous day.

I have however a outside temp sensor as well:
Screenshot 2023-12-01 at 09.24.15

Has anybody formed any opinions about which entities should be retained in the recorder, logbook or history with EMHASS in mind? Cleaning up db before it gets too big to manage?
Also what is the common practice with historic_days_to_retrieve and purge_keep_days?