If the optimisation result is infeasible, then the numbers generally don’t make sense.
You need to correct the infeasible issue, which is difficult as it could be many different things, before trying to make sense of the numbers.
I run EMHASS regularly with negative feedin and general prices without issue, I also run a local curtailment which EMHASS is happy with if my solar is producing 0 W.
I don’t really understand what’s going on, it’s been going on for a long time, but I’m seeing this now. It could be that I missed this now at this time as the new electricity prices for tomorrow do not arrive until after 13:00
2024-06-11 11:39:30,889 - web_server - ERROR - ERROR: The passed data is either not a list or the length is not correct, length should be 13
2024-06-11 11:39:30,889 - web_server - ERROR - Passed type is <class 'list'> and length is 1
2024-06-11 11:39:30,890 - web_server - INFO - Retrieve hass get data method initiated...
2024-06-11 11:39:31,450 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2024-06-11 11:39:32,298 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 3653, in get_loc
return self._engine.get_loc(casted_key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pandas/_libs/index.pyx", line 147, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 176, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'LONGI_Green_Energy_Technology_Co_Ltd_LR4_72HPH_72HPH_425M'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1473, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 882, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 880, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 111, in action_call
input_data_dict = set_input_data_dict(emhass_conf, costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 145, in set_input_data_dict
P_PV_forecast = fcst.get_power_from_weather(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 434, in get_power_from_weather
module = cec_modules[self.plant_conf['module_model'][i]]
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/frame.py", line 3761, in __getitem__
indexer = self.columns.get_loc(key)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/core/indexes/base.py", line 3655, in get_loc
raise KeyError(key) from err
KeyError: 'LONGI_Green_Energy_Technology_Co_Ltd_LR4_72HPH_72HPH_425M'
2024-06-11 11:39:34,318 - web_server - INFO - Passed runtime parameters: {}
2024-06-11 11:39:34,318 - web_server - INFO - >> Setting input data dict
2024-06-11 11:39:34,318 - web_server - INFO - Setting up needed data
2024-06-11 11:39:34,320 - web_server - INFO - >> Publishing data...
2024-06-11 11:39:34,320 - web_server - INFO - Publishing data to HASS instance
2024-06-11 11:39:34,343 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 5253.0
2024-06-11 11:39:34,350 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 600.0
2024-06-11 11:39:34,357 - web_server - INFO - Successfully posted to sensor.p_pv_curtailment = 0.0
2024-06-11 11:39:34,364 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2024-06-11 11:39:34,371 - web_server - INFO - Successfully posted to sensor.p_deferrable1 = 0.0
2024-06-11 11:39:34,378 - web_server - INFO - Successfully posted to sensor.p_deferrable2 = 0.0
2024-06-11 11:39:34,385 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 0.0
2024-06-11 11:39:34,392 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 2.0
2024-06-11 11:39:34,399 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = -4653.0
2024-06-11 11:39:34,406 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = 27.26
2024-06-11 11:39:34,411 - web_server - INFO - Successfully posted to sensor.optim_status = Optimal
2024-06-11 11:39:34,418 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 0.58
2024-06-11 11:39:34,425 - web_server - INFO - Successfully posted to sensor.unit_prod_price = 0.58
2024-06-11 11:39:40,774 - web_server - INFO - EMHASS server online, serving index.html...
post_naive_mpc_shell_v3: >
curl -i -H "Content-Type: application/json" -X POST -d '{
"load_cost_forecast":{{((state_attr("sensor.nordpool_tibber", "raw_today") | map(attribute="value") | list + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value") | list))[now().hour:][:24] }},
"prod_price_forecast":{{((state_attr("sensor.nordpool_tibber", "raw_today") | map(attribute="value") | list + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value") | list))[now().hour:][:24] }},
"prediction_horizon":{{min(24, (((state_attr("sensor.nordpool_tibber", "raw_today")|map(attribute="value")|list + state_attr("sensor.nordpool_tibber", "raw_tomorrow") | map(attribute="value")| list)[now().hour:][:48]|list|length)))}},
"pv_power_forecast":{{([states("sensor.inverter_input_power")|int(0)] + state_attr("sensor.solcast_pv_forecast_forecast_today", "detailedHourly")|selectattr("period_start","gt",utcnow()) | map(attribute="pv_estimate")|map("multiply",1000)|map("int")|list + state_attr("sensor.solcast_pv_forecast_forecast_tomorrow", "detailedHourly")|selectattr("period_start","gt",utcnow()) | map(attribute="pv_estimate")|map("multiply",1000)|map("int")|list)| tojson}},
"load_power_forecast":{{[states('sensor.power_load_no_var_loads')|int] +(states('input_text.fi_fo_buffer').split(', ')|map('multiply',1000)|map('int')|list)[1:]}},
"soc_init":{{ max(0,states("sensor.battery_state_of_capacity")|int(0))/100 }},
"soc_final":{{ max(100,states("number.battery_end_of_discharge_soc")|int(0))/100 }},
"delta_forecast":2,
"def_total_hours":[{{ states('input_number.p_deffereble0_x_hours') | default(0) }},{{ states('sensor.p_deferrable1_hours') | default(0) }},{{ states('sensor.charger_def_hours') | default(0) }}],
"def_end_timestep": [0, 0, {{ max(0,states("sensor.emhass_deferrable2_end_timeslots")|int(0)) }}]
}' http://localhost:5000/action/naive-mpc-optim
Hi,
I need a little help getting data for “pv_power_forecast”.
I created a rest sensor like this:
sensor:
- platform: rest
name: "Solar Production Estimate ForecastSolar"
json_attributes:
- result
resource: https://api.forecast.solar/estimate/watts/52/12/37/0/5.67
method: GET
value_template: "{{ (value_json.result)|round(2) }}"
unit_of_measurement: "W"
device_class: power
scan_interval: 3600
force_update: true
Seems ok, but the returned data are not only values, so I get an error in the HA logging. The API return is this:
{
"result": {
"2024-06-09 04:52:37": 0,
"2024-06-09 05:15:00": 95,
"2024-06-09 06:00:00": 285,
"2024-06-09 07:00:00": 950,
"2024-06-09 08:00:00": 2029,
"2024-06-09 09:00:00": 2597,
"2024-06-09 10:00:00": 3077,
"2024-06-09 11:00:00": 3766,
"2024-06-09 12:00:00": 3901,
"2024-06-09 13:00:00": 3605,
"2024-06-09 14:00:00": 3002,
"2024-06-09 15:00:00": 2297,
"2024-06-09 16:00:00": 1600,
"2024-06-09 17:00:00": 1052,
"2024-06-09 18:00:00": 671,
"2024-06-09 19:00:00": 366,
"2024-06-09 20:00:00": 204,
"2024-06-09 21:30:07": 0,
"2024-06-10 04:52:16": 0,
"2024-06-10 05:15:00": 246,
"2024-06-10 06:00:00": 666,
"2024-06-10 07:00:00": 1421,
"2024-06-10 08:00:00": 2017,
"2024-06-10 09:00:00": 1840,
"2024-06-10 10:00:00": 1481,
"2024-06-10 11:00:00": 1517,
"2024-06-10 12:00:00": 1563,
"2024-06-10 13:00:00": 1532,
"2024-06-10 14:00:00": 1453,
"2024-06-10 15:00:00": 1290,
"2024-06-10 16:00:00": 1082,
"2024-06-10 17:00:00": 781,
"2024-06-10 18:00:00": 474,
"2024-06-10 19:00:00": 244,
"2024-06-10 20:00:00": 139,
"2024-06-10 21:30:52": 0
},
"message": {
"code": 0,
"type": "success",
"text": "",
"pid": "MvqPy976",
"info": {
"latitude": 52,
"longitude": 12,
"distance": 0,
"place": "L 51, Schora, Moritz, Zerbst/Anhalt, Anhalt-Bitterfeld, Sachsen-Anhalt, 39264, Deutschland",
"timezone": "Europe/Berlin",
"time": "2024-06-09T13:24:12+02:00",
"time_utc": "2024-06-09T11:24:12+00:00"
},
"ratelimit": {
"zone": "IP 84.194.12.249",
"period": 3600,
"limit": 12,
"remaining": 2
}
}
}
I only need the values from ‘result’, is there an easy way to retrieve these data?
thx
Use the below instead (and adjust as needed).
Then I created a cycle to extract the attributes values, using something like
for item in state_attr('sensor.forecast_solar_estimate_watts','watts').items()
# Forecast.solar
rest:
- resource_template: "https://api.forecast.solar/estimate/{{state_attr('zone.home','latitude')}}/{{state_attr('zone.home','longitude')}}/17/45/8.473?damping=0.15&inverter=8"
scan_interval: 86400
sensor:
- unique_id: forecast_solar_estimate_watts
name: Solar Forecast estimate watts
icon: mdi:solar-power
unit_of_measurement: W
value_template: >
{% set today = now().replace(minute=0, second=0).strftime('%Y-%m-%d %H:%M:%S') %}
{% set data = value_json.result.watts.items()|selectattr('0','search',today)|sort(attribute='1',reverse=True) %}
{{(data|first|default(('unknown',0)))[1]}}
# {% set nu = now().replace(minute=0, second=0).strftime('%Y-%m-%d %H:%M:%S') %}
# {% set watts = state_attr('sensor.forecast_solar_estimate_watts','watts')%}
# {% set data = watts.items() %}
# {{watts[nu] if nu in watts else 'unknown'}}
json_attributes_path: "$.result"
json_attributes:
- watts
- unique_id: forecast_solar_estimate_watt_hours_period
name: Solar Forecast estimate watt hours period
icon: mdi:solar-power-variant
unit_of_measurement: Wh
value_template: >
{% set today = now().replace(minute=0, second=0).strftime('%Y-%m-%d %H:%M:%S') %}
{% set data = value_json.result.watt_hours_period.items()|selectattr('0','search',today)|sort(attribute='1',reverse=True) %}
{{(data|first|default(('unknown',0)))[1]}}
json_attributes_path: "$.result"
json_attributes:
- watt_hours_period
- unique_id: forecast_solar_estimate_watt_hours
name: Solar Forecast estimate watt hours
icon: mdi:sun-clock-outline
unit_of_measurement: Wh
value_template: >
{% set today = now().replace(minute=0, second=0).strftime('%Y-%m-%d %H:%M:%S') %}
{% set data = value_json.result.watt_hours.items()|selectattr('0','search',today)|sort(attribute='1',reverse=True) %}
{{(data|first|default(('unknown',0)))[1]}}
json_attributes_path: "$.result"
json_attributes:
- watt_hours
- unique_id: forecast_solar_estimate_watt_hours_day
name: Solar Forecast estimate watt hours day
icon: mdi:calendar-today
unit_of_measurement: Wh
value_template: >
{% set today = now().strftime('%Y-%m-%d') %}
{{value_json.result.watt_hours_day[today]}}
# {% set today = now().replace(minute=0, second=0).strftime('%Y-%m-%d %H:%M:%S') %}
# {{(value_json.result.watt_hours_day.items()|list
# |selectattr('0','search',today)
# |sort(attribute='1',reverse=True)
# |first|default(('unknown',0)))[1]}}
json_attributes_path: "$.result"
json_attributes:
- watt_hours_day
- unique_id: forecast_solar_estimate_message
name: Solar Forecast estimate message
icon: mdi:message-text-outline
force_update: true
value_template: >
{% set type = value_json.message.type %}
{{type if type == 'success' else value_json.message.text}}
json_attributes_path: "$.message"
json_attributes:
- code
- type
- text
- info
- ratelimit
Today is looking alot better, I tied the battery change gateway mode automation using the Tesla Custom intergration to charge my powerwall overnight which worked out well as my retalier was not going to iniate a charge.
@markpurcell Are you running custom curl scripts to set the TOU plan details for your automation? if so…would you mind sharing?
At the moment I have a binary sensor setup if the battery if the P-Batt is below -1000 to move to TOU and set the backup to 100% (its not always working 100% but that could be due to conflicting smartshift commands)
Hi, thank you!
I will try and rewrite this for my application, I probably will have some for more questions about your code.
Wondering. How can i post sensor p grid positive and negativ from emhass to home assistant? Is it possible without changing code of emhass?
Not possible at the moment but you can easily split p_grid_forecast using sensor templates.
I am currently using NetZero to inject the dynamic pricing into the Powerwall TBC mode, then in Home Assistant I switch to TBC mode whenever EMHASS wants to export or import.
I was mucking around with a python script to inject the prices and teslemetery now has a tariff service call that can also inject the prices.
I managed to implement the code for the creation of the sensors 
You use the Public service of forecast.solar?
I have 3 planes, what if I create 3 times these sensors (with adapted names) and combine them in to one sensor by calculation (sum). Would that still work?
Or do I need the professional account plan, so I can do one API call in 3 planes?
How did you implemented the cycle and how does it function? That isn’t clear to me
Yes, I am.
Not sure combining three calls will work, you have to consider they do a check on the IP to limit the number of calls. You can try but I guess you’ll have to go for a paid subscription.
I implemented the cycle directly in my automation code.
Below you can find a rough adaptation of that code, that returns a list you can pass to EMHASS; you should be able to do that using a template sensor or so. Just keep in mind that the state has a limited number of characters available, so you’d maybe better store in the state the current PV forecast and the full list in the attribute, and then pass the latter to EMHASS.
{% set ns_production_forecast2days = namespace(production_forecast2days = []) %}
{% for item in state_attr('sensor.forecast_solar_estimate_watts','watts').items() %}
{% set ns_production_forecast2days.production_forecast2days = ns_production_forecast2days.production_forecast2days + [item[1]] %}
{% endfor %}
{{ ns_production_forecast2days.production_forecast2days }}
[0, 169, 483, 1135, 2050, 2805, 3881, 5406, 6194, 5270, 4974, 4499, 3720, 2661, 1483, 695, 0, 0, 169, 491, 1000, 1534, 1932, 2245, 2483, 2644, 2635, 2491, 2220, 1830, 1356, 746, 347, 0]
Good to know. If I end up with another house might look at a battery that allows local control.
Not sure if your using solcast but seems it’s been removed from HACS
Search back in this thread; it was mentioned somebody picked it up and is still maintaining the project.
BTW EMHASS natively supports it; check the documentation.
Yep, Found it in another thread.
Is weight_battery_discharge specifically for selling back to the grid? When I set to 0.50 the optimal solution prefers to use the grid (at around 0.30) instead of the battery at all.
Not specifically for selling back to the grid.
Given your settings above it makes sense that it is prioritising using the grid as the grid only costs 0.30 which is a lot less than the ‘cost’ to use the battery which you have set at 0.5 + whatever the prod_price is at that time (maybe 0.2 or 0.3). If you specifically want to use the battery at that time weight_battery_discharge + prod_price should be less than load_cost.
So continuing the example above (load_cost = 0.30 & prod_price = 0.2), then if weight_battery_discharge is less than 0.10 it will discharge the battery but if it is greater than 0.10 it will will supply from the grid, all other things being equal.
If you share your cost table chart we can interpolate a lot better.
Cheers! Yeah that explains it and lines up with exactly what I’m seeing. My cost chart has been through quite a few iterations since that post :).
Now onto how I can shoehorn HVAC as a deferred load.
I required a minor tweak parseFloat
return entity.attributes.battery_scheduled_soc.map((entry) => {
return [new Date(entry.date), parseFloat(entry.soc_batt_forecast)]; });
Other than this - these charts are solid gold!
Summer
HVAC over summer worked quite well for me as it was all about the cooling and the hottest part of the day, was also the highest PV production, so lowest costs.
I defined an input_number for set point (°C) and another input_number for my cooling coefficient (Cooling W / °C).
I set def_hours_hvac using a weather integration to count how many timesteps the outdoor forecast would exceed my set point.
Then for each timestep, I determined the forecast power consumption by multiplying the difference from the set-point by the cooling coefficient. I.e. if my cooling coefficient is 1500 W/ °C, the set-point is 24 °C and the outdoor temp is 26 °C, then the power forecast for that timestep was 3000 W. I then included this power forecast into my load_forecast array.
This approach is good for measurement, but doesn’t adjust my HVAC load according to cost. I.e. when cost is cheap I want to run HVAC hard and when it is expensive I to switch down consumption.
I kindoff having something working that adjusts p_nom_hvac (which adjusts the maximum allowed value) according to my excess solar production and another automation that changes the HVAC setpoint up and down according to the deferrable load forecast. So using the examples above a deferrable load forecast of 3000W would change the HVAC set point to 24 °C (2 degrees below the external temp), a deferrable load forecast of 4500W would change the set point to 23 °C (3 degrees below the external temp), a deferrable load of 0W would change the set point to match the external temp (24 °C), but then if the external temp goes up to 26 °C, the set point would also go up to 26 °C.
Winter
We are now heading into winter and the problem is reversed as it is now all about heating and the coldest part of the day is generally overnight and just before sunrise, but morning high peak prices are occurring at the same time (as the grid has high demand at this time).
I can see my HVAC heating captured in my load power forecast, running from around 1800-0800 each night at a constant 2 kW, with a spike at 0600 the morning when I switch the thermostat from night (18 °C) to morning (20 °C). The def_end_timestep will be useful here as I can schedule my HVAC to complete by 0800, and then potentially modulate my HVAC setpoint to heat during the cheapest timesteps. But I don’t really need it ramping up the heating at 0300 just because it is the cheapest time overnight, but somehow I want to hit that 20 °C at 0700 and avoid the 0600 morning price peak.
I’m quite interested in how others are optimising for the winter overnight heating problem.
Okay that is intense and i’m going to have to do some thinking. I don’t have a sensor to measure the consumption of HVAC so I’m not even sure how to subtract the HVAC load (anywhere from 2kw to 8kw) from var_load.
I could see a naïve HVAC needs 4kWh (or whatever the calculated spike load will be) between 6 and 7am possibly working. Where the spike loading is the deferred load and the constant 2kw is just ‘normal house’ load?
PS what’s the logic to derive savings vs DMO?
