Can you share your config? Looks good!
Apex chart card
type: custom:apexcharts-card
span:
start: minute
header:
show: true
title: EMHASS Daily Forecast
show_states: true
colorize_states: true
now:
show: false
label: now
yaxis:
- min: -7000
max: 13000
decimals: 2
apex_config:
forceNiceScale: true
tick_amount: 4
series:
- entity: sensor.p_pv_forecast
curve: stepline
stroke_width: 2
color: orange
show:
in_header: false
legend_value: false
data_generator: |
return entity.attributes.forecasts.map((entry) => {
return [new Date(entry.date), entry.p_pv_forecast];
});
- entity: sensor.p_load_forecast
curve: stepline
type: line
color: purple
show:
in_header: false
legend_value: false
stroke_width: 1
data_generator: |
return entity.attributes.forecasts.map((entry) => {
return [new Date(entry.date), entry.p_load_forecast];
});
- entity: sensor.p_batt_forecast
curve: stepline
color: blue
show:
in_header: false
legend_value: false
stroke_width: 1
type: area
data_generator: |
return entity.attributes.battery_scheduled_power.map((entry) => {
return [new Date(entry.date), entry.p_batt_forecast];
});
- entity: sensor.p_grid_forecast
curve: stepline
color: teal
type: area
show:
in_header: false
legend_value: false
stroke_width: 1
data_generator: |
return entity.attributes.forecasts.map((entry) => {
return [new Date(entry.date), entry.p_grid_forecast];
});
- entity: sensor.p_deferrable0
curve: stepline
name: Pool Pump
show:
in_header: false
legend_value: false
stroke_width: 1
data_generator: |
return entity.attributes.deferrables_schedule.map((entry) => {
return [new Date(entry.date), entry.p_deferrable0];
});
- entity: sensor.p_deferrable1
curve: stepline
name: Pool Heater
show:
in_header: false
legend_value: false
stroke_width: 1
data_generator: |
return entity.attributes.deferrables_schedule.map((entry) => {
return [new Date(entry.date), entry.p_deferrable1];
});
- entity: sensor.p_deferrable2
curve: stepline
name: Car
show:
in_header: false
legend_value: false
stroke_width: 1
data_generator: |
return entity.attributes.deferrables_schedule.map((entry) => {
return [new Date(entry.date), entry.p_deferrable2];
});
- entity: sensor.p_deferrable3
curve: stepline
name: Heating/ Cooling
color: red
show:
in_header: false
legend_value: false
stroke_width: 1
data_generator: |
return entity.attributes.deferrables_schedule.map((entry) => {
return [new Date(entry.date), entry.p_deferrable3];
});
- entity: sensor.p_deferrable4
curve: stepline
name: Hot Water
show:
in_header: false
legend_value: false
stroke_width: 1
data_generator: |
return entity.attributes.deferrables_schedule.map((entry) => {
return [new Date(entry.date), entry.p_deferrable4];
});
- entity: sensor.total_cost_fun_value
unit: $
name: Plan Value (+ve credit)
show:
legend_value: false
in_chart: false
- entity: sensor.pv_forecast_energy
unit: kWh
name: Solar Production Forecast
show:
legend_value: false
in_chart: false
view_layout:
position: main
type: custom:apexcharts-card
experimental:
color_threshold: true
span:
start: minute
header:
show: true
title: Battery, Price & Cost Forecast
show_states: true
colorize_states: true
series:
- entity: sensor.amber_general_forecast
float_precision: 2
yaxis_id: first
show:
in_header: after_now
legend_value: false
stroke_width: 1
color: red
color_threshold:
- value: 0
color: cyan
- value: 0.19
color: green
- value: 0.3
color: yellow
- value: 0.4
color: red
name: general cost
data_generator: |
return entity.attributes.forecasts.map((entry) => {
return [new Date(entry.start_time), entry.per_kwh];
});
- entity: sensor.amber_feed_in_forecast
name: feedin min
float_precision: 2
yaxis_id: first
show:
in_header: after_now
legend_value: false
color: cyan
stroke_width: 1
data_generator: |
return entity.attributes.forecasts.map((entry) => {
return [new Date(entry.start_time), entry.range_min];
});
- entity: sensor.amber_feed_in_forecast
float_precision: 2
yaxis_id: first
show:
in_header: after_now
legend_value: false
color: blue
stroke_width: 1
name: feed in price
data_generator: |
return entity.attributes.forecasts.map((entry) => {
return [new Date(entry.start_time), entry.per_kwh];
});
- entity: sensor.soc_batt_forecast
float_precision: 2
yaxis_id: second
show:
in_header: before_now
legend_value: false
color: green
stroke_width: 2
name: battery SOC
data_generator: |
return entity.attributes.battery_scheduled_power.map((entry) => {
return [new Date(entry.date), entry.soc_batt_forecast]; });
yaxis:
- id: first
decimals: 1
max: 1
min: 0
apex_config:
tickAmount: 2
logarithmic: false
- id: second
show: false
opposite: false
decimals: 1
max: 100
min: 0
apex_config:
tickAmount: 2
view_layout:
position: main
3 Likes
Mabey this function could be added to emhass to easier improve this on our systems. When you look at your forcast you are also going to sell before charing your battery because of the energy prices it is beeng delayed. but if this is not configured to the inverter it will charge here. and I think its not correct to add this with a deferable load as we want it on most of the time just only delay when lots of pv.
Sure, to implement energy management strategies, we’ll have to compare the forecasted battery power (p_batt_forecast) with the energy price (raw_today). For this example, I’ll only consider a few instances for clarity. Please note, negative p_batt_forecast values imply the battery is charging, and positive values mean it’s discharging.
-
At 10:00:
-
p_batt_forecast= -380.0 W (battery charging) -
raw_today= 0.831 (low energy price) - In this case, charging the battery when energy prices are low aligns with an optimal strategy.
-
-
At 13:00:
-
p_batt_forecast= -3242.0 W (battery charging) -
raw_today= 0.708 (low energy price) - Again, this is optimal as the battery charges when energy prices are low.
-
-
At 17:00:
-
p_batt_forecast= 354.0 W (battery discharging) -
raw_today= 0.968 (rising energy price) - This is beneficial too as the battery discharges (or sells back to the grid) when energy prices are higher.
-
-
At 21:00:
-
p_batt_forecast= 989.0 W (battery discharging) -
raw_today= 1.057 (high energy price) - Again, discharging when prices are high is an optimal strategy.
-
These are simplified examples. A comprehensive analysis should consider the battery’s state of charge, energy demand, PV production, and grid reliability, among other factors. Also, the decision-making process can be complex if you factor in peak demand charges, battery wear and tear due to frequent charging/discharging, etc. But the primary goal should be to charge when energy is cheap (and/or abundant from PV) and discharge when it’s expensive or when the household load is high.
Electric power (P) is given by the product of voltage (V) and current (I), as per the formula:
P = V * I
If you know the power (in Watts) and the voltage (in Volts), you can calculate the current (in Amperes) by rearranging the formula:
I = P / V
Assuming a voltage of 230V.
Let’s calculate the charging current for a few time points:
-
At 10:00:
-
p_batt_forecast= -380.0 W - I = P / V = -380 W / 230 V = -1.65 A
-
-
At 13:00:
-
p_batt_forecast= -3242.0 W - I = P / V = -3242 W / 230 V = -14.09 A
-
-
At 15:00:
-
p_batt_forecast= -493.0 W - I = P / V = -493 W / 230 V = -2.14 A
-
The negative sign denotes that the current is flowing into the battery (charging).
Remember that these calculations assume a perfect power conversion efficiency and do not account for any power conversion losses or inefficiencies in the battery charging process. Also, the amperage should not exceed the battery manufacturer’s recommended charging current to avoid damaging the battery.
A Home Assistant template could look something like this:
template:
- sensor:
- name: "Charging Amps"
unit_of_measurement: "A"
state: "{{ (states('sensor.p_batt_forecast') | float(0) / 230) | round(2) }}"
This sensor will take the value of sensor.p_batt_forecast, convert it to a float, divide by 230 (for 230V service), and then round the result to two decimal places. The result is the estimated charging current in Amperes.
Negative values mean charging while positive values mean discharging. You might want to handle these cases differently depending on your use case.
Just got back to trying the ML load forecaster. I am trying to use the model-predict endpoint and getting an error related to not enough data.
2023-05-21 18:11:36,069 - web_server - INFO - Setting up needed data
2023-05-21 18:11:36,071 - web_server - INFO - Retrieve hass get data method initiated...
2023-05-21 18:11:36,081 - web_server - ERROR - The retrieved JSON is empty, check that correct day or variable names are passed
2023-05-21 18:11:36,081 - web_server - ERROR - Either the names of the passed variables are not correct or days_to_retrieve is larger than the recorded history of your sensor (check your recorder settings)
2023-05-21 18:11:36,081 - web_server - ERROR - Exception on /action/forecast-model-predict [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2190, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1486, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 170, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 146, in set_input_data_dict
rh.get_data(days_list, var_list)
File "/usr/local/lib/python3.9/dist-packages/emhass/retrieve_hass.py", line 147, in get_data
self.df_final = pd.concat([self.df_final, df_day], axis=0)
UnboundLocalError: local variable 'df_day' referenced before assignment
Here is the call I am making to the endpoint:
forecast_model_predict: 'curl -i -H "Content-Type:application/json" -X POST -d
''{"model_type": "load_forecast",
"model_predict_publish": "True",
"model_predict_entity_id": "sensor.p_load_forecast_custom_model",
"model_predict_unit_of_measurement": "W",
"model_predict_friendly_name": "Load Power Forecast custom ML model"}'' http://localhost:5000/action/forecast-model-predict'
What am I missing? I am able to run the ‘fit’ and ‘backtest’ steps.
Ignore me, I was missing:
"var_model": "sensor.load_with_no_var_loads", "days_to_retrieve": 15
It was not clear from the docs that this need to be provided for this endpoint.
hi David and the rest
getting an error “no file found” after I did “Perfect Optimization” and then press “publish …” in the GUI.
Am i still missing anything, that i missed in the Docs?
2023-05-23 10:17:20,018 - web_server - INFO - Setting up needed data
2023-05-23 10:17:20,020 - web_server - INFO - Retrieve hass get data method initiated...
2023-05-23 10:17:24,099 - web_server - INFO - >> Performing perfect optimization...
2023-05-23 10:17:24,100 - web_server - INFO - Performing perfect forecast optimization
2023-05-23 10:17:24,102 - web_server - INFO - Perform optimization for perfect forecast scenario
2023-05-23 10:17:24,103 - web_server - INFO - Solving for day: 21-5-2023
2023-05-23 10:17:24,157 - web_server - INFO - Status: Optimal
2023-05-23 10:17:24,158 - web_server - INFO - Total value of the Cost function = -542.0
2023-05-23 10:17:24,165 - web_server - INFO - Solving for day: 22-5-2023
2023-05-23 10:17:24,220 - web_server - INFO - Status: Optimal
2023-05-23 10:17:24,220 - web_server - INFO - Total value of the Cost function = -522.21
2023-05-23 10:17:28,847 - web_server - INFO - Setting up needed data
2023-05-23 10:17:28,849 - web_server - INFO - >> Publishing data...
2023-05-23 10:17:28,849 - web_server - INFO - Publishing data to HASS instance
2023-05-23 10:17:28,849 - web_server - ERROR - File not found error, run an optimization task first.
For some reason the optimization result was not saved. That’s weird.
Do you have the same problem with the day-ahead optimization?
The “perfect optimization” is using past historical values, thus “perfect” forecasts data, to perform the optimization. It is there just as a reference to evaluate the performance of a day-ahead optimization.
recorder was not saving the data so i have to wait for that
2023-05-23 11:02:40,689 - web_server - INFO - Setting up needed data
2023-05-23 11:02:40,690 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-05-23 11:02:41,535 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-05-23 11:02:41,536 - web_server - INFO - Retrieve hass get data method initiated...
2023-05-23 11:02:41,545 - web_server - ERROR - The retrieved JSON is empty, check that correct day or variable names are passed
2023-05-23 11:02:41,545 - web_server - ERROR - Either the names of the passed variables are not correct or days_to_retrieve is larger than the recorded history of your sensor (check your recorder settings)
2023-05-23 11:02:41,545 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2190, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1486, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 177, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 91, in set_input_data_dict
P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'])
File "/usr/local/lib/python3.9/dist-packages/emhass/forecast.py", line 585, in get_load_forecast
rh.get_data(days_list, var_list)
File "/usr/local/lib/python3.9/dist-packages/emhass/retrieve_hass.py", line 147, in get_data
self.df_final = pd.concat([self.df_final, df_day], axis=0)
UnboundLocalError: local variable 'df_day' referenced before assignment
also here’s my config file, if it could help
hass_url: empty
long_lived_token: empty
costfun: self-consumption
optimization_time_step: 30
historic_days_to_retrieve: 2
method_ts_round: nearest
set_total_pv_sell: false
lp_solver: COIN_CMD
lp_solver_path: /usr/bin/cbc
set_nocharge_from_grid: true
load_forecast_method: naive
sensor_power_photovoltaics: sensor.inverter_active_power
sensor_power_load_no_var_loads: sensor.power_load_no_var_loads
number_of_deferrable_loads: 2
list_nominal_power_of_deferrable_loads:
- nominal_power_of_deferrable_loads: 2200
- nominal_power_of_deferrable_loads: 750
list_operating_hours_of_each_deferrable_load:
- operating_hours_of_each_deferrable_load: 3
- operating_hours_of_each_deferrable_load: 2
list_peak_hours_periods_start_hours:
- peak_hours_periods_start_hours: "02:54"
- peak_hours_periods_start_hours: "17:24"
list_peak_hours_periods_end_hours:
- peak_hours_periods_end_hours: "15:24"
- peak_hours_periods_end_hours: "20:24"
list_treat_deferrable_load_as_semi_cont:
- treat_deferrable_load_as_semi_cont: true
- treat_deferrable_load_as_semi_cont: true
load_peak_hours_cost: 0.1644
load_offpeak_hours_cost: 0.1644
photovoltaic_production_sell_price: 0.11
maximum_power_from_grid: 9200
list_pv_module_model:
- pv_module_model: Canadian_Solar_Inc__CS1U_410MS
list_pv_inverter_model:
- pv_inverter_model: Huawei_Technologies_Co___Ltd___SUN2000_5KTL_USL0__240V_
list_surface_tilt:
- surface_tilt: 30
list_surface_azimuth:
- surface_azimuth: 205
list_modules_per_string:
- modules_per_string: 16
list_strings_per_inverter:
- strings_per_inverter: 1
set_use_battery: false
battery_discharge_power_max: 1000
battery_charge_power_max: 1000
battery_discharge_efficiency: 0.95
battery_charge_efficiency: 0.95
battery_nominal_energy_capacity: 5000
battery_minimum_state_of_charge: 0.3
battery_maximum_state_of_charge: 0.9
battery_target_state_of_charge: 0.6
Ok that’s better information with those logs. There seems to be nothing wrong with your config. You just have to wait for the recorder to fill that database with a little bit more data.
1 Like
Reading through this thread it looks like you currently don’t have temperature as an input into the predictive load model, is that right?
Would this do anything different during a power outage or is there a way to input model parameters to “simulate” an outage like making all energy importing and exporting infinity? I am looking for something like this to optimize my energy usage during power outages
the skforecast package that the emhass docs mention is used for prediction has exogenous variables so it seems like temperature could easily be added into energy use forecasts
How do you do long term history for only sensor.power_load_no_var_loads?
yes. but is this the recommended way to do this? Because I do not want to fill the batteries in the most expensive hours if there is calculated to be more awailable pv later that day it would be smarter to just move the charging untill then an export instead earlier. If I was to follow this from emhass on every thing would not be smart since the inverters it self are very smart but do not know of nordpool prices every day.
That is right, it is not yet possible to provide additional exogenous data to train the model.
The model is trained based on previous load power data.
However I’m completely aware that we can use exogenous data with skforecast. I’m actually using exogenous data but building it inside EMHASS. I’m creating new features based on the future timestamps. That is the thing with the exogenous data, that is should be known in the future. So for what you propose that will be the forecast time series of the temperature. And it is not always easy to treat and manage future time series with Home Assistant. But still this is something that I will integrate in the future, the possibility to provide additional exogenous data for forecasting, but you will need to provide the know future values for that.
For power outage do you think that there is any room for optimization? I mean in the case of a power outage you are on a survival mode, trying to provide your house load with any available power production means. In that case I think that you may have to let your grid-forming-capable inverter to deal with your system power management.
Use the recorder settings and use the include option for specific entities.
See: Recorder - Home Assistant
If you are stating all this then I don’t think that you understand the goal of EMHASS.
The results from EMHASS will provide you with best “smartest” optimal operating conditions that you will need to minimize your monthly energy bill for example. Of course this is assuming that the provided forecasts are fairly good.
Then you can use that information and control your inverter accordingly.
Like we have discussed many times previously on this thread is that the “smartest” control is not necessary to blindly operate your battery and your deferrable loads based on if your PV is producing lots of power or not. It is much more complicated than that because of the many factors involved and there is where EMHASS comes in.
1 Like
For a power outage I’d imagine you could look at the problem like there are a few types of loads
- Base load (fridge, networking, etc). Small loads that are unlikely to be turned off
- Nice to have loads in order of decreasing necessity, like HVAC, water heater, oven, washer/dryer, pool heater etc.
Then I think the problem becomes
What is the predicted battery SOC over time once (1) is subtracted out and PV forecast is added in.
How much energy from (2) can be used and how is the energy distributed over time to the deferred loads if an outage is expected to last X days.
An example would be that if when you add the PV forecasted and the base load and it causes the battery SOC to ever exceed 100% you would want to use that excess energy rather than curtailing. You could predict how much energy the HVAC system would use based on the outdoor temperature. If there is enough energy to maintain a reasonable temperature then move onto the next load etc.
Because the energy from 2 is variable this seems like it might break a lot of the assumptions from your optimization and not share too much in common