Could you re-phrase this please? What is higher than 0?
Iām getting this error.
Gone from 23 elements down to 4 during the day
ValueError: Length mismatch: Expected axis has 4 elements, new values have 24 elements
Anybody know how to solve it?
2023-03-26 21:24:38,516 - web_server - INFO - Setting up needed data
2023-03-26 21:24:38,520 - web_server - ERROR - ERROR: The passed data is either not a list or the length is not correct, length should be 24
2023-03-26 21:24:38,521 - web_server - ERROR - Passed type is <class 'list'> and length is 24
2023-03-26 21:24:38,521 - web_server - ERROR - ERROR: The passed data is either not a list or the length is not correct, length should be 24
2023-03-26 21:24:38,522 - web_server - ERROR - Passed type is <class 'list'> and length is 24
2023-03-26 21:24:38,529 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-03-26 21:24:45,839 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-03-26 21:24:45,842 - web_server - INFO - Retrieve hass get data method initiated...
2023-03-26 21:24:49,481 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2528, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 170, in action_call
input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 91, in set_input_data_dict
P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'])
File "/usr/local/lib/python3.9/dist-packages/emhass/forecast.py", line 597, in get_load_forecast
forecast_out.index = self.forecast_dates
File "/usr/local/lib/python3.9/dist-packages/pandas/core/generic.py", line 5596, in __setattr__
return object.__setattr__(self, name, value)
File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
File "/usr/local/lib/python3.9/dist-packages/pandas/core/generic.py", line 769, in _set_axis
self._mgr.set_axis(axis, labels)
File "/usr/local/lib/python3.9/dist-packages/pandas/core/internals/managers.py", line 214, in set_axis
self._validate_set_axis(axis, new_labels)
File "/usr/local/lib/python3.9/dist-packages/pandas/core/internals/base.py", line 69, in _validate_set_axis
raise ValueError(
ValueError: Length mismatch: Expected axis has 4 elements, new values have 24 elements
If I set the hours a deferrable load should run to higher then 0 (fot example with the command def_total_hoursā:[0.59]), the status is infeasinle and those high values in P_grid_pos and _neg appear. If I do def_total_hoursā:[0.0], everything is working fine.
Iām pretty sure def_total_hours should be an integer. I.e. [0] or [2].
Are you setting it to a float value?
Hi @markpurcell you are right! This was the root cause of status being infeasible. Thanks a lot!
@davidusb while the status is now optimal, the selection of times when to run the deferrable load is not:
As you can see it still runs the deferrable load when the unit_load_cost is low. At first this seems to be correct, but as it would be wiser to put every produced kwh back to the grid (getting 0.26836 EUR per kwh) and running the deferrable load later, for example where unit_load_cost is at 0.224173.
Thanks!
Mmm Iām not so sureā¦
In that table you have all what you need to check that optimal solution. When in doubt I invite you to just check the results using that very same table.
I will do it for you this time.
Letās focus on just two time slots to simplify: the one at 12:30 and the other at 18:00.
At 12:30 the load cost is 0.1965 and the production sell price is 0.268.
At 18:00 the load cost is 0.2242 and the production sell price is still 0.268.
Ok letās define two options for schedules:
- Option #1: Is the optimal schedule obtained with EMHASS which is 643 Watts of deferrable load at 12:30 and no deferrable for 18:00.
- Option #2: The eventual schedule that you are proposing thinking that will be better.
Now letās compute the costs using the cost function that you are which seems to be āprofitā.
This cost function is defined by this equation:
f_cost = -0.001*delta_t*(load_cost*(p_load+p_def-p_pv)+prod_price*pgrid_neg)
Ok in your case you have 30 min time step so delta_t=0.5. If we agree will all this then letās just compute the costs for each option and each of the time slots shall we?
- Option #1 @ 12:30:
f_cost = -0.001*0.5*(0.1965*(1889+643-569)) = -0.1929(which is the result shown in the table) - Option #1 @ 18:00:
f_cost = -0.001*0.5*(0.2242*(927-98.5)) = -0.0929(again this is the result shown in the table) - Option #2 @ 12:30:
f_cost = -0.001*0.5*(0.1965*(1889-569)) = -0.1297 - Option #2 @ 18:00:
f_cost = -0.001*0.5*(0.2242*(927+643-98.5)) = -0.165
Now the final computation is the sum of these costs (in the real calculation we will sum the values for a complete day):
- Total cost Option #1:
total_cost_option1 = -0.1929-0.0929 = -0.2858 - Total cost Option #2:
total_cost_option2 = -0.1297-0.165 = -0.2947
As you can see the option obtained with EMHASS is the correct option which maximizes this āprofitā cost function because: total_cost_option1 > total_cost_option2.
The beauty of the Linear Programming formulation is that this optimization is done automatically considering not just the cost function but also a good number of other constraints and all that in just some seconds.
2 Likes
Hi!
Thanks for your detailed explanation.
But from what I see, option1 does not take into consideration the used PV power which would have been sold at a higher price if it wasnāt used for the deferrable load.
I donāt know if itās different elsewhere, but here in Austria and from what I know in most other European countries, you first use the power from your pv. If thatās not sufficient to cover your power usage, it would use some additional power or the complete power from the grid.
But we can not decide to simultaneously feed back to grid and consume the energy.
If there is more pv_power available then used, while the prod_sell_price is higher than the load_cost (during the whole day), it would always be wise to sell every bit of pv_power and later use energy from the grid.
There might be better days to show this (where there is more pv production and EMHASS still deciding to use the (if not used) high priced energy which would have been put back to the grid) but itās cloudy today.
It is considered in this self-consumption approach. The thing is that in the example given for the time slot at 12:30 you just donāt have enough PV to become a net producer.
This is exactly what is going on the detailed calculation I gave you and it is consistent with the results.
We are NOT deciding that. This is actually forbidded by one of the main constraints.
I managed to demonstrate that it is not wise 
But of course any exceeded net PV production is counted and considered using the sell price.
Hi, thanks for this amazing project.
I have a house with 4kW PV and a lot of (dumb) devices that I want to test this in. I am just trying to understand some things from the documentation, so I hope you donāt mind some dumb questions.
- The LP page talks about cost functions. In particular, you talk about deferrable loads. What is the difference between
P_loadandP_defSum? The docs sayP_loadis āthe electricity load consumption (positive defined)ā, but I canāt understand what that means. - What exactly do you consider a deferrable load? I understand that a load at a certain index can be moved backwards or forwards in time to optimize some cost function. But how does that work for loads that run a program, for example a washing machine that runs a certain washing program? I read something about ānumber of start-upsā, is that related to my question?
- I read that for the PV forecast you scrape a webpage. Do you think it would have any value to do the forecasting locally? Considering we already capture a lot of PV and weather data it would seem logical to build a model of our own PV plant rather than using some grid based forecasting method.
And lastly:
- Do you need any help, any cool things on the wishlist? I am a MSc student EE and I did a lot of stuff related to this project, specifically on smart grids and (non-linear) optimization. I will follow a course on MPC next quartile so I hope to learn more about that specific part then. I could help with testing or perhaps adding some new features. I have a lot of experience with python but no experience developing for HA, but Iām sure it can be learned.
1 Like
How have others solved it with charging of ev? Since it has to be calculated how much energy needed to be added and current that would be recommended to charge with? Since I only run the day ahead optimization once a day its would not work out of the box?
P_load is your house electric power consumption in Watts but without the power in Watts of the deferrable loads. P_load in this context is considered as always a positive value (no negative values). P_defSum is the sum of all your deferrable loads in Watts.
A deferrable load is a load that you can manage to turn on/off at your will. It is a load that you can control with a switch or something similar. For loads like a washing machine is different because we donāt have complete control. But still you can estimate its consumption and use the results from EMHASS and manually activate your machine.
I donāt think that it is a good idea to go fully local with PV forecasts. EMHASS supports not only the web scrap but also Solar.Forecast and Solcast. All these are proved methods that work really well. What could be interesting is to use your local data to try to improve those forecasts and adapt them to your local conditions for more precision.
Pull requests are always welcomed. If you have any new idea we can discuss it on github and probably some proof of concept is a good idea.
The āroadmapā is here: Home Ā· davidusb-geek/emhass Wiki Ā· GitHub
No need to learn to ādevelopā anything in Home Assistant because EMHASS is pure Python and the connection to Home Assistant to retrieve data for example is robust and directly uses the official API, so nothing to do there.
1 Like
Hi,
Iām using EMHASS successfully to charge the EV, there should be no issue with running day ahead. (deferrable2 in my case)
I use the following template to calculate the run hours:
- name: def_total_hours_ev
state: "{{is_state('automation.p_deferable2_automation','on')|abs
* (is_state('device_tracker.duka_location_tracker','home')|abs)
* (is_state('binary_sensor.duka_charger', 'on') | abs)
* (((states('number.duka_charge_limit')|int(0) - states('sensor.duka_battery')|int(0))/10+0.9)|int(0))|int(0)}}"
state: "{{is_state('automation.p_deferable2_automation','on')|abs
Is the EV charging automation turned on, I turn off when I donāt want EMHASS to schedule.
* (is_state('device_tracker.duka_location_tracker','home')|abs)
Only calculate charging hours if the car is home.
* (is_state('binary_sensor.duka_charger', 'on') | abs)
Only calculate charging hours if the car is physically plugged into the charger.
* (((states('number.duka_charge_limit')|int(0) - states('sensor.duka_battery')|int(0))/10+0.9)|int(0))|int(0)}}"
Calculate the number of hours for the car charger to get from current SOC to the charge limit set in the car.
I then have an automation for deferrable 2 that every time the deferable 2 forecast changes, I update the charging amps on the car charger.
alias: p_deferrable2 automation (EV)
description: EV charging
trigger:
- platform: state
entity_id:
- sensor.p_deferrable2
condition:
- condition: zone
entity_id: device_tracker.duka_location_tracker
zone: zone.home
- condition: state
entity_id: binary_sensor.duka_charger
state: "on"
action:
- service: number.set_value
data:
value: >-
{{(states('sensor.p_deferrable2')|int(0) /
(state_attr('sensor.duka_charger_power','charger_phases')*3/2)|int(1)
/
max(220,state_attr('sensor.duka_charger_power','charger_volts')|int(230)))|int(0)}}
target:
entity_id: number.duka_charging_amps
mode: single
1 Like
thanks, but my qestion. if we will be leaving in 7am we would like the charge to be completed here. So if we have lots of pv later that thay it would be prefeared not to be charged withing the time needed to be completed.?
Nice set of automations based on the EMHASS optimized schedule 
1 Like
The day ahead optimisation will provide a plan that will charge your car over a 24 hour period using the cheapest electricity you have access to.
I currently have the charge limit in my car set to 50% during the week as this covers my daily commute and then on the weekends I set to 90% as energy is a lot cheaper. If I have a big drive coming up Iāll set the charge limit to 90% two or three days in advance to ensure the car fills up.
One limitation of the day ahead optimisation is it doesnāt know about changes during its optimisation window. So for example it will not know if your car was out for a drive during a planned charging time. For this reason I run the MPC optimisation frequently (every minute) so my EV charging plan includes precise details including if the car is plugged in and precise values for costs, solar production and household load.
Currently EMHASS doesnāt allow scheduling for a 7am departure, but maybe that is something that could be considered for the future?
I have installed EMHASS since some weeks, and my setup is:
- PV 4kw
- 1x deferrable load (water resistance)
- regular price/ kw (no difference between night/ day/ weekend)
- no selling of eletricitcity to the grid
I have the following results for a typical day from EMHASS:
And have a ādumbā system where, if PV > house consumption for > 2000w, turn the resistance ON
I am assessing replacing my dumb system with EMHASS, so I created a virtual switch āEMHASSā which Iām comparing with my dumb āSHELLYā which I am also comparing:
Snag with EMHASS is that it is turning ON in times of the day where I donāt have PV, meaning it is not an wise suggestion to turn the resistance ON based on EMHASS.
Am I missing any important variable in my setup to get to better results?
You are using the default value of prod price of 0.065ā¬/kWh.
Just to be clear, you are injecting your excess PV for free? If that is the case then set that prod price to zero.
1 Like
Indeed, well spotted! Thanks for checking this 
days_to_retrieve: the total days to retrieve from Home Assistant for model training. Defined this in order to retrieve as much history data as possible.
If I want to store the sensordata from āsensor.power_load_no_var_loadsā longer than 15 days. Can I use Home Assistant long-term statistics for this (Long- and short-term statistics | Home Assistant Data Science Portal)?
I use for the moment purge_keep_days: 40 in the recorder setting and the home-assistant_v2.db size is now 1,211.36 MB which is to big a file for me.
recorder:
purge_keep_days: 40
Here is the template I use for getting long-term statistics.
- name: "Power load no var loads"
unique_id: fbfeef21-1aa3-4a54-b781-426f46fef597
unit_of_measurement: kWh
device_class: energy
state_class: total_increasing
state: >
{% set powerload = states('sensor.power_adresse') | float(default=0) %}
{% set vvb = states('sensor.bryter_varmvannsbereder_electric_consumption_w') | float(default=0) %}
{% set varmekabler = states('sensor.power_varmekabler') | float(default=0) %}
{% set varmepumpe = states('sensor.strombryter_varmepumpe_electric_consumption_w') | float(default=0) %}
{% set value = ( powerload - vvb - varmekabler - varmepumpe) | round(1,default=0) %}
{{ value }}
No, that wonāt work. We need the history values of the exact same sensor without any transformation.
The only solution that I see is to increase the purge_keep_days on the recorder. In my case I fixed it to 365 and the database doesnāt grows bigger than 4Gb, which is ok for me since Iām running HA on a mini PC with 256Gb hdd.
In the recorder settings you can state which sensors are to be stored, thatās what I do to keep the database from growing too much.