You can feed a set of interim load_forecast values so you don’t need to be offline for 48 hours.
@davidusb Do you want to put a folder in your github repo to store sample battery/inverter automation scripts so new users has someone to refer to rather than starting from scratch? I’m happy to contribute my Sungrow one.
I used to manipulate the charging power but it turned out to be overthinking it.
If you run mpc-optim frequently (I run it every minute) then I just charge/discharge at full power and when EMHASS decides its reached the desired SoC, it will set the grid_forecast/batt_forecast to 0 and stop the charging/discharging
So instead of discharging at rate of 2000w over 30 min, I discharge at 10,000w and EMHASS will stop the discharge after 6 min.
2 Likes
Your approach may work for discharging but it could cause issues when charging, among others when certain other heavy users are active. The beauty of emhass is that it operates within the constraints you set out.
my growatt inverter expects a % of max. I solve for this by using the emhass generated a discharge/charge in kw. My solution is to divide that over the max discharge to arrive at the percentage - rounded and floored at 5% (by conditioning minimum 400watt)
value: >-
{{((states('sensor.p_batt_forecast') | float(default="0") / 8000 * -100 ) |
round(0) | float(default=0))}}
I run MPC only once per hour. There is no need to run it more frequently with my tariff structure
1 Like
I have created helpers in HA to let me set all additional values like VAT, tax refund, transfer fee etc, then add/subtract these to both purchase and sale prices in the call to emhass optimization.
Something like this. Fetch all the inputs.
Create the nordpool list of prices for the next (up to) 24 hours.
Loop though the list and add the input values to the nordpool values in the list. This uses a rather weird namespace workaround that I don’t really understand, because Homeassistant doesn’t allow modifying list values otherwise… But it does the trick. The nordpool_sell and nordpool_buy lists are then sent to the optimization.
{%- set tax=states('input_number.elpris_skatt_per_kwh_ink_moms')|float %}
{%- set transfer_fee=states('input_number.elpris_vattenfall_overforingsavgift_per_kwh_inkl_moms')|float %}
{%- set extra_fee=states('input_number.elpris_tranas_paslag_per_kwh_ink_moms')|float %}
{%- set tax_return=states('input_number.elpris_skatteaterbaring_per_kwh')|float %}
{%- set extra_sales_fee=states('input_number.elpris_tranas_avdrag_per_kwh_ink_moms')|float %}
{%- set energy_compensation=states('input_number.elpris_vattenfall_energiersattning_per_kwh')|float %}
{%- if state_attr('sensor.nordpool_kwh_se3_sek_0_10_025','tomorrow_valid') == true %}
{%- set nordpool=((state_attr('sensor.nordpool_kwh_se3_sek_0_10_025', 'raw_today') | map(attribute='value') | list
+ state_attr('sensor.nordpool_kwh_se3_sek_0_10_025', 'raw_tomorrow') | map(attribute='value') | list)) [now().hour:][:24] %}
{%- else %}
{%- set nordpool=(state_attr('sensor.nordpool_kwh_se3_sek_0_10_025', 'raw_today')
| map(attribute='value') | list) [now().hour:][:24] %}
{%- endif %}
{%- set nordpool_sell = namespace(values=[]) %}
{%- set nordpool_buy = namespace(values=[]) %
{%- for i in nordpool %}
{%- set nordpool_sell.values=nordpool_sell.values+[((i+energy_compensation-extra_sales_fee)/100)|round(2)]%}
{%- set nordpool_buy.values=nordpool_buy.values+[((i+transfer_fee+extra_fee+tax-tax_return)/100)|round(2)]%}
{%- endfor %}
...
1 Like
Same thing today where emhass again chose to alternately charge at max and then discharge some, even though the energy price was level.
As you can see, the gross energy price (buy and sale) differs by almost nothing and is close to zero. Adding the additional charges and additions, the net price is 0.38 SEK buy and 0.15 SEK sell. Again, I have the battery charge and discharge weights set to 0.5 (SEK).
Still, without any changes to sales price, emhass thinks it should charge and discharge repeatedly for each time slot.
It doesn’t make the least sense to me. Considering I buy at 0.38 SEK, sell at 0.15, and also have a 0.50 SEK this behavior is not the least economical. There’s apparently something wrong with optimization and I suspect the battery charge/discharge weights to be at fault, but I don’t understand the code well enough to know.
I’ve added an issue to github.
What is the status of the “Emhass optimization status” sensor?
Can you share the charge/discharge efficiency you applied and the code for the price add-on? Any other restrictions you included?
Hi, yes of course. Sample automation are very welcome to help people setting this up. Probably some place in the documentation along with the other current examples is a good option to place these.
1 Like
Something else is driving EMHASS to make these optimisations. I have requested additional information from you in the GitHub issue to provide a full picture for debug purposes.
I must be blind but I don’t see where the other current examples are.
There are some examples here for Amber, Nordpool, Solcast forecasts: https://emhass.readthedocs.io/en/latest/forecasts.html#example-using-solcast-forecast-amber-prices
And also here for some example configurations: https://emhass.readthedocs.io/en/latest/study_case.html
With new installs I’m seeing very verbose logging from the solver.
Is this able to be reduced?
2024-05-12 08:45:18,101 - web_server - INFO - Perform an iteration of a naive MPC controller
2024-05-12 08:45:18,357 - web_server - WARNING - Solver default unknown, using default
Welcome to the CBC MILP Solver
Version: 2.10.10
Build Date: Sep 26 2023
command line - /usr/local/lib/python3.11/dist-packages/pulp/solverdir/cbc/linux/arm64/cbc /tmp/605d15421dad456e947c8318c18c8e87-pulp.mps -max -timeMode elapsed -branch -printingOptions all -solution /tmp/605d15421dad456e947c8318c18c8e87-pulp.sol (default strategy 1)
At line 2 NAME MODEL
At line 3 ROWS
At line 437 COLUMNS
At line 4533 RHS
At line 4966 BOUNDS
At line 5357 ENDATA
Problem MODEL has 432 rows, 312 columns and 3822 elements
Coin0008I MODEL read with 0 errors
Option for timeMode changed from cpu to elapsed
Continuous objective value is -3.19511 - 0.01 seconds
Cgl0003I 0 fixed, 0 tightened bounds, 57 strengthened rows, 0 substitutions
Cgl0003I 0 fixed, 0 tightened bounds, 13 strengthened rows, 0 substitutions
Cgl0004I processed model has 267 rows, 234 columns (78 integer (78 of which binary)) and 3498 elements
Cbc0038I Initial state - 22 integers unsatisfied sum - 4.59574
Cbc0038I Pass 1: suminf. 3.25435 (19) obj. -4.51297 iterations 44
Cbc0038I Solution found of -4.51297
Cbc0038I Relaxing continuous gives -4.49083
Cbc0038I Before mini branch and bound, 49 integers at bound fixed and 100 continuous
Cbc0038I Full problem 267 rows 234 columns, reduced to 5 rows 17 columns
Cbc0038I Mini branch and bound improved solution from -4.49083 to -3.19511 (0.10 seconds)
Cbc0038I After 0.11 seconds - Feasibility pump exiting with objective of -3.19511 - took 0.03 seconds
Cbc0012I Integer solution of -3.1951128 found by feasibility pump after 0 iterations and 0 nodes (0.11 seconds)
Cbc0001I Search completed - best objective -3.195112814650332, took 0 iterations and 0 nodes (0.11 seconds)
Cbc0035I Maximum depth 0, 0 variables fixed on reduced cost
Cuts at root node changed objective from -3.19511 to -3.19511
Probing was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Gomory was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Knapsack was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Clique was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
MixedIntegerRounding2 was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
FlowCover was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
TwoMirCuts was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
ZeroHalf was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Result - Optimal solution found
Objective value: -3.19511281
Enumerated nodes: 0
Total iterations: 0
Time (CPU seconds): 0.14
Time (Wallclock seconds): 0.15
Option for printingOptions changed from normal to all
Total time (CPU seconds): 0.15 (Wallclock seconds): 0.16
2024-05-12 08:45:19,278 - web_server - INFO - Status: Optimal
2024-05-12 08:45:19,306 - web_server - INFO - Total value of the Cost function = -3.20
2024-05-12 08:45:21,830 - web_server - INFO - Passed runtime parameters: {}
2024-05-12 08:45:21,831 - web_server - INFO - >> Setting input data dict
2024-05-12 08:45:21,831 - web_server - INFO - Setting up needed data
2024-05-12 08:45:21,844 - web_server - INFO - >> Publishing data...
2024-05-12 08:45:21,846 - web_server - INFO - Publishing data to HASS instance
Yes we could set msg=False to suppress all logs from the solver.
However these messages could be useful for debugging.
Probably a good idea would be to only show these messages if we set the EMHASS logger to DEBUG. It can be done of course, its just that our logger and the solver logger are two separate instances, so this needs some rework.
1 Like
Just released a new version v0.9.0: https://github.com/davidusb-geek/emhass-add-on/releases/tag/v0.9.0
Thanks to @gieljnssns and @GeoDerp for the hard work!
@gieljnssns has proposed a new class within EMHASS to do regression using a variety of Scikit-Learn methods. This is useful to do machine learning with any sensors that you want and then generate predictions from these regressions. That data can be used to set some parameters for the EMHASS optimization. For example, @gieljnssns is using a regression method to obtain a prediction of the def_total_hours needed for its deferrable loads.
Here is our complete discussion that we had on this topic: https://github.com/davidusb-geek/emhass/pull/147
But more generally and out of EMHASS, this feature can be used in HA to do machine learning with virtually any sensor with enough history data.
This comes as a complement of the already existent machine learning forecaster, but with the difference that this is just regression using known dependent variables. Whereas the forecaster method generates predictions on future timestamps.
This new features comes with fit and predit end points, following the sklearn convention. In the future we might separate a tune method for hyperaparameter optimization of these machine learning models.
2 Likes
Just updated to v0.9.0
Getting interesting behaviour where EMHASS decide not to sell back to grid. Purchase at $0.02, next day is $0.02c. Evening is $0.3, but no sale. Previous version would sell back to grid.
Was there a change in logic in V0.9.0 or something at miss on my end?
Odd,
My update to 0.9.0 still wants to sell tonight, maybe relax your battery dis/charge weights?
Your overnight forecast load looks quite high at 4 kW, perhaps it is holding back to cover that? But you still have 50% SOC at the end of the day so it should be able to export at least some.
Relaxed setting as much as possible, still same.
But, 24 hours ago I tested the system, run it for 30 min in “self consume” setting under EMHAS three option menu.
Maybe system got locked in this mode for some reason. I will wait for 48 hours, it may reset just fine.
After update I get error every 5 minutes now
2024-05-13 13:05:00,315 - web_server - INFO - Passed runtime parameters: {'load_cost_forecast': [13.2835, 13.4139, 13.5792, 20.8349, 24.0722, 27.4345, 27.8055, 24.2937, 22.9401, 20.3484], 'prod_price_forecast': [-1.015, -0.892, -0.736, 6.109, 9.163, 12.335, 12.685, 9.372, 8.095, 5.65], 'prediction_horizon': 10, 'alpha': 1, 'beta': 0, 'soc_init': 0.68, 'soc_final': 0.12}
2024-05-13 13:05:00,316 - web_server - INFO - >> Setting input data dict
2024-05-13 13:05:00,316 - web_server - INFO - Setting up needed data
2024-05-13 13:05:00,322 - web_server - INFO - Retrieve hass get data method initiated...
2024-05-13 13:05:00,628 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2024-05-13 13:05:01,462 - web_server - ERROR - Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1473, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 882, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 880, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 109, in action_call
input_data_dict = set_input_data_dict(emhass_conf, costfun,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 164, in set_input_data_dict
P_PV_forecast = fcst.get_power_from_weather(df_weather, set_mix_forecast=True, df_now=df_input_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 420, in get_power_from_weather
cec_modules = bz2.BZ2File(self.emhass_conf['root_path'] / 'src/emhass/data/cec_modules.pbz2', "rb")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/bz2.py", line 81, in __init__
self._fp = _builtin_open(filename, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/app/src/emhass/data/cec_modules.pbz2'
2024-05-13 13:05:01,472 - web_server - INFO - Passed runtime parameters: {}
2024-05-13 13:05:01,472 - web_server - INFO - >> Setting input data dict
2024-05-13 13:05:01,472 - web_server - INFO - Setting up needed data
2024-05-13 13:05:01,473 - web_server - INFO - >> Publishing data...
2024-05-13 13:05:01,473 - web_server - INFO - Publishing data to HASS instance
2024-05-13 13:05:01,481 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 3817.63
2024-05-13 13:05:01,484 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 321.17
2024-05-13 13:05:01,486 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 2000.0
2024-05-13 13:05:01,489 - web_server - INFO - Successfully posted to sensor.p_deferrable1 = 0.0
2024-05-13 13:05:01,491 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = -1496.46
2024-05-13 13:05:01,494 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 76.18
2024-05-13 13:05:01,497 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 0.0
2024-05-13 13:05:01,500 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = 122.29
2024-05-13 13:05:01,502 - web_server - INFO - Successfully posted to sensor.optim_status = Optimal
2024-05-13 13:05:01,505 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 13.293
2024-05-13 13:05:01,507 - web_server - INFO - Successfully posted to sensor.unit_prod_price = -1.006
And thus no updates on any of the calculations, think I need to rollback to v0.8.6, is that easy to do? Need some help here
EDIT:Rolled back to v0.8.6 and is working again.
Great! And sorry to hear about your issue. I did tested everything before releasing the new version. This missing file was not an issue when I tested, it was able to find that file without issue. We will check this. Anyone else is having this issue? Probably try to uninstall, erase build files and then rebuild?
Could you please share how did you rolled back to a previous version. It may help other people.
We have a procedure to roll back to a previous version manually using the local add-on functionality. The procedure is decribed here: https://github.com/davidusb-geek/emhass-add-on/blob/dabeaaaf71074367390bb9d7906b40dee9281193/Test.md
