WattWise - Energy forecasting and battery control for dynamic energy tariffs like Tibber

bullitt186 · November 4, 2024, 8:58pm

Over the last days, i worked on WattWise.

WattWise is an AppDaemon application for Home Assistant that intelligently optimizes battery usage based on consumption forecasts, solar production forecasts, and dynamic energy prices. By leveraging historical data and real-time information, it schedules battery charging and discharging actions to minimize energy costs and maximize efficiency.

WattWise WattWise leverages linear programming via Pulp to optimize the charging and discharging schedule the home battery system.

Here’s a screenshot comparing the actual production/consumtion and the actuals.
I have tested it on my system over the last days and am quite happy so far.

I explained the functionality and how to set it up in the repository:
https://github.com/bullitt186/ha-wattwise/

Please note that this is totally work-in-progress without any warranties and i also cannot foresee whether i will be able to maintain this in the long run.
But maybe you want to give it a try in your system and report back your experience and suggestions to improve it further.

jh537 · November 8, 2024, 4:10am

Hi, thanks for the great implementation and the idea. I installed it straight away.

I’m missing the sensor for the actual PV production. Have I overlooked it?

I am new to the HA. How can I insert the map for the diagram display? I have inserted it in Map, but only see a blank sheet.

Unfortunately, I have not yet been able to read out any sensors. Otherwise I have no errors in the log.

2024-11-08 04:00:01.679770 INFO wattwise: Waiting for all forecasts to be ready...
2024-11-08 04:00:01.680267 INFO wattwise: Retrieving energy price forecast.
2024-11-08 04:00:01.680956 INFO wattwise: Energy price forecast data for today is unavailable.
2024-11-08 04:52:13.760766 WARNING HASS: Disconnected from Home Assistant, retrying in 5 seconds
2024-11-08 04:52:18.765860 WARNING HASS: Disconnected from Home Assistant, retrying in 5 seconds
2024-11-08 04:52:23.772586 WARNING HASS: Disconnected from Home Assistant, retrying in 5 seconds
2024-11-08 04:52:28.776945 WARNING HASS: Disconnected from Home Assistant, retrying in 5 seconds
2024-11-08 04:52:33.789269 WARNING HASS: Disconnected from Home Assistant, retrying in 5 seconds
2024-11-08 04:52:38.796155 WARNING HASS: Disconnected from Home Assistant, retrying in 5 seconds
2024-11-08 04:52:43.843403 INFO HASS: Connected to Home Assistant 2024.11.0
2024-11-08 04:52:44.035758 INFO HASS: Evaluating startup conditions
2024-11-08 04:52:44.048257 INFO HASS: Startup condition met: hass state=RUNNING
2024-11-08 04:52:44.048500 INFO HASS: All startup conditions met
2024-11-08 04:52:44.084291 INFO AppDaemon: Processing restart for HASS
2024-11-08 04:52:44.084580 INFO AppDaemon: Terminating hello_world
2024-11-08 04:52:44.085032 INFO AppDaemon: Terminating wattwise
2024-11-08 04:52:44.085956 INFO AppDaemon: Reloading Module: /config/apps/hello.py
2024-11-08 04:52:44.086867 INFO AppDaemon: Reloading Module: /config/apps/wattwise.py
2024-11-08 04:52:44.095020 INFO AppDaemon: Loading app hello_world using class HelloWorld from module hello
2024-11-08 04:52:44.097762 INFO AppDaemon: Loading app wattwise using class WattWise from module wattwise
2024-11-08 04:52:44.100340 INFO AppDaemon: Calling initialize() for hello_world
2024-11-08 04:52:44.102117 INFO hello_world: Hello from AppDaemon
2024-11-08 04:52:44.102767 INFO hello_world: You are now ready to run Apps!
2024-11-08 04:52:44.103414 INFO AppDaemon: Calling initialize() for wattwise
2024-11-08 04:52:44.105772 INFO wattwise: Scheduled hourly optimization starting at 2024-11-08 05:00:00+01:00.
2024-11-08 04:52:44.107125 INFO wattwise: Listening for manual optimization trigger event 'MANUAL_BATTERY_OPTIMIZATION'.
2024-11-08 05:00:00.108429 INFO wattwise: Retrieving consumption forecast.
2024-11-08 05:00:00.321834 INFO wattwise: Loaded existing consumption history. Path: /config/apps/wattwise_consumption_history.json
2024-11-08 05:00:01.070245 INFO wattwise: Consumption history saved. Path: /config/apps/wattwise_consumption_history.json
2024-11-08 05:00:01.153367 INFO wattwise: Consumption forecast retrieved: [688.2685973878478, 834.8808319280495, 866.7527839643652, 714.2358175750834, 452.06939625260236, 612.0743550834597, 684.0053475935829, 2270.464484679666, 1435.7568149210904, 502.4457579972184, 550.4614315496873, 596.4377622377623, 586.0264127764128, 760.5156950672646, 849.4503496503496, 640.6872222222222, 719.396878483835, 749.1601307189543, 1015.2623716153128, 811.3830892772792, 762.5187826913932, 652.5087885985748, 546.179646440516, 642.9162870159454, 688.2685973878478, 834.8808319280495, 866.7527839643652, 714.2358175750834, 452.06939625260236, 612.0743550834597, 684.0053475935829, 2270.464484679666, 1435.7568149210904, 502.4457579972184, 550.4614315496873, 596.4377622377623, 586.0264127764128, 760.5156950672646, 849.4503496503496, 640.6872222222222, 719.396878483835, 749.1601307189543, 1015.2623716153128, 811.3830892772792, 762.5187826913932, 652.5087885985748, 546.179646440516, 642.9162870159454]
2024-11-08 05:00:01.154263 INFO wattwise: Waiting for all forecasts to be ready...
2024-11-08 05:00:01.154814 INFO wattwise: Retrieving solar production forecast.
2024-11-08 05:00:01.159149 INFO wattwise: Solar production forecast for hour 43 not found.
2024-11-08 05:00:01.159803 INFO wattwise: Setting time horizon to 43 hours.
2024-11-08 05:00:01.160420 INFO wattwise: Solar production forecast retrieved: [0.0, 0.0, 0.0561, 0.437, 1.1265, 1.6031, 1.6667, 1.3575, 1.0984, 0.7839, 0.3622, 0.0376, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.2591, 1.4506, 1.8968, 1.7829, 1.2364, 0.5198, 0.3697, 0.3574, 0.2325, 0.0331, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
2024-11-08 05:00:01.161052 INFO wattwise: Waiting for all forecasts to be ready...
2024-11-08 05:00:01.161550 INFO wattwise: Retrieving energy price forecast.
2024-11-08 05:00:01.162286 INFO wattwise: Energy price forecast data for today is unavailable.
2024-11-08 05:01:29.781799 INFO AppDaemon: New client Admin Client connected
2024-11-08 05:12:17.974867 INFO AppDaemon: Terminating wattwise
2024-11-08 05:12:17.976150 INFO AppDaemon: Reloading Module: /config/apps/wattwise.py
2024-11-08 05:12:17.988384 INFO AppDaemon: Loading app wattwise using class WattWise from module wattwise
2024-11-08 05:12:17.989817 INFO AppDaemon: Calling initialize() for wattwise
2024-11-08 05:12:17.992757 INFO wattwise: Scheduled hourly optimization starting at 2024-11-08 06:00:00+01:00.
2024-11-08 05:12:17.994322 INFO wattwise: Listening for manual optimization trigger event 'MANUAL_BATTERY_OPTIMIZATION'.

jh537 · November 8, 2024, 5:09am

Retrieving the North Pole data seems to work. However, the format probably does not fit…

2024-11-08 06:00:01.205698 INFO wattwise: Setting time horizon to 42 hours.
2024-11-08 06:00:01.206304 INFO wattwise: Solar production forecast retrieved: [0.0, 0.0561, 0.437, 1.1265, 1.6031, 1.6667, 1.3575, 1.0984, 0.7839, 0.3622, 0.0376, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.2591, 1.4506, 1.8968, 1.7829, 1.2364, 0.5198, 0.3697, 0.3574, 0.2325, 0.0331, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
2024-11-08 06:00:01.206928 INFO wattwise: Waiting for all forecasts to be ready...
2024-11-08 06:00:01.207430 INFO wattwise: Retrieving energy price forecast.
2024-11-08 06:00:01.209451 INFO wattwise: Tomorrow's energy price data is not available yet.
2024-11-08 06:00:01.209706 WARNING wattwise: ------------------------------------------------------------
2024-11-08 06:00:01.209801 WARNING wattwise: Unexpected error in worker for App wattwise:
2024-11-08 06:00:01.209911 WARNING wattwise: Worker Ags: {'id': '3bee429c2b9345568a3538c5a60f1e36', 'name': 'wattwise', 'objectid': '5025e0b2728e4b37b1df0d8fb9d3f0a9', 'type': 'scheduler', 'function': <bound method WattWise.start_optimization_process of <wattwise.WattWise object at 0x7f7359f2d0>>, 'pin_app': True, 'pin_thread': 1, 'kwargs': {'interval': 3600, '__thread_id': 'thread-1'}}
2024-11-08 06:00:01.209996 WARNING wattwise: ------------------------------------------------------------
2024-11-08 06:00:01.210600 WARNING wattwise: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/threading.py", line 1022, in worker
    funcref(self.AD.sched.sanitize_timer_kwargs(app, args["kwargs"]))
  File "/config/apps/wattwise.py", line 198, in start_optimization_process
    self.get_energy_price_forecast()
  File "/config/apps/wattwise.py", line 458, in get_energy_price_forecast
    price = price_entry["total"] * 100  # Convert EUR/kWh to ct/kWh
            ~~~~~~~~~~~^^^^^^^^^
TypeError: 'float' object is not subscriptable
2024-11-08 06:00:01.210700 WARNING wattwise: ------------------------------------------------------------

state_class: total
average: 0.13941666666666666
off_peak_1: 0.127
off_peak_2: 0.136
peak: 0.14883333333333335
min: 0.12
max: 0.163
mean: 0.137
unit: kWh
currency: EUR
country: Germany
region: GER
low_price: true
price_percent_to_average: 0.9468021518230724
today:
  - 0.122
  - 0.123
  - 0.121
  - 0.12
  - 0.12
  - 0.121
  - 0.132
  - 0.157
  - 0.163
  - 0.156
  - 0.144
  - 0.136
  - 0.13
  - 0.131
  - 0.138
  - 0.152
  - 0.159
  - 0.158
  - 0.163
  - 0.156
  - 0.142
  - 0.135
  - 0.139
  - 0.128
tomorrow: []
tomorrow_valid: false
raw_today:
  - start: "2024-11-08T00:00:00+01:00"
    end: "2024-11-08T01:00:00+01:00"
    value: 0.122
  - start: "2024-11-08T01:00:00+01:00"
    end: "2024-11-08T02:00:00+01:00"
    value: 0.123
  - start: "2024-11-08T02:00:00+01:00"
    end: "2024-11-08T03:00:00+01:00"
    value: 0.121
  - start: "2024-11-08T03:00:00+01:00"
    end: "2024-11-08T04:00:00+01:00"
    value: 0.12
  - start: "2024-11-08T04:00:00+01:00"
    end: "2024-11-08T05:00:00+01:00"
    value: 0.12
  - start: "2024-11-08T05:00:00+01:00"
    end: "2024-11-08T06:00:00+01:00"
    value: 0.121
  - start: "2024-11-08T06:00:00+01:00"
    end: "2024-11-08T07:00:00+01:00"
    value: 0.132
  - start: "2024-11-08T07:00:00+01:00"
    end: "2024-11-08T08:00:00+01:00"
    value: 0.157
  - start: "2024-11-08T08:00:00+01:00"
    end: "2024-11-08T09:00:00+01:00"
    value: 0.163
  - start: "2024-11-08T09:00:00+01:00"
    end: "2024-11-08T10:00:00+01:00"
    value: 0.156
  - start: "2024-11-08T10:00:00+01:00"
    end: "2024-11-08T11:00:00+01:00"
    value: 0.144
  - start: "2024-11-08T11:00:00+01:00"
    end: "2024-11-08T12:00:00+01:00"
    value: 0.136
  - start: "2024-11-08T12:00:00+01:00"
    end: "2024-11-08T13:00:00+01:00"
    value: 0.13
  - start: "2024-11-08T13:00:00+01:00"
    end: "2024-11-08T14:00:00+01:00"
    value: 0.131
  - start: "2024-11-08T14:00:00+01:00"
    end: "2024-11-08T15:00:00+01:00"
    value: 0.138
  - start: "2024-11-08T15:00:00+01:00"
    end: "2024-11-08T16:00:00+01:00"
    value: 0.152
  - start: "2024-11-08T16:00:00+01:00"
    end: "2024-11-08T17:00:00+01:00"
    value: 0.159
  - start: "2024-11-08T17:00:00+01:00"
    end: "2024-11-08T18:00:00+01:00"
    value: 0.158
  - start: "2024-11-08T18:00:00+01:00"
    end: "2024-11-08T19:00:00+01:00"
    value: 0.163
  - start: "2024-11-08T19:00:00+01:00"
    end: "2024-11-08T20:00:00+01:00"
    value: 0.156
  - start: "2024-11-08T20:00:00+01:00"
    end: "2024-11-08T21:00:00+01:00"
    value: 0.142
  - start: "2024-11-08T21:00:00+01:00"
    end: "2024-11-08T22:00:00+01:00"
    value: 0.135
  - start: "2024-11-08T22:00:00+01:00"
    end: "2024-11-08T23:00:00+01:00"
    value: 0.139
  - start: "2024-11-08T23:00:00+01:00"
    end: "2024-11-09T00:00:00+01:00"
    value: 0.128
raw_tomorrow: []
current_price: 0.132
additional_costs_current_hour: 0
price_in_cents: false
unit_of_measurement: EUR/kWh
device_class: monetary
icon: mdi:flash
friendly_name: nordpool_kwh_ger_eur_3_10_023

bullitt186 · November 8, 2024, 11:51am

the actual PV production is not relevant for calculating the forecast, thus it is not needed in the algorithm.
I use the official Tibber Integration, so it’s odd that you have to use another one.
Maybe you want to try using the tibber integration in addition to the nordpol one (which is unknown to me)

Regarding the diagram:
The easiest way to get it on a dashboard is to add a manual card (on the bottom of the cards list) and copy/paste the yaml and then edit/replace the entity names for your PV system.

I am constantly updating wattwise so you may want to always use the latest version from my git

jh537 · November 8, 2024, 12:54pm

Thanks, great for the answers. I have just learned that I need to build a template to retrieve the data from Tibber.

I am still unsure which custom sensors I need to customize. Can you perhaps mark this better. Or is it all sensors without wattwise?

bullitt186 · November 8, 2024, 2:39pm

based on some other feedback i improved the structuring and readme.md just now.
i also had an error in my instructions, since i forgot i used a custom rest api sensor for fetching the prices from tibber.
with the most recent changes, it should work now and is also better maintainable.
So i suggest to follow the updated readme “from scratch” and let me know if this works for you.

jh537 · November 10, 2024, 7:37am

So, it’s running - I had some problems with the units on the original sensors - I had forgotten to set them to the right unit. This of course resulted in incorrect calculations. I have deleted the data. You can still see remnants of the incorrect values in the chart. After checking the next forecasts, I will then convert the switches to memory.
Many thanks for this great system, for which I had to wait so long.

bullitt186 · November 11, 2024, 8:33pm

happy to hear that its working! its far from perfect but a start

jh537 · November 18, 2024, 5:46am

Switch for PV modules covered (snow)