Integrating Anglia water usage for smart meters (UK)

Thanks @Zeunas I’m fairly new to this so any help you could offer would be appreciated

Let me see if I can find some time later this week to look at it :wink: have you had any troubles setting up everything else?

I get daily usage in fine. The only issue is that it shows the usage today on the energy dashboard rather than yesterday.

Any help you could give would be gratefully appreciated. :+1:

Unfortunately that’s how HA logs things, it can’t log on the previous day unless you manually amend the value under Developer Tools > Statistics, but that’s not practical…Getting Hourly data won’t help either as that will also be from previous day as far as I am aware (at least it is on mine).

Anyways, this should give you an hourly reading, however I strongly recommend not to use this as it’s set up up to login to your Anglia Water account every hour to extract the readings and not sure if they will put a tap on the amount of logins you can have daily…

from selenium import webdriver
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
import appdaemon.plugins.hass.hassapi as hass
import time
import json
import ctypes
import datetime

#####################
# App configuration #
#####################
# Complete with your settings
anglia_username = "_YOUR_ANGLIA_WATER_EMAIL-"
anglia_password = "_YOUR_ANGLIA_WATER_PASSWORD_"
usagesensor = "sensor.water_usage"
# Optional: Uncomment the below line in case you want to have a switch to run the script on demand if for eg. the script doesn't run at 11am because your HA is restarting. Make sure your input_boolean matches the below:
#force_reading = "input_boolean.water_reading"

###################
# WATER USAGE APP #
###################

webpage = r"https://my.anglianwater.co.uk/"

class GetWaterUsage(hass.Hass):

# Starting APP #

  def initialize(self):
    self.log("Starting Water Usage App")
    runtime  = datetime.time(0, 0, 0)
    self.run_hourly(self.StartProcess, runtime)
    self.run_daily(self.EndProcess, runtime)    
    self.listen_state(self.StartProcessForced,force_reading, attribute="state", new="on", duration=10 )

  #Callback Function to Start the process 
  def StartProcess(self, kwargs):
    self.log("starting process")
    usage = self.GetUsage()
    self.log("Usage: " + usage)
    try:
      self.set_state(usagesensor, state = usage)
    except Exception as e:
      self.log("Set Sensor State Error: " + str(e))

  #FORCED Callback Function to Start the process 
  def StartProcessForced(self, entity, attribute, old, new, kwargs):
    self.log("starting process")
    self.turn_off(force_reading)
    usage = self.GetUsage()
    self.log("Usage: " + usage)
    try:
      self.set_state(usagesensor, state = usage)
    except Exception as e:
      self.log("Set Sensor State Error: " + str(e))

  #Callback Function to Zero the readings 
  def EndProcess(self, kwargs):
    self.log("zero readings")
    usage = "0.0"
    self.log("Usage: " + usage)
    self.set_state(usagesensor, state = usage)
    
# Get Water Usage from Anglia Water website #

  def GetUsage(self):
    usage = "0.0"
    try:
      self.log("starting request")
      service = Service(executable_path=r'/usr/bin/chromedriver')
      chrome_options = webdriver.ChromeOptions()
      chrome_options.add_argument('--headless')
      chrome_options.add_argument('--no-sandbox')
      chrome_options.add_argument('--disable-gpu')
      chrome_options.add_argument('--disable-dev-shm-usage')
      browser = webdriver.Chrome(service=service, options=chrome_options)
      browser.set_window_size(1600, 1200)
      try:
        self.log("logging in")
        browser.get(webpage)
        sbox = browser.find_element(By.ID,'existUser')
        sbox.send_keys(anglia_username)
        sbox = browser.find_element(By.ID,'existPass')
        sbox.send_keys(anglia_password)
        button = browser.find_element(By.ID,'existingLogIn')
        button.click()

        self.log("selecting usage")
        button = browser.find_element(By.ID,'btnViewUsage')
        button.click()
        
        self.log("selecting hourly")
        button = browser.find_element(By.XPATH,'//*[@id="dbserialnumber"]/div/div[1]/div[1]/label/span')
        button.click()
        
        time.sleep(2)

        self.log("saving usage")
        html = browser.page_source
        browser.quit()
        startdata = html.find("var myUsageDetails_days = ") + 26
        enddata = html.find(";;", startdata)
        result = html[startdata: enddata]
        data = json.loads(result.replace("'", '"'))
        json_body = []

        for x in data:
            item = x["usage"] * 1000
            json_body.append(item)

        usage = str(json_body[-1])

      except Exception as e:
        self.log("Request Error: " + str(e))
        browser.close()
    except Exception as e:
      self.log("GetUsage Error: " + str(e))
    return usage

hmm so I thought I was following this all fine but getting the following error in the logs, this is the first time i’ve tried to use it:-

2023-12-02 11:33:07.851199 INFO WaterUsage: Starting Water Usage App
2023-12-02 11:33:07.852957 WARNING WaterUsage: ------------------------------------------------------------
2023-12-02 11:33:07.853256 WARNING WaterUsage: Unexpected error running initialize() for WaterUsage
2023-12-02 11:33:07.853497 WARNING WaterUsage: ------------------------------------------------------------
2023-12-02 11:33:07.854421 WARNING WaterUsage: Traceback (most recent call last):
File “/usr/lib/python3.11/site-packages/appdaemon/app_management.py”, line 162, in initialize_app
await utils.run_in_executor(self, init)
File “/usr/lib/python3.11/site-packages/appdaemon/utils.py”, line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/concurrent/futures/thread.py”, line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/config/apps/GetWaterUsage.py”, line 38, in initialize
self.listen_state(self.StartProcessForced,force_reading, attribute=“state”, new=“on”, duration=10 )
^^^^^^^^^^^^^
NameError: name ‘force_reading’ is not defined

2023-12-02 11:33:07.855173 WARNING WaterUsage: ------------------------------------------------------------

Do you mind sharing you’re config please? Be mindful of not sharing your credentials.

Did you have any luck fixing the hour version?

I keep getting influxDB error and I can’t for the life of me work out why! As all other addons are connecting to influx db fine and I’ve changed no settings

Appdaemon keeps crashing as it can’t did chromium-chromedriver

ERROR: unable to select packages:
chromium-chromedriver (no such package):
required by: world[chromium-chromedriver]
[10:48:55] FATAL: Failed installing package chromium-chromedriver

I was having appdaemon problems last month with latest update as the file structure changed for add-ons. That seems to be resolved and it briefly worked. But now this!

Not sure what it could be but can you try updating your appdaemon.yaml as per below example?

secrets: /homeassistant/secrets.yaml
appdaemon:
  latitude: xx.xxxxxx #keep you're current one
  longitude: xx.xxxxxx #keep you're current one
  elevation: xxx #keep you're current one
  time_zone: Europe/London #keep you're current one
  thread_duration_warning_threshold: 80
  plugins:
    HASS:
      type: hass
http:
  url: http://127.0.0.1:5050
admin:
api:
hadashboard:

Hopefully someone is able to help me.
Firstly - thanks for putting this together. I’ve got it setup and running in docker on Unraid but I’m seeing the following error on startup:

I’m not too sure on how to best resolve the force_reading issue? Cheers for any help :slight_smile:

File “/usr/local/lib/python3.10/site-packages/appdaemon/app_management.py”, line 162, in initialize_app
await utils.run_in_executor(self, init)
File “/usr/local/lib/python3.10/site-packages/appdaemon/utils.py”, line 304, in run_in_executor
response = future.result()
File “/usr/local/lib/python3.10/concurrent/futures/thread.py”, line 58, in run
result = self.fn(*self.args, **self.kwargs)
File “/conf/apps/GetWaterUsage.py”, line 38, in initialize
self.listen_state(self.StartProcessForced,force_reading, attribute=“state”, new=“on”, duration=10 )
NameError: name ‘force_reading’ is not defined

Hiya,
Have you read this bit and have you created the input_boolean?

# Optional: Uncomment the below line in case you want to have a switch to run the script on demand if for eg. the script doesn't run at 11am because your HA is restarting. Make sure your input_boolean matches the below:
#force_reading = "input_boolean.water_reading"

You can always delete this bit from the code (should work):

#FORCED Callback Function to Start the process 
  def StartProcessForced(self, entity, attribute, old, new, kwargs):
    self.log("starting process")
    self.turn_off(force_reading)
    usage = self.GetUsage()
    self.log("Usage: " + usage)
    try:
      self.set_state(usagesensor, state = usage)
    except Exception as e:
      self.log("Set Sensor State Error: " + str(e))

My file was basically the same.

It has randomly allowed it to start today though so all good.

Strange it failed as it couldn’t find the packages. So probably something out of my control.

I’ve installed influxdb2 now. And it allows flux queries which. So should be able to create better sensors for hourly data. I’ll look at that in new year.

Just wondering if anyone else is having trouble recently with no usage being returned?

The integration worked seamlessly up until 4th December. Since then it’s been very hit and miss. Even manually refreshing does not not seem to work consistently.

The output from AppDaemon is as follows:

2024-01-01 16:00:10.247584 INFO WaterUsage: starting process
2024-01-01 16:00:10.251082 INFO WaterUsage: starting request
2024-01-01 16:00:11.145024 INFO WaterUsage: logging in
2024-01-01 16:00:17.666603 INFO WaterUsage: selecting usage
2024-01-01 16:00:21.495264 INFO WaterUsage: saving usage
2024-01-01 16:00:21.578389 INFO WaterUsage: Usage: 0

I have checked in my Anglian Water account and hourly readings are being recorded. The only difference I can think of is that when logging in to Anglian Water it asks if you would like to try the new MyAccount - maybe they have changed something?

Still working fine for me, yes I have noticed the new “MyAccount login” and I am keeping an eye on that, as I assume they may change the way you login soon. It might break this integration to a point I can’t fix it but so far they are letting you login the normal way, so that shouldn’t be affecting this integration at the moment. In fact your logs show that it’s logging into the website and accessing the option it needs to download the values.

You wrote “hourly readings” but I have developed this to just read the “daily readings”, have you made the changes to the script as per initial script from Dan? Also referenced on my post below?

The only other thing I can think of is if you are indeed just recording the daily readings is that you might have this script running at a time of the day where anglia waters has not uploaded you’re previous day readings. Just double checking you have checked that?

Thank you for your reply. Just to clarify I am fetching the daily usage from the Anglian Water website. I have checked my online account and can see that my water meter is sending hourly usage to them.

Originally I was fetching the data at 13:00 but moved the time a little later in the day when it started to report no usage.

It’s really odd as it worked flawlessly in November.

So just to be sure, with the time adjustment are you positive that values will be ready before that now? The reason I am asking this it’s because for me the values were available from 9am but few months later I had to change it because values were only staring to be available 1pm.

I’ve tried retrieving the usage at 1pm, 2pm, 3pm and even 4pm. No specific time consistently report any usage.

But is data available from the previous day on your Anglia waters account for those specific timings?

I have done some investigation and whilst it seems the hourly usage is reported on the website it does not immediately update the daily. Even with all of the previous days hourly usage showing (up to midnight), the daily total is not updating until after 6pm. At the moment it seems to be working as it should. @Zeunas thankyou for your help.

Yes, it’s a bit bizarre. Glad you found the root cause :wink: