Thames Water data

Third party integrations
1

UK Water Utility suppliers have been rolling out meters for many years with a view to encouraging a reduction in water consumption by customers. A general rule of thumb is that if the number of people in a household is less than the number of bedrooms you would likely find your bill cheaper as a result.

More recently the Water companies have been catching up with energy providers and upgrading these meters which formerly had to be manually read with newer smart meters that automatically upload consumption data.

About a year ago they replaced my older meter with a new smart one.

At the time I was not aware of any way to access this data myself and both due to location - under the public footpath outside my house, and the fact that the water companies would get confused and upset if I tried clipping a sensor to it to monitor it directly, options to monitor it were not feasible. However now that it is a smart meter the data is uploaded to their website and it turns out it can be accessed free of charge.

I did find an article about this however it was in relation to Anglian Water whereas my provider is Thames Water. (I am pretty sure there is no standardisation to this.)

I have now found details on how to create an online account with Thames Water and can login and see the data. Disappointingly it seems to be several days behind as the most recent data is from the beginning of the week, it may also only show hourly figures. Still it is more data than I had before.

I am therefore wondering if there are any more generalised solutions for incorporating such Water utility data in to Home Assistant? Does anyone know how frequently these smart water meters actually report data? (If their concern is that actually reporting more frequently would lead to excessive power consumption then they could solve this by sending the data in batches e.g. hourly but containing in each batch more frequent readings.) I did find the following which I have not yet tried.

1 Like
2

Hi @jelockwood,

It is good to see you still around in some Home Automation topics. I haven’t heard of you for years on the SmartThings forums.

I did a quick search regarding Thames Water and this comes up combined with the information in the other topic:

After a search for FlexNet and SensusRF lead me to a dead end on GitHub, as they are proprietary protocols and nobody managed to decrypt them yet.

A general code search for “sensus flexnet” on Github brings up a lot of interesting code, but nothing useful just out of the box.

3

@GSzabados
I am thinking that scraping the website is going to be the only option but this is likely give minimal capabilities.

It looks like the Water utilities are still some way behind the energy companies and that there is no standardisation equivalent to that provided by the DCC for energy smart meters.

I did find an OfWat document indicating that OfWat had cracked down on the Water utilities to make them provide the data available free of charge but not any details of how they are required to do so.

https://www.ofwat.gov.uk/consultation/notice-of-intention-to-accept-binding-commitments-from-thames-water-utilities-ltd-in-relation-to-the-provision-of-access-to-its-smart-meters-and-digital-data-services/

4

Unfortunately yes, probably the only way now. And as the data is outdated by a few days, I would scrape it and put it into an InfluxDB and display it with Grafana. I do not know any way how data could be logged into the longterm stats database (table) in HA retroactively.

5

I would love to be able to get live readings from my TW meter. It would have helped me catch a leak I had earlier this year where tens of thousands of litres of water had leaked under my and my neighbours house.
The leak was at the actual meter box which had been damaged by a nearby tree but the meter readings were ticking up.

In case it’s helpful for any future readers of this thread, this is what a new (as of mid-2022 anyway) smart meter looks like.

image
image1570Ă—1384 224 KB

6

@white.hat3867
Virtually identical situation to what happened to me a couple of years ago.

As the leak was at the meter, fitting a sensor in my house to measure flow would not have helped at all. The water company would not let me fit an optical sensor or similar to the meter but in your case as it is an LCD display an optical sensor would not have worked anyway. Also at least in my case the water meter was under the public footpath so too far to have my own sensor transmit to my house.

As I believe I mentioned above, in my case the water companies website does show data but is about 3-4 days behind so right now it shows up to 12th December and today is the 15th.

My leak did not seem to be as bad as yours so it was only when I had had a couple of bills and could see they where more than double normal that something was wrong, there was no evidence physically visible of a leak.

Now that I have a water company smart meter I can login to their website and would be able to see a chart of the last 30 day at a time usage and see if it had suddenly gone up or down but as mentioned this would be 3-4 days after the event.

Screen Shot 2022-12-15 at 14.42.50

7

Yup, mine is exactly the same as yours - I only see up to the 12th December.

I’m not opposed to attaching something to my meter and I would get a Zigbee connection to the house from the footpath but there are other things to worry about such as humidity, temperature and battery life of this mini IoT device under the footpath :smiley:

I wouldn’t be surprised if the meter only transmitted its readings once per day or something like that as they are battery powered. Similar to why gas meters don’t transmit as frequently as electric meters.

8

@white.hat3867
I have read that optical sensors do not work reliable with LCD displays like your meter has. They did work fine with the older rotating odometer style meters. This was mentioned in a discussion of new electric smart meters which also have LCD displays rather than odometers.

It would be a big help if the water companies provided an API to access the data. The DCC network for smart meters do but not as far as I am aware the water companies who a years behind in tech.

9

In case it’s useful, I came across this document which is actually quite helpful in providing information about meters and data accessibility. I’m going to try my luck with TW to see if I can somehow get “backend” access to data for my meter.

10

@white.hat3867
Thanks for linking to that document, very interesting.

I did however get the impression that TW only offer the date to resellers and similar and not customers directly. It also implies they will charge a fee for providing such access.

I did previously find articles suggesting OFWAT had reprimanded TW over this. The following is one such article, there have been follow up ones.

https://www.ofwat.gov.uk/pn-04-22-competition-act-investigation-thames-water-proposes-to-change-smart-metering-policy-for-business-retail-customers/

So if TW try charging you refer them to this.

11

I’m based in London and I’ve tried a few things:

  1. I had a go setting up the RTL-SDR radio to decode the signals from the newly installed TW meter (installed in June 2022 and looks like the picture above). I tried the code found here: https://github.com/allangood/rtlamr2mqtt Unfortunately, after running for a few days there were no decoded messages so I figured that the TW readings are encrypted (as mentioned above).

  2. Tried the (now 4+ years old) python scraper by papadeltasierra/thameswater (link above) and had random success. In the end it required nursing and would not run autonomously on my HA so I’ve left that to one side now.

So if you can’t decode the real-time data, and the scraping method (even if it works) will be 3+ days old, I am thinking the only real way to do this will be to install a personal meter.

I am having difficulty finding a meter that I can install on my incoming mains, hence this post.

Does anyone know of a mains water capable meter (i.e. not irrigation meter) that can be fit and send real-time usage via Zigbee, Z-Wave or Wifi to a LOCAL machine (no cloud or 3rd party company reliance preferable)?

My thinking thus far is a water meter that provides a pulse on usage with an ESPHome to receive and transmit onward via MQTT.

Any thoughts or suggestions before I cut into pipes and fiddle with ESPHome?

Cheers

12

@Grish
If you are going to fit your own water meter then an approach to consider is a smart water shutoff valve which as a side effect of detecting leaks also does some measurements.

I am not convinced typical method of these is likely to provide the same accuracy of flow measurement but may be close enough and this type of product is at least designed for home owners to have fitted in their house. I am not yet fully sold on available UK solutions for a variety of reasons but found the following.

The above is I believe now a Belkin product but is not on the Belkin site. I did see a recent press release about extending the rollout of the product to Europe.

The above is available in the UK I know an insurance company that also promotes it.

https://www.grohe.co.uk/en_gb/smarthome/grohe-sense-guard/

The above is Germany but as you can see also available in the UK. I did not like the fact it seems designed for a horizontal incoming water supply pipe. In most houses the pipe usually enters vertically from below ground up in to the e.g. kitchen.

There is the Flo by Moen but this does not seem to be available in the UK and they seem to have no interest in doing so. It also seems to require an ongoing subscription according to feedback on Amazon.

https://leaksafe.com/products/

The above seems to do the job but I get the impression it is less friendly to any smart home integrations.

There are a lot of controllable shut off valves but they themselves have no flow measurement, they would potentially work with a leak sensor but they merely detect the presence of water and again not the flow. There are similarly others which might be suitable except the makers think US=World.

13

Hi John - thanks for the reply.

I hadn’t seen the Phyn or the Grohe, but did have a look at SONIC by HERO and the Flo devices. All of them are quite pricey and some have an app - cloud based - meaning at risk of the service being removed, or changed to charge money in the future (or already charging). I am also concerned with fitting one as they are all non-UK devices - will they fit a UK pipe? Will I need to add an adapter which may reduce the flow in the house and would this diameter change to the pipe make the device inaccurate for usage measurements?

My annual water bill is about £500 - so spending £700 + (monthly) service fee isn’t worth it for me to know how much water is used when I flush the loo : :woozy_face:

The auto-shut off is a game changer (as you point out) and could make it worth-while for some.

It did for me, I already have an automatic cut-off valve (not unlike the leaksafe you mention). Its been in place for 15 years (and was expensive “but worth it” back then) - its an autostopcock: https://www.autostopcock.co.uk/. It definitely works - a running toilet (or badly closed tap) after 15 minutes will cut off all water to the house. It’s simple and it works - great device, just not smart. I see they do a “Plus” version which includes flow now - but that’s over £700… I might have been a little ahead of the curve when I installed it :slight_smile:

So I’m not really concerned with flood/auto water cut-off - just the measuring of use in quasi real-time. Delayed metering just does not allow for making a connection between what is non-essential use vs required use. I can’t remember what I had for breakfast, let alone how and why I used water three days ago! Measuring flow in real time means I can set an alarm to say “hey you’re using a lot of water now” and maybe think twice about it - or know that my 15 min setting for garden irrigation uses 200+ litres of water a day, when I only really need 30 - or how many times a year do I really need to jet-wash the garden.

I’ve seen pulse meters for circa £40. Add an ESP and a few other bits and the total cost would be around £60 - and all self-hosted.

I’m not in a rush, so might kick the can down the road too, but I thought someone might have a cheap and cheerful solution already…

However, there may be another way…

Reading the TW document linked to above, I see:

“Alternatively, you can either fit a splitter that will allow you to install your own
logger next to ours, at your cost, or ask us to fit the splitter for you through our Fitting
Splitters for data loggers service (charges apply).”

It also says -

“Do I need to ask for your permission to install a logger on your meter?
No, you do not need to ask for our consent to install a logger on a meter.”

Pin outs for a logger cable “thames water compatible” is also included in the document.

So this opens another direction to a solution!

Fit an independent data logger to the meter that is currently in place. It would need to be small and run on batteries. Any suggestion on a data logger that would fit this requirement and provide the info to HA?

Wouldn’t need to cut the mains feed, just set it up and drop it in the hole!

The only caveat is that not all meters that TW fit have a pulse output. Mine has an obvious side attachment from Sensus (The meter is Sensus, too) which uses “FlexNet”. This is the radio that sends usage data. It is slid down and next to the meter - receiving some kind of pulse (not a wired pulse, it’s radio) from the meter for counting - not sure if they are paired & encrypted devices (which they probably are so you don’t inadvertently pick up and send the neighbour’s data as yours…)

But it does give me (some) hope I could just pop another box in the hole and get the readings…

Maybe?

14

@Grish
The Sonic, Grohe and Leaksafe are definitely available in the UK and will fit UK pipes. The Phyn should become available in the UK via Belkin but I don’t think this is true yet.

15

If they are in the UK then they should be compatible. Most of them look good and will do the job, so I am sure those without auto-water shutoff will find the most value there (with the nice side-effect of getting water usage info, too).

For my situation (just need to read the flow), I’m more and more of the opinion that I should fit a second water meter (seen some relatively cheap LoRaWAN meters in the UK (circa £40) that I think might work - not entirely sure at the moment) https://247able.com/cat/flow/water-meters/- this allows me to integrate to HA as I like, I’ll get real time usage, and be able to check against Thames Water’s meter for accuracy.

I’ve got a LoRaWAN gateway already installed which should work - just does not seem like WiFi or ZigBee devices exist for metering…

16

Also -

Incidentally, The Sensus 640 meter (the standard meter TW now installs) is “Digital-only”

I believe you can have the 620 (which is a pulse meter) fit “at your request” - which opens up the possibility of pulse meter reading. As previously mentioned, this is allowed by Thames water too. The pulse goes to their radio for the TW network, but you can splice into the sensor cable and add your own logger (and this is permitted, and you don’t need to ask TW).

Here’s how they fit the radio: How to install Sensus HRI Sensor to Sensus 620 Water Meter - YouTube

So you could set up an ESP32 to read the pulses and transmit them back into HA.

Problem is power for the ESP32 - which will be in a hold in the public pavement…

17

@Grish did you end up finding / installing something for your purposes? Your needs are about the same as mine - interested but not hugely invested. ÂŁ500 + installation seems like a tall ask when I just wanna know how much a shower is costing me.

18

Just following back to this discussion. I did not get a personal meter or integrate it into the HA. I thought maybe waiting for a time - there might be an API or something that would scrape the TW website data for my account that could then show up in HA. Obviously the granularity of the data would not be as good, nor the “post-next day” nature of the TW website - so not real time.

Had a goal of adding water stats to my end of bathroom routine which gives me the weather, a joke and could have told me how many liters of water was used… And if you put two meters - one on cold and one on hot water you could measure that too…

Was always put down as a nice to have, not a requirement…

Any updates from others on this front?

19

I’ve done some digging into the Thames Water Digital Meter data service and from https://www.thameswater.co.uk/media-library/home/wholesale/our-charges/2023-24/wholesale-tariff-document-2023-24.pdf have discovered that it is a SFTP data dump - see page 72 and as such isn’t fit for use for individual home users even if they could be persuaded to provide it at not cost to homeowners.

Their own web site uses a rest-like API with https://myaccount.thameswater.co.uk/ajax/dashboard/getmeterdata returning json details used to build the table, e.g

    "MyMeterTableModelList": [
        {
         ....
        {
            "DeviceId": "31******6",
            "MeterreadDate": "18 Nov 2023",
            "Meterreadtype": "ACTUAL",
            "MeterReading": "29",
            "WaterConsumption": 29000
        },

Although I’ve not managed to call this successfully from curl yet.

1 Like
20

I’ve just found this discussion as I got fitted a smart water meter just a few hours ago. And now I’m also looking into getting this data into HA.

I’d like to throw this idea into the mix:

It should be possible to tweak this to detect the LCD digits. Of course, this only works for where you have easy access to the meter and a power source for the ESP and light nearby. Never tested it myself. But sounds promising.