this looks very interesting
was told by my city that hour water meters report usage every hour
they were nice enought to provide a printout of usage and across bottom I found
https://CTYST.sensus-analytics.com/deviceaccess#?/returnable=true&devicenuber=########
your google link shows indeed many cities are using sensus-analytics.com services
Did your city provide access credentials for you ?
Have you found more success in your efforts here ?
thx
Wondering if anyone has looked into this further. Very much interested now that water usage has been added to the energy dashboard and my utility company uses this portal.
Thanks!
Check in after a year, unfortunately there’s still no movement here. With my very minimal level understanding I was able to reproduce these JSON endpoints reliably in Postman to give me my current AND historical usage data in a JSON format, but I know nothing about authentication so as soon as my login expires I’ve got to manually re-log in, and re-copy the authentication headers from my browser’s dev console.
If someone actually knew what they were doing here it seems like this would be a slam dunk integration that would get lots more users being able to integrate their Utilities into their Energy Dashboard. I know I can access both my water and electric meters through this website, and I’d be happy to help make this happen however I can.
And just to again show, there are even more utilities using this platform than before:
https://www.google.com/search?q=inurl:sensus-analytics.com&filter=0
Yeah, our city did provide login access for customers. Looks like the login page isn’t working for yours tho. Our login page is at “/login.html” but when I try to go to that on your url there, it just gives an error page. I’d reach out to your utility and see if you can convince them to give you user access to your dashboard. This is what it looks like for us. Assming they’re using the same playform this should be available to you 
I’m also very interested in trying to make this work. I did a bit of poking around but couldn’t figure out the authentication since it seems to be quite complex and a lot of transactions occur.
This is the output from my City’s portal in Coppell Texas.
Just make sure you’re voting for it at the top of the thread, unfortunately it looks like there’s not enough interest yet to attract anyone to do anything with it.
I would love to see someone develop a connector for sensus-analytics (.com/.ca). As more cities start to transition to smart meters I expect the potential user base for this will grow
Just wondering if anyone has been able to extract the daily water usage data out of this site?
I was able to get it half way working in node red. Lost my setup going to work on it again. Just a clue for authentication, the use an old Spring Authentication feature.
I was able to write the following BASH script to extract data from my local town’s utility. Not sure if it is helpful or not or if someone would want to add on. Basically I used developer tools in Chrome and network captured each step from logging in to capturing the daily and hourly data.
#! /bin/bash
# Ask for user input for start date/time
read -p "Enter start date/time (mm/dd/yy hh:mm): " start_datetime
# Ask for user input for end date/time
read -p "Enter end date/time (mm/dd/yy hh:mm): " end_datetime
# Function to convert date/time to Unix timestamp
orig_start=$start_datetime
epoch_start=`expr $(date -d "${orig_start}" +"%s") \* 1000`
epoch_to_date_start=$(date -d @$epoch_start +%Y%m%d_%H%M%S)
orig_end=$end_datetime
epoch_end=`expr $(date -d "${orig_end}" +"%s") \* 1000`
epoch_to_date_end=$(date -d @$epoch_end +%Y%m%d_%H%M%S)
curl 'https://my-town.sensus-analytics.com/j_spring_security_check' \
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
-H 'accept-language: en-US,en;q=0.9' \
-H 'cache-control: max-age=0' \
-H 'content-type: application/x-www-form-urlencoded' \
-H 'cookie: defaultUnits=rain%3DINCHES%7Ctemp%3DF%7Celectric%3DKWH%7Cgas%3DCCF%7Cwater%3DGAL%7C; widgetOrderWATER=Alerts%2CBill%2CGoal%2CWater%2CMActivity%2CNotifications%2CThreshold%2CUsage%2CBudget%2COutdoors%2COUTDOORS%2CPEERS%2CTERMS; widgetOrderGAS=Alerts%2CBill%2CGoal%2CMActivity%2CNotifications%2CThreshold%2CUsage%2COUTDOORS%2CTERMS; widgetOrderELECTRIC=Alerts%2CBill%2CGoal%2CMActivity%2CNotifications%2CThreshold%2CUsage%2COUTDOORS%2CTERMS; disclaimers=false; theme=white; JSESSIONID=074EC68AB845DD09D1E067679EFDCB81; AWSALB=PtQkySWIrRXGSdIb1DjZXkcoT7E1sN1h2JTEcnGzJ8pagm8SzO/p2xIqYVL6VghbMeQllJNOsMC/Ud317sQO7XbCps8mN9ko2Ppbu+x81GhoaPsmxefwNCH4tocC; AWSALBCORS=PtQkySWIrRXGSdIb1DjZXkcoT7E1sN1h2JTEcnGzJ8pagm8SzO/p2xIqYVL6VghbMeQllJNOsMC/Ud317sQO7XbCps8mN9ko2Ppbu+x81GhoaPsmxefwNCH4tocC' \
-H 'dnt: 1' \
-H 'origin: https://my-town.sensus-analytics.com' \
-H 'priority: u=0, i' \
-H 'referer: https://my-town.sensus-analytics.com/login.html' \
-H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: document' \
-H 'sec-fetch-mode: navigate' \
-H 'sec-fetch-site: same-origin' \
-H 'sec-fetch-user: ?1' \
-H 'upgrade-insecure-requests: 1' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
-c cookies.txt \
--data-raw 'j_username=myemailaddress%40gmail.com&j_password=plaintextpassword' \
--silent --output /dev/null ;
curl 'https://my-town.sensus-analytics.com/main.html' \
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
-H 'accept-language: en-US,en;q=0.9' \
-H 'cache-control: max-age=0' \
-b cookies.txt \
-c cookies2.txt \
-H 'dnt: 1' \
-H 'priority: u=0, i' \
-H 'referer: https://my-town.sensus-analytics.com/login.html' \
-H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: document' \
-H 'sec-fetch-mode: navigate' \
-H 'sec-fetch-site: same-origin' \
-H 'sec-fetch-user: ?1' \
-H 'upgrade-insecure-requests: 1' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
--silent --output /dev/null ;
curl 'https://my-town.sensus-analytics.com/main.html' \
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
-H 'accept-language: en-US,en;q=0.9' \
-b cookies2.txt \
-c cookies.txt \
-H 'dnt: 1' \
-H 'priority: u=0, i' \
-H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: document' \
-H 'sec-fetch-mode: navigate' \
-H 'sec-fetch-site: none' \
-H 'sec-fetch-user: ?1' \
-H 'upgrade-insecure-requests: 1' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
--silent --output /dev/null ;
curl 'https://my-town.sensus-analytics.com/login.html' \
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
-H 'accept-language: en-US,en;q=0.9' \
-b cookies.txt \
-c cookies2.txt \
-H 'dnt: 1' \
-H 'priority: u=0, i' \
-H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: document' \
-H 'sec-fetch-mode: navigate' \
-H 'sec-fetch-site: none' \
-H 'sec-fetch-user: ?1' \
-H 'upgrade-insecure-requests: 1' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
--silent --output /dev/null ;
curl 'https://my-town.sensus-analytics.com/styles/bootstrap.min.css?v=3.5.12.202404221740' \
-H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
-H 'Referer: https://my-town.sensus-analytics.com/login.html' \
-H 'DNT: 1' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
-H 'sec-ch-ua-platform: "Linux"' \
--silent --output /dev/null ;
curl 'https://my-town.sensus-analytics.com/styles/login.css?v=3.5.12.202404221740' \
-H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
-H 'Referer: https://my-town.sensus-analytics.com/login.html' \
-H 'DNT: 1' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
-H 'sec-ch-ua-platform: "Linux"' \
--silent --output /dev/null ;
curl 'https://my-town.sensus-analytics.com/brand/public/theme?v=3.5.12.202404221740' \
-H 'accept: text/css,*/*;q=0.1' \
-H 'accept-language: en-US,en;q=0.9' \
-b cookies2.txt \
-c cookies.txt \
-H 'dnt: 1' \
-H 'priority: u=0' \
-H 'referer: https://my-town.sensus-analytics.com/login.html' \
-H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: style' \
-H 'sec-fetch-mode: no-cors' \
-H 'sec-fetch-site: same-origin' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
--silent --output /dev/null ;
eval "
curl 'https://my-town.sensus-analytics.com/water/usage/019146-003/78543712?start="${epoch_start}"&end="${epoch_end}"&zoom=custom&page=null&weather=1' \
-H 'accept: application/json, text/javascript, */*; q=0.01' \
-H 'accept-language: en-US,en;q=0.9' \
-b cookies.txt \
-c cookies2.txt \
-H 'dnt: 1' \
-H 'priority: u=1, i' \
-H 'referer: https://my-town.sensus-analytics.com/main.html' \
-H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-origin' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
-H 'x-requested-with: XMLHttpRequest' \
-o waterdata_daily.txt"
eval "
curl 'https://my-town.sensus-analytics.com/water/usage/019146-003/78543712?start="${epoch_start}"&end="${epoch_end}"&zoom=day&page=null&weather=1' \
-H 'accept: application/json, text/javascript, */*; q=0.01' \
-H 'accept-language: en-US,en;q=0.9' \
-b cookies2.txt \
-c cookies.txt \
-H 'dnt: 1' \
-H 'priority: u=1, i' \
-H 'referer: https://my-town.sensus-analytics.com/main.html' \
-H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-origin' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
-H 'x-requested-with: XMLHttpRequest' \
-o waterdata_hourly.txt"
Hi- any chance you’ve made this into an integration?
Unfortunately no, but it is not a personally high priority item. The output text files are json format as the OP discussed, and I am not familiar enough with HA and what method would be best for importing it. I am open to ideas if anyone has done something similar before. I am not sure how to do the authentication outside of my curl method.
I recently had my meter upgraded to use the same service so was poking around on how to integrate it into HA.
I did some reverse-engineering of their APIs and I was able to successfully poll data from my account using the following python script:
#!/usr/bin/python3
import datetime
import requests
import time
base_url = 'https://<login_subdomain>.sensus-analytics.com/'
username = '<login_email>'
password = '<login_password>'
# login
r_sec = requests.post(f'{base_url}/j_spring_security_check', data={'j_username':username, 'j_password':password}, allow_redirects=False)
# query account data
r_acct = requests.post(f'{base_url}/init/init', json={}, cookies=r_sec.cookies)
for account in r_acct.json()['userSettings']['accounts']:
acct_num = account['accountId']
print(f'Account Number: {acct_num}')
print();
for commodity in account['commodities']:
# query meter data
r_meter = requests.post(f'{base_url}/account/details', json={'accountNumber':acct_num, 'meterTypeByValue':commodity}, cookies=r_sec.cookies)
for meter_num in r_meter.json()['deviceIdList']:
print(f'{commodity.capitalize()} Meter: {meter_num} ({r_meter.json()["devices"][meter_num]["state"]})')
# query meter data
r_data = requests.get(f'{base_url}/{commodity}/usage/dailyavg/{acct_num}/{meter_num}', cookies=r_sec.cookies)
print(f'Daily avg: {r_data.json().get("data").get("dailyAvg", {}).get("usage", "-")} {r_data.json().get("data").get("dailyAvg", {}).get("usageUnit", "-")}')
r_data = requests.get(f'{base_url}/{commodity}/usage/multidayavg/{acct_num}/{meter_num}', cookies=r_sec.cookies)
print(f'Weekly avg: {r_data.json().get("data").get("multidayAvg", {}).get("usage", "-")} {r_data.json().get("data").get("multidayAvg", {}).get("usageUnit", "-")}')
r_data = requests.get(f'{base_url}/{commodity}/usage/billingavg/{acct_num}/{meter_num}', cookies=r_sec.cookies)
print(f'Billing avg: {r_data.json().get("data").get("billingAvg", {}).get("usage", "-")} {r_data.json().get("data").get("billingAvg", {}).get("usageUnit", "-")}')
# query usage data
r_data = requests.get(f'{base_url}/{commodity}/usage/{acct_num}/{meter_num}?start={int(time.mktime(datetime.date.today().timetuple())*1000)}&end={int(time.mktime(datetime.date.today().timetuple())*1000)}&zoom=custom&page=null&weather=false', cookies=r_sec.cookies)
print('Usage data for today:')
usageUnit = r_data.json()['data']['usage'][0][0]
for usage in r_data.json()['data']['usage'][1:]:
tick = datetime.datetime.fromtimestamp(usage[0] / 1000.0)
print(f' {tick.strftime("%Y/%m/%d %H:%M")}: {usage[1]} {usageUnit}')
It looks like the primary auth flow is just to call the security check endpoint to get the cookie returned then pass that back into the usage endpoints. There are a few endpoints that return generic information about the accounts and meters associated with a given user.
My account only has a water meter on it, but based on the APIs it looks like technically there are probably gas and electric endpoints as well. Provided they all follow the same API structure this should work for any of them.
This should be enough data to throw together a custom integration with configuration where you provide the endpoint and auth and it can add various meters as devices. Could query the usage endpoint hourly for updates.
Not sure if/when I’ll get time to actually create an integration but wanted to throw this script out there in case I don’t get time and someone else wants to take a stab at it.
I “simplified” @cvpcs’s script above (thanks!) to return only the current water meter reading. Saved the script as /config/water.py:
#!/usr/bin/python3
import datetime
import requests
import time
base_url = 'https://<your_utility>.sensus-analytics.com/'
username = '<your_userid>'
password = '<your_password>'
acct_num = '<your_account>'
meter_num = '<your_meter>'
# login
r_sec = requests.post(f'{base_url}/j_spring_security_check', data={'j_username':username, 'j_password':password}, allow_redirects=False)
#read meter
r_data = requests.post(f'{base_url}/water/widget/byPage', json={"group":"meters","accountNumber":acct_num,"deviceId":meter_num,"minSize":4,"maxSize":4,"touch":"false","desktop":"true","meterCount":2}, cookies=r_sec.cookies)
print(f'{r_data.json().get("widgetList")[0].get("data").get("devices")[0].get("latestReadUsage")}')
Then created a command line sensor to read this (add to configuration.yaml):
command_line:
- sensor:
unique_id: water_<your_meter>
name: water_meter
command: "/usr/local/bin/python /config/water.py"
unit_of_measurement: "gal"
scan_interval: 3600
state_class: total_increasing
device_class: water
My utility only updates the number every 4 hours. So, I run the script hourly (3,600 seconds).
This is not a very “sophisticated” solution, but it’s enough to get the water meter onto my Energy dashboard (I may look at a “proper” integration… eventually)! I’m not sure how consistent the json structures returned by the various utilities are, this one works for Salt Lake City water. You may need to tweak the print() line for your use.
Thanks for this. Got it working for my utility provider in Texas.
I was able to get this to work for my local util that uses that Sensus analytics platform.
One thing to watch for is units. Code above returns the meter reading & labels it “gal”. In my case, I had the Sensus portal configured to display in gallons - but the meter reading turned out to be still in cubic feet. I decided I wanted HomeAssistant to be gallons, so I changed the last line in water.py to
print(f’{float(r_data.json().get(“widgetList”)[0].get(“data”).get(“devices”)[0].get(“latestReadUsage”)) * 7.48}')
(7.48 gallons/cubic foot)
The alternative would have been to change the unit_of_measure to “ft³”.
Thank you so much! I got this working for my city water provider in Georgia 
Hi, gave this a try. It seems to have connected nicely to my water utility. But the sensors don’t appear in the Energy Configuration drop down. From the comments, that might be because the device_class & state_class aren’t set (I think they should be “water” & “total” or “total_increasing”). Also, units; I set the integration to return gallons, but the sensors it created still say CCF. And, again looking at the comments for sensors that the Energy dashboard will allow, it says it wants the units to be “gal” (not G).