If you are using a bash or python script to get the data (in JSON format), instead of writing it to a file, why not publish it to an MQTT topic? You can then use an MQTT Sensor to subscribe to the topic (or multiple MQTT sensors, each one parsing out whatever it needs from the received payload).
Simple Python MQTT Publish and Subscribe Example Script
For a shell script, you’ll need to use mosquitto_pub.
Using The Mosquitto_pub and Mosquitto_sub MQTT Client Tools- Examples
Once again, 4th time this thread, it’s NOT JSON it’s xml as per the post at the top of the thread.
However… maybe I can do this via mqtt… it would certainly be more elegant…
mosquitto_pub -h localhost -t test -m "{\"value1\":20,\"value2\":40}"
So that would publish to a topic called test right? (and 2 values) Then in HA I would need to subscribe to it…
OK so I solved this… 123 is getting the credit here for pointing me in the right direction.
My script running on the host via a cronjob every 5 minutes:
#!/bin/bash
curl -c cookie.txt -b cookie.txt -sf -L -d 'login_username=***username***' -d 'login_password=***pass***' --url 'https://my.aussiebroadband.com.au/usage.php?xml=yes' -k -o abbusage.log
# This next bit builds JSON out of the XML
grep "down1" abbusage.log | sed 's/<down1>/{"download":/g' | sed 's/<[/]down1>/,/g' > abbusageedit.log
grep "up1" abbusage.log | sed 's/<up1>/"upload":/g' | sed 's/<[/]up1>/,/g' >> abbusageedit.log
grep "allowance1_mb" abbusage.log | sed 's/<allowance1_mb>/"allowance":/g' | sed 's/<[/]allowance1_mb>/,/g' >> abbusageedit.log
grep "lastupdated" abbusage.log | sed 's/<lastupdated>/"updated":"/g' | sed 's/<[/]lastupdated>/",/g' | sed 's/[ ]/T/g' | sed 's/T"/ "/g' >> abbusageedit.log
grep "rollover" abbusage.log | sed 's/<rollover>/"rollover":/g' | sed 's/<[/]rollover>/}/g' >> abbusageedit.log
# Publish to MQTT Broker
mosquitto_pub -h 10.90.11.100 -u ***username**** -P '***password***' -t abb/abbusage -f abbusageedit.log -r
This then creates 1 sensor with attributes if I include this in config yaml:
sensor:
# ABB Usage
- platform: mqtt
name: "ABB Last Updated"
state_topic: "abb/abbusage"
device_class: timestamp
value_template: "{{ value_json.updated }}"
json_attributes_topic: "abb/abbusage"
Result:
Now onto a lovelace card…
(I’m running the script on the host and I had to install mosquitto-clients package. From memory, the last time I tried a mosquitto_pub command via ssh addon, it is not available)
What?!? XML?!?! How did I not see that!?!
Just kidding. 
As you’ve already discovered, it’s not a hardship to convert XML to JSON.
FWIW, there are python packages to perform the conversion (examples) and, for shell script, there are command-line tools available (like xml2json or yq and others). The key piece of the puzzle was to get the data into Home Assistant and that’s where MQTT makes it easy.
I’m sure but it’s simpler to just do it in the bash script along with the curl command… plus I can just run the one cronjob to update the sensor every 15 minutes… But it was a genius idea to use MQTT - I hadn’t thought of that and it was simple PLUS no issues with permissions and no kludgey multiple text files and text sensors… Would be easier if the file sensor could read multiple lines… but I already use MQTT so it’s a no brainer to leverage that.
Thanks again.
Why not just make this into a custom component? Provide the login info in the config, make the GET request on each update, put each field into a separate sensor or make one sensor with a bunch of attributes.
Because I have no ability to do that and how many people would use it?
I was (originally) thinking along the lines of uses pipes to connect curl to yq to mosquitto_pub.
Something like:
curl <options> | yq <options> | mosquitto_pub <options>
Anyway, it’s moot now because you already have a working script. 
It’s really not that complicated as it’s mostly all boilerplate. You really only need to implement the update method, which in this case is just a GET request and parsing the XML. Honestly, it’s probably less work than what it took to do it this way. Unless, of course, you don’t know Python and don’t want to learn it. Which is understandable. But it does make things like this so much easier.
Anyway, sounds like you got it working so 
I don’t know Python… happy to learn… but /bin/bash…
Fair enough. If you ever decide to learn, this does look like a decent project to start with as it shouldn’t be overly complicated and probably makes sense as a component (even if you’re the only one who would use it).
So I found a better way to do this… still using bash but no longer creating any files. I am running this as a cronjob every 15 minutes but you could make it a shell_command and run an automation to update it on your schedule. You also need mosquitto-clients installed. From memory, this may not be available in a shell_command in hass.io… can’t remember. (it may be available for ssh but that doesn’t mean a shell_command will be able to find/use it)
This is the script:
#!/bin/bash
# Only edit the following 5 lines with correct login information and make sure to wrap in single quotes as shown
abblogin='login_username=*****your abb login here*****'
abbpassword='login_password=*****your abb password here*****'
mqttuser='*****your mqtt user name here*****'
mqttpassword='*****your mqtt password here*****'
mqttbroker=*****your broker ip address here*****
# DO NOT ALTER ANYTHING BELOW THIS LINE
abbusagestring=$(curl -c name=cookie -b name=cookie -sf -L -d $abblogin -d $abbpassword --url 'https://my.aussiebroadband.com.au/usage.php?xml=yes')
zone=$(date +"%:z")
download=$(echo "$abbusagestring" | grep "down1" | sed 's/<down1>//g' | sed 's/<[/]down1>//g')
upload=$(echo "$abbusagestring" | grep "up1" | sed 's/<up1>//g' | sed 's/<[/]up1>//g')
allowance=$(echo "$abbusagestring" | grep "allowance1_mb" | sed 's/<allowance1_mb>//g' | sed 's/<[/]allowance1_mb>//g')
lastupdated=$(echo "$abbusagestring" |grep "lastupdated" | sed 's/<lastupdated>//g' | sed 's/<[/]lastupdated>//g' | sed 's/[. ]/T/g' | sed 's/^[T]/ /g' | awk '{$1=$1};1')
lastupdated=$lastupdated$zone
rollover=$(echo "$abbusagestring" | grep "rollover" | sed 's/<rollover>//g' | sed 's/<[/]rollover>//g')
mqttstring=$( echo -e "{\"usage\":\"\",\"download\":$download,\"upload\":$upload,\"allowance\":$allowance,\"lastupdated\":\""$lastupdated"\",\"rollover\":$rollover}")
# echo "$abbusagestring"
# echo "$zone"
# echo "$download"
# echo "$upload"
# echo "$allowance"
# echo "$lastupdated"
# echo "$rollover"
# echo "$mqttstring"
# Publish to MQTT Broker
mosquitto_pub -h $mqttbroker -u $mqttuser -P $mqttpassword -t abb/abbusage -m "$mqttstring" -r
I also use this in configuration:
# Sensors
sensor:
# ABB Usage
- platform: mqtt
name: "ABB Usage"
state_topic: "abb/abbusage"
icon: mdi:blank
value_template: "{{ value_json.usage }}"
json_attributes_topic: "abb/abbusage"
- platform: template
sensors:
abbupdate:
value_template: "{{ states.sensor.abb_usage.attributes.lastupdated }}"
device_class: timestamp
I am currently reading the last updated as a separate sensor so I can get the datetime in lovelace. If I can get the template working correctly, I’ll likely drop that out.
I’m also planning on calculating the rollover date (not just the number from ABB) which will them mean I can show days left in month and maybe do a bar graph etc… some of the stuff I have in my Rainmeter meter… so I will be updating this.
Here is my lovelace config for the card currently:
- type: 'custom:config-template-card'
variables:
- states['sensor.abb_usage'].state
- states['sensor.abb_usage'].attributes['download']
- states['sensor.abb_usage'].attributes['upload']
- states['sensor.abb_usage'].attributes['allowance']
- states['sensor.abb_usage'].attributes['rollover']
- states['sensor.abb_usage'].attributes['lastupdated']
entities:
- sensor.abb_usage
card:
type: 'custom:hui-entities-card'
title: ABB Usage
show_header_toggle: false
entities:
- entity: sensor.abbupdate
name: "${'Last Updated '}"
icon: mdi:clock
format: datetime
- entity: sensor.abb_usage
name: "${'Rollover Date ' +'\xa0'.repeat(2) + vars[4]}"
icon: mdi:calendar
- entity: sensor.abb_usage
name: "${'Downloaded ' + '\xa0'.repeat(4) + ((vars[1]/1E9).toFixed(2)) + ' GB' }"
icon: mdi:file-download
- entity: sensor.abb_usage
name: "${'Uploaded ' + '\xa0'.repeat(9) + ((vars[2]/1E9).toFixed(2)) + ' GB' }"
icon: mdi:file-upload
- entity: sensor.abb_usage
name: "${'Total ' + '\xa0'.repeat(17) + (((vars[1] +vars[2])/1E9).toFixed(2)) + ' GB of ' + (vars[3]/1E3 < 1E5 ? vars[3]/1E3 : 'Unlimited' + ' GB')}"
icon: mdi:lan-connect
To add yet another possible solution… I can use a webhook to populate the sensor directly!
Ironically, a few days after I started working on this my ISP changed from an XML file to Json!!!
I did this for 2 reasons… mosquitto_pub is not available under a shell_command in Hassio so it won’t run there. I also found basic date commands are similarly unavailable in hassio so this limits you to having to use cron and a script - again, not available for hassio users.
So here is my script:
#!/bin/bash
###### Only edit the following setup lines with correct login information #####
# Aussie Broadband Details
abblogin=******
abbpassword=******
usageid=******
# Home Assistant Config
server=http://ipaddress:8123
token=******
entitypicture=/local/icons/abb/abb.png
###### DO NOT ALTER ANYTHING BELOW THIS LINE ######
# Retrieving ABB Usage Data
cookie="$(mktemp)"
curl -c $cookie -d "username=$abblogin" -d "password=$abbpassword" --url 'https://myaussie-auth.aussiebroadband.com.au/login'
abbusagestring=$(curl -b $cookie --url "https://myaussie-api.aussiebroadband.com.au/broadband/$usageid/usage")
rm $cookie
# Get Variables from String
usedMb=$(echo "$abbusagestring" | jq '.["usedMb"]')
downloadedMb=$(echo "$abbusagestring" | jq '.["downloadedMb"]')
uploadedMb=$(echo "$abbusagestring" | jq '.["uploadedMb"]')
remainingMb=$(echo "$abbusagestring" | jq '.["remainingMb"]')
daysTotal=$(echo "$abbusagestring" | jq '.["daysTotal"]')
daysRemaining=$(echo "$abbusagestring" | jq '.["daysRemaining"]')
lastUpdated=$(echo "$abbusagestring" | jq '.["lastUpdated"]')
# Remove leading and trailing "'s
lastUpdated=$(echo "$lastUpdated" | sed 's/.$//g')
lastUpdated=$(echo $lastUpdated | cut -c2-)
# Calculate Dates
todaydatetime=$(date -Is)
lastUpdatedISO=$(echo "$todaydatetime" |sed "s/${todaydatetime:11:8}/${lastUpdated:11:8}/g")
lastUpdated=$(date -d "$lastUpdated" +%s)
daysRemainingEpochs=$(($daysRemaining*86400))
nextRollover=$(($lastUpdated + $daysRemainingEpochs))
nextRollover=$(date -d @"$nextRollover" -Is)
nextRollover=$(echo "$nextRollover" |sed "s/${nextRollover:11:8}/00:00:00/g")
lastUpdated=$lastUpdatedISO
####### ONLY FOR HASSIO.... COMMENT OUT FOR OTHERS #######
nextRollover=$(echo "$nextRollover" |sed "s/.\{2\}$/:&/")
lastUpdated=$(echo "$lastUpdated" |sed "s/.\{2\}$/:&/")
####### END FOR HASSIO ONLY COMMENTS #######
# Build daysUsed from daysTotal and daysRemaining
daysUsed=$(echo "$(($daysTotal - $daysRemaining))")
# Build JSON output
abbusagestring='{"state":"","attributes":{"usage":"","usedMb":'"$usedMb"',"downloadedMb":'"$downloadedMb"',"uploadedMb":'"$uploadedMb"',"remainingMb":'"$remainingMb"',"daysTotal":'"$daysTotal"',"daysRemaining":'"$daysRemaining"',"lastUpdated":"'"$lastUpdated"'","nextRollover":"'"$nextRollover"'","daysUsed":'"$daysUsed"',"friendly_name":"ABB Usage","icon":"mdi:blank","entity_picture":"'"$entitypicture"'"}}'
# Publish to HA with token
curl -X POST -H "Authorization: Bearer $token" \
-H "Content-Type: application/json" \
-d "$abbusagestring" \
"$server"/api/states/sensor.abb_usage
Then you need an automation (so it populates on HA start and every 15 minutes)…
- id: 'updateabbusage'
alias: Update ABB Usage
trigger:
- event: start
platform: homeassistant
- platform: time_pattern
minutes: '/15'
action:
- service: shell_command.abbusage
initial_state: 'true'
Also need to create the shell_command:
shell_command:
abbusage: /bin/bash /config/abbusage_ha.sh
Lovelace: (Uses Template card and bar cards)
- type: vertical-stack
cards:
- type: 'custom:config-template-card'
variables:
- states['sensor.abb_usage'].state
- states['sensor.abb_usage'].attributes['usedMb']
- states['sensor.abb_usage'].attributes['downloadedMb']
- states['sensor.abb_usage'].attributes['uploadedMb']
- states['sensor.abb_usage'].attributes['remainingMb']
- states['sensor.abb_usage'].attributes['daysTotal']
- states['sensor.abb_usage'].attributes['daysRemaining']
- states['sensor.abb_usage'].attributes['lastUpdated']
- states['sensor.abb_usage'].attributes['nextRollover']
- states['sensor.abb_usage'].attributes['daysUsed']
entities:
- sensor.abb_usage
card:
type: 'custom:hui-entities-card'
title: ABB Usage
show_header_toggle: false
entities:
- entity: sensor.abb_usage
name: "${'Last Update at' + '\xa0'.repeat(3) + new Date(vars[7]).toLocaleDateString('en-AU', {weekday: 'short', year: 'numeric', month: 'short', day: 'numeric'}) + '\xa0' + 'at' + '\xa0' + new Date(vars[7]).toLocaleTimeString('en-AU')}"
- entity: sensor.abb_usage
name: "${'Updated' + '\xa0'.repeat(14) + Number.parseInt((new Date().getTime() - new Date(vars[7])) / 60000) + '\xa0' + 'minutes ago'}"
icon: mdi:clock
- entity: sensor.abb_usage
name: "${var usedinfo= ((vars[5] - vars[6]) + Number.parseFloat(((new Date(vars[7]) - new Date().setHours(0,0,0,0)) / 86400000))).toFixed(2); var usedpercent= ((usedinfo / vars[5])*100).toFixed(2); 'Used' + '\xa0'.repeat(20) + usedinfo + '\xa0' + 'days' + '\xa0'.repeat(2) + '-' + '\xa0'.repeat(2) + usedpercent + '%' + '\xa0' + 'of month'}"
icon: mdi:calendar
- entity: sensor.abb_usage
name: "${var remaininfo= ((vars[5] - vars[9]) - Number.parseFloat(((new Date(vars[7]) - new Date().setHours(0,0,0,0)) / 86400000))).toFixed(2); var remainpercent= ((remaininfo / vars[5])*100).toFixed(2); 'Remaining' + '\xa0'.repeat(10) + remaininfo + '\xa0' + 'days' + '\xa0'.repeat(2) + '-' + '\xa0'.repeat(2) + remainpercent + '%' + '\xa0' + 'of month'}"
icon: mdi:calendar
- entity: sensor.abb_usage
name: "${var usedinfo= ((vars[5] - vars[6]) + Number.parseFloat(((new Date(vars[7]) - new Date().setHours(0,0,0,0)) / 86400000))).toFixed(2); var usedpercent= ((usedinfo / vars[5])*100).toFixed(2); var projected= ((vars[1] / usedpercent) / 10 ).toFixed(2); 'Projected Use' + '\xa0'.repeat(4) + projected + '\xa0' + 'GB'}"
icon: mdi:lan-connect
- entity: sensor.abb_usage
name: "${'Next Rollover' + '\xa0'.repeat(5) + new Date(vars[8]).toLocaleDateString('en-AU', {weekday: 'short', year: 'numeric', month: 'short', day: 'numeric'})}"
icon: mdi:calendar
- entity: sensor.abb_usage
name: "${'Today is Day' + '\xa0'.repeat(6) + ((vars[9] + 1)) + '\xa0'.repeat(2) + 'of' + '\xa0'.repeat(2) + vars[5] + '\xa0'.repeat(2) + 'days'}"
icon: mdi:calendar
- entity: sensor.abb_usage
name: "${'Downloaded' + '\xa0'.repeat(6) + ((vars[2]/1E3).toFixed(2)) + '\xa0' + 'GB' }"
icon: mdi:file-download
- entity: sensor.abb_usage
name: "${'Uploaded' + '\xa0'.repeat(11) + ((vars[3]/1E3).toFixed(2)) + '\xa0' + 'GB' }"
icon: mdi:file-upload
- entity: sensor.abb_usage
name: "${'Total' + '\xa0'.repeat(19) + (((vars[2] +vars[3]) / 1E3).toFixed(2)) + '\xa0' + 'GB of' + '\xa0' + (vars[4] =! 'null' ? vars[4] : 'Unlimited' + '\xa0' + 'GB')}"
icon: mdi:lan-connect
- type: 'custom:bar-card'
title: Month Used
entity: sensor.abb_usage
attribute: daysUsed
min: 1
max: sensor.abb_usage.attributes.daysTotal
height: 20px
direction: right
unit_of_measurement: days
Here’s what the card looks like:
Hope this will help someone else.
You don’t really need to use Curl if you use the rest sensor in HA:
However it looks like your script is doing thing to get a cookie so maybe in this case if it’s working stik with the script. You may find the rest sensor cleaner however if you need more sensors in the future which use curl 
Cant use it.
Rest also won’t take the json from a file…
Usage sets a cookie and then goes back to a different url for the usage.
Thanks for working on this @DavidFW1960 just setup it up and it works great. I would like to try make it into a component at some point
No worries. Fun playing.
Just be aware I found that it didn’t refresh the cookie properly at the half life like it’s supposed to so you may need to run the creds script again after 6 months… (I’m assuming you d/l latest from my github?)
I migrated to Superloop a week ago for a few reasons so I’m not using this for now. Might go back to ABB at some point.
I am trying to get this abbusage working but it doesn’t seem to be logging in by the looks as the abbtoken.json just has {“message”:“Unauthenticated.”} after running it
When I run the creds script all but the usageid is populated it just say null but it is in json. file.
sent a PM.
Hi @DavidFW1960 Maybe you can help me with this script. I use it to record the consumption meter reading. The readout already works. Now I want to import the data into Home Assistant. I’m trying MQTT here. The connection seems to be working. However, the sensor in HA shows me no data. Where could the problem be?
#!/bin/bash
# The values are send to Home Assistant via MQTT
# Smart Meter 0: Logarex LOG4LK13BD102015
mqttuser='mqtt'
mqttpassword='secret_pw'
mqttbroker=localhost
cutValue() {
value=$(echo $1 | cut -d'(' -f 2 | cut -d'*' -f 1)
value=$(echo $value | sed 's/^0*//')
if [ "$value" = ".0000" ]; then
value="0.0"
fi
readLesekopf0() {
stty -F /dev/ttyUSB0 300 -parodd cs7 -cstopb parenb -ixoff -crtscts -hupcl -ixon -opost -onlcr -isig -icanon -iexten -echo -echoe -echoctl -echoke
sleep 1
tmpfile=/tmp/ttyDump1.data
# open Serial Port
cat /dev/ttyUSB0 >$tmpfile &
PID=$!
sleep 1
/bin/echo -e "\x2F\x3F\x21\x0D\x0A" >/dev/ttyUSB0 #Send WAKEUP
sleep 1
/bin/echo -e "\x06\x30\x30\x30\x0D\x0A" >/dev/ttyUSB0 #Send DATA REQUEST
sleep 11
kill $PID
# close Serial Port
while read p; do
# echo $p
case "$p" in
*"1.8.0"*)
# echo 'found 1.8.0'
cutValue $p
mosquitto_pub -h $mqttbroker -u $mqttuser -P $mqttpassword -t strom/zaehlerstand -m "$value" -r
echo $value
;;
*) ;;
esac
done <$tmpfile
}
# call readouts
readLesekopf0 &
# wait until returned
wait %1
exit
sensor.yaml:
- platform: mqtt
name: "Zählerstand"
state_topic: "strom/zaehlerstand"
value_template: "{{ value_json.usage }}"
json_attributes_topic: "strom/zaehlerstand"