josephny
(Joseph)
December 11, 2024, 1:12am
1
I use curl to scrape data from a web site about my propane tank levels.
Every now and then older data disappears.
I have never understand long term stats. Is that what the problem is? Is there an easy/quick change I can make to ensure I have this data?
Here is the code:
command_line:
- sensor:
command: "curl -k -u <user>:<pass> https://ws.otodatanetwork.com/neevoapp/v1/DataService.svc/GetAllDisplayPropaneDevices --header Content-Type:application/json"
name: "Propane Tank Neevo 371 Gallons"
unique_id: propane_tank_371_gallons
value_template: "{{ value_json[0].Level|float(0) * 2.4 if value_json[0].Level is defined else this.state }}"
scan_interval: 14400
state_class: measurement
unit_of_measurement: "Gallons"
# 371 Propane
- sensor:
command: "curl -k -u <user>:<pass> https://ws.otodatanetwork.com/neevoapp/v1/DataService.svc/GetAllDisplayPropaneDevices --header Content-Type:application/json"
name: "Propane Tank Neevo 629 Gallons"
unique_id: propane_tank_629_gallons
value_template: "{{ value_json[1].Level|float(0) * 1.2 if value_json[1].Level is defined else this.state }}"
scan_interval: 14400
state_class: measurement
unit_of_measurement: "Gallons"
# 629 Propane
- sensor:
command: "curl -k -u <user>:<pass> https://ws.otodatanetwork.com/neevoapp/v1/DataService.svc/GetAllDisplayPropaneDevices --header Content-Type:application/json"
name: "Propane Tank Neevo 255 Gallons"
unique_id: propane_tank_255_gallons
value_template: "{{ value_json[3].Level|float(0) * 1.2 if value_json[2].Level is defined else this.state }}"
scan_interval: 14400
state_class: measurement
unit_of_measurement: "Gallons"
# 255 Propane
- sensor:
command: "curl -k -u <user>:<pass> https://ws.otodatanetwork.com/neevoapp/v1/DataService.svc/GetAllDisplayPropaneDevices --header Content-Type:application/json"
name: "Propane Tank Neevo 76 Gallons"
unique_id: propane_tank_76_gallons
value_template: "{{ value_json[4].Level|float(0) * 1.2 if value_json[3].Level is defined else this.state }}"
scan_interval: 14400
state_class: measurement
unit_of_measurement: "Gallons"
#76 Propane
- sensor:
command: "curl -k -u <user>:<pass> https://ws.otodatanetwork.com/neevoapp/v1/DataService.svc/GetAllDisplayPropaneDevices --header Content-Type:application/json"
name: "Propane Tank Neevo 125 Gallons"
unique_id: propane_tank_125_gallons
value_template: "{{ value_json[5].Level|float(0) * 1.2 if value_json[4].Level is defined else this.state }}"
scan_interval: 14400
state_class: measurement
unit_of_measurement: "Gallons"
# 125 Propane
- sensor:
command: "curl -k -u <user>:<pass> https://ws.otodatanetwork.com/neevoapp/v1/DataService.svc/GetAllDisplayPropaneDevices --header Content-Type:application/json"
name: "Propane Tank Neevo 630 Gallons"
unique_id: propane_tank_630_gallons
value_template: "{{ value_json[2].Level|float(0) * 2.4 if value_json[5].Level is defined else this.state }}"
scan_interval: 14400
state_class: measurement
unit_of_measurement: "Gallons"
# ??? Propane
- sensor:
command: "curl -k -u <user>:<pass> https://ws.otodatanetwork.com/neevoapp/v1/DataService.svc/GetAllDisplayPropaneDevices --header Content-Type:application/json"
name: "Propane Tank Neevo 6 Gallons"
unique_id: propane_tank_6_gallons
value_template: "{{ value_json[6].Level|float(0) * 2.4 if value_json[6].Level is defined else this.state }}"
scan_interval: 14400
state_class: measurement
unit_of_measurement: "Gallons"
# ??? Propane
And this is what my current screen looks like. Note missing data from late November, as well as the past couple of day (but only for a few sources):
tom_l
December 11, 2024, 1:32am
2
Are there any errors in Settings → System → Logs related to your recorder database?
Go to Developer Tools → Statistics, are there any FIX ISSUE buttons?
EDIT: what graph card is that (it’s not the core card as that only supports one Y asis)?
It may only support displaying state data, not LTS. Or you may have to configure it to display LTS (like apex charts).
josephny
(Joseph)
December 11, 2024, 1:40am
3
I am using plotly.
Do you think the solution lies with the card?
Thank you!
josephny
(Joseph)
December 11, 2024, 2:26am
5
I’m sure it’s me, and I apologize, but I don’t see anything in that link that explains what I’m experiencing.
tom_l
December 11, 2024, 2:28am
6
Your graph is currently plotting state data which only goes back 10 days by default. To plot statistics data instead (which goes back to whenever the sensor was defined) use the statistic configuration option shown in the link.
1 Like
josephny
(Joseph)
December 11, 2024, 11:27am
7
Wow!
You did it!
It was all a matter of plotly understanding the type of data.
BTW, I read the plotly link and if only it would have included language such as this I might have understood what to do:
“To plot statistics date instead (which goes back to whener the sensor was defined) use the statistic configuration option.”
If only the developer had you write the read.me (
Thank you so very much for your generous help.
I still have one data source that isn’t playing nice: tank_utility.
This sensor (sensor.tank_utility_003200223638383015473830) is created by the tank_utility integration.
DEVELOPER | STATES show it’s details as follows:
And here is the full card showing my propane levels going back as far I’d like:
Thank you again!
EDIT: I checked the entity directly and the data goes back only as far as plotly is showing it. So, maybe this is an LTS issue?
tom_l
December 11, 2024, 8:52pm
8
That sensor has no state_class
and thus is not generating statistics. Which integration did you use to add it to home assistant?
josephny
(Joseph)
December 12, 2024, 11:46pm
9
The integration is TANKUTILITY
tom_l
December 13, 2024, 12:11am
10
So this one?
If so that does not seem to support the state_class
option. You can open an issue for that here: Issues · home-assistant/core · GitHub
However in the meantime you can add the state class using manual customisation, see: Home Assistant Core Integration - Home Assistant
e.g.
# configuration.yaml
homeassistant:
customize:
sensor.tank_utility12345677: # change this to your sensor entity id
state_class: measurement
Then restart HA.
josephny
(Joseph)
December 13, 2024, 9:39am
11
Perfect!
I made the change and it seems that it works:
I added to the plotly card:
statistics: state
period: 5minute
If I use a longer period, the value disappears and in its place is “n/a”
I assume because as a sensor with state_class: measurement, it simply hasn’t existed long enough for an hour or day period to be populated.
Interestingly, if I remove the statistics: and period: from the card, the 7 or so days of data show up.
This is such a great solution for rogue intrgrations.
Thank you again!
tom_l
December 13, 2024, 10:12am
12
josephny:
If I use a longer period, the value disappears and in its place is “n/a”
I assume because as a sensor with state_class: measurement, it simply hasn’t existed long enough for an hour or day period to be populated.
Correct.
Yeah that’s the state data, not statistics. It only goes back 10 days by default.
The developers don’t like people relying on it. You should definitely still open the issue and get the integration fixed.
josephny
(Joseph)
December 13, 2024, 11:56am
13
The problem, as I understand it, with TankUtility, is that Generac bought the small company (TankUtility) that developed the product and changed it substantially and now does not support it.
That means that the HA integration is no longer being worked on.
There are stories out there of people trying to simply change the SSID that the device connects to and winding up with a non-functioning and non-supported device.
One would think that this means there’s an opportunity in the market for a wifi-connected propane tank level sensor. If I was 30 years younger, I might just jump on that.
Thank you again so much for your help, now, and in the past, and for the entire “Cause” (HA, that is).
1 Like
tomayr
(Tomayr)
January 22, 2025, 1:00am
14
Does tank utility actually still work? I am struggling to get it working with my setup and I have a older wifi unit, but my dashboard automatically defaults to app.generacfuel.com when I login.
josephny
(Joseph)
January 23, 2025, 11:42am
15
Sorry for the delay – just seeing this.
Yes, tank utility is working for me.
Config.yaml simply has:
sensors:
- platform: tank_utility
email: "<EMAIL_ADDRESS)"
password: "<PASSWORD>"
devices:
- "<DEVICE_NUMBER>"
I am far from an expert at anything HA, but I hope this helps.