So looking at the API, there are ways to trim the information, but everything returns a list, which shoots us in the foot basically.
Ooh this is a fun one. Ok so my understanding is that you’re calling this API:
https://api.tfl.gov.uk/Line/Mode/dlr,elizabeth-line,overground,tube/Status
And want a sensor with its state set to a count of the number of lineStatuses in all the items in the response that have something other then Good Service in their statusSeverityDescription description field. And you also want an attribute which has all the lines and a list of disruptions for each one.
So as noted above you’re going to have challenges trying to do this with a REST sensor. Fortunately, we have curl and jq if we use the shell. Instead of a rest sensor we’ll use a command line sensor that curls the response and then manipulates it into a way that works better for HA like this:
- platform: command_line
name: London lines disruptions
command: >-
curl https://api.tfl.gov.uk/Line/Mode/dlr,elizabeth-line,overground,tube/Status 2> /dev/null
| jq '{"data":[.[] | {"name": .name, "lineStatuses": [.lineStatuses[] | {"disruption": .statusSeverityDescription, "name": .lineId}]}]}'
value_template: >-
{{ value_json.data
| map(attribute="lineStatuses")
| map('rejectattr', 'disruption', 'eq', 'Good Service')
| map('list') | map('count')
| select('gt', 0) | list | count }}
json_attributes:
- data
What I do here is first wrap their array response in an object. This way I can stash an attribute containing all the data returned. I also use jq to strip out everything we don’t need (basically just keeping the name and the list of disruptions). Then the template uses rejectattr within a map to turn each item into a count of actual disruptions and sums them up, filters out the lines without a disruption and counts the remaining ones .
I just tested this and got 10. Let me know if that’s wrong.
Also the attribute will contain all disruptions including Good Service (which isn’t actually a disruption). JQ can be used to filter out Good Service and just leave the actual disruptions its just more work. Let me know if its necessary.
1 Like
well, you don’t actually need to do the curl to make this easier. The api has a direct disruptions call, but it doesn’t list the lines. So simply…
- platform: rest
resource: https://api.tfl.gov.uk/Line/Mode/dlr,elizabeth-line,overground,tube/Disruption
name: TfL All Lines Status
value_template: >
{{ value_json | count }}
But he’s looking for the name of the line and the disruption as well.
It’s always bothered me that the rest integration doesn’t have attribute templates. And having that (like template sensors do) would solve this problem.
The rest sensor isn’t built for a list as the first object, which makes json_attributes 100% useless.
3 Likes
Ok like the lineId from within lineStatuses? Ok I’ll modify to add that
Yea I know, it’s very annoying. I basically always just use command_line sensors with curl and jq instead. I rarely can make rest sensor work as is.
Ah, I see what you did with the command line, you mapped it to data. That would totally solve this.
1 Like
Yep exactly. When the API returns a top-level list, jq to the rescue!
1 Like
Thanks, I really appreciate the help here!
@CentralCommand
That almost works brilliantly. One small point, It should be value_template not state.
More importantly, it does count the lines with disruption wrongly. It looks like it is returning the count of disruptions. As said above somewhere one line can have more than one disruption.
I’m not a Linux person so curl and (especially) jq are new to me but now I have something to go on.
I do also agree with @petro about not being able to template the attributes.
Whoops, my bad
Oh I think I misunderstood. So you want a count of lines that have at least one disruption in lineStatuses, not a count of the number of disruptions in lineStatuses across all lines.
Ok going to adjust to account for those two in a moment, give it a shot
Not quite there…
TypeError: object of type 'generator' has no len()
1 Like
Ugh. I admit I didn’t test that one first. I really don’t understand why jinja makes you call list before count. Other reducing filters like sum handle generators just fine but not the most common one. Annoying. Anyway fixed that.
2 Likes
Perfect!
Thank you so much, both of you.
Now I just have go and have a read about what jq actually does!!
1 Like
jq is your absolute friend. Look at this crazy template I use to reorganize JSON from the insteon integation:
- platform: command_line
scan_interval: 30
name: insteon_groups
command: "jq '{ groups: [ .[0].\"address\" as $modemaddress | .[] | .\"address\" as $device_address | select(.\"address\" != $modemaddress) | .aldb | .[] | .target=$modemaddress | { group: .\"group\", in_use: .\"in_use\", device_address: $device_address, target: .\"target\", controller: .\"controller\", brightness: .\"data1\", ramp_rate: .\"data2\", button: .\"data3\"} ] | sort_by(.group) | map(select(.\"group\" > 20)) | map(select(.in_use))}' insteon_devices.json"
value_template: "{{ now() }}"
json_attributes:
- groups
You can do everything you want and put it in attributes.
I’ve been playing with jq.
I agree… it is very powerful!!
One quick question.
Do command line sensors have a deafult scan_interval?
The docs say so but mine don’t seem to be updating by themselves.
should be 30 seconds IIRC
I just looked at the source and I think it defaults to 60 secs.
SCAN_INTERVAL = timedelta(seconds=60)
I did a restart rather than just reloading all my shiny new command sensors and I think they are updating now. But I have to catch a change in disruptions to be sure.
Does it sound plausible that reloading sensors doesn’t force a scan interval but an HA restart does?
Reloading should work, but it might take the full scan interval for the sensor to update.
I can’t believe I’m back here but see my very first statement…
Now I’ve had a good go with jq I am thinking I’d be better off with a single sensor for each line.
I want the state to be a concatenation of all the disruptions giving something like:
Minor Delays + Part Suspended
But is this really the best way to do it?
(Copy it into the Dev Tools Template page if you want to 
)
{% set value_json =
{
"data": [
{
"name": "London Overground",
"lineStatuses": [
{
"disruption": "Minor Delays",
"reason": "London Overground: Minor delays between Euston and Watford Junction due to an earlier track fault at Hatch End. GOOD SERVICE on all other routes. "
},
{
"disruption": "Part Suspended",
"reason": "London Overground: Minor delays between Euston and Watford Junction due to an earlier track fault at Hatch End. GOOD SERVICE on all other routes. "
}
],
"validityPeriods": [
{
"fromDate": "2022-08-12T14:34:40Z",
"toDate": "2022-08-13T00:29:00Z"
},
{
"fromDate": "2022-09-12T14:34:40Z",
"toDate": "2022-09-13T00:29:00Z"
}
]
}
]
}
%}
{% set disruption_count = value_json.data
| map(attribute="lineStatuses")
| map('selectattr', 'disruption')
| map('list')
| map('count')
| sum %}
{#
Getting the above disruption_count can be reduced to this line when not using the test data
{% set disruption_count = states('sensor.tfl_london_overground') | int %}
It is the next six lines that seem a bit clumsy to me.
#}
{% set ns = namespace(disruptions=[]) %}
{% for x in range(0, disruption_count) %}
{% set disruption = value_json.data[0].lineStatuses[x].disruption %}
{% set ns.disruptions = ns.disruptions + [ disruption ] %}
{% endfor %}
{{ ns.disruptions | list | join(' + ') }}
Is this really the best way to do this?
(My money is on ‘no’ but I’ve been looking at this for far too long)
If you use the rest integration rather than the rest sensor platform you can make as many sensors as you want from the one call to the resource.
Each sensor can have its own value template (unlike what is shown in the rather poor example that uses mainly attributes).
Pretty sure he can’t because the rest integration doesn’t support listed data going into attributes