Creating a list from json keys (Rest sensor)

[cklit]

Hi all,

I would like to get a list/array of Radio station names. When I make the rest call in Postman, the json response looks like this:

[
    {
        "order": 0,
        "id": "6336061628069729",
        "title": "DR P1",
        "slogan": "DR P1 er DR's første radiokanal",
        "description": "DR P1 er DR's første radiokanal, hvis historie går helt tilbage til 1925, hvor DR begyndte sin virksomhed. P1 er en ren talekanal med fokus på oplysning; nyheder og dokumentar samt debat-, kultur-, videnskab- og samfundsprogrammer.",
        "popularity": 30458,
        "country": "Denmark",
        "language": "Danish",
        "images": {
            "64x64": "http://static.airable.io/64/02/797233.png",
            "128x128": "http://static.airable.io/82/44/246012.png",
            "256x256": "http://static.airable.io/45/45/402641.png",
            "512x512": "http://static.airable.io/14/37/146822.png",
            "1024x1024": "http://static.airable.io/84/05/782897.png"
        },
        "isAvailable": true,
        "lossless": false,
        "isCustomStation": false
    },
    {
        "order": 1,
        "id": "5207345892675221",
        "title": "DR P6 BEAT",
        "slogan": "",
        "description": "DR P6 BEAT er for de nysgerrige, der gerne vil overraskes. Det er til dem, der gerne vil have mere at vide og gå med bagom musikken. P6 BEAT favner de alternative strømninger - både i den udfordrende og fornyende undergrundsmusik og i mere populære genrer.",
        "popularity": 10902,
        "country": "Denmark",
        "language": "Danish",
        "images": {
            "64x64": "http://static.airable.io/41/03/687570.png",
            "128x128": "http://static.airable.io/69/12/658941.png",
            "256x256": "http://static.airable.io/78/82/218449.png",
            "512x512": "http://static.airable.io/32/27/546736.png"
        },
        "isAvailable": true,
        "lossless": false,
        "isCustomStation": false
    },
    {
        "order": 2,
        "id": "2972408424131572",
        "title": "BBC Radio 2",
        "slogan": "Radio 2 - There's more 2 it!",
        "description": "The home of great music - Pop and Rock from the sixties, seventies, eighties and beyond to blues, big band, country and jazz with the best live music and documentaries.",
        "popularity": 47886,
        "country": "United Kingdom",
        "language": "English",
        "images": {
            "64x64": "http://static.airable.io/35/81/161765.png",
            "128x128": "http://static.airable.io/09/38/859737.png",
            "256x256": "http://static.airable.io/45/89/593079.png",
            "512x512": "http://static.airable.io/54/78/274723.png",
            "1024x1024": "http://static.airable.io/54/42/509775.png"
        },
        "isAvailable": true,
        "lossless": false,
        "isCustomStation": false
    }
]

I’ve created this entity:

rest:
  - resource: https://cloud.bang-olufsen.com/api/v1/netradio/favorites
    verify_ssl: false
    headers:
      Authorization: Bearer <token>
      Content-Type: application/json
    scan_interval: 360
    timeout: 30
    sensor:
      - name: "Beoradio favourites"
        value_template: "{{ now() }}"
        json_attributes:
          - title

But all I get is this:

I only get the title of the first item in the array. I would like it to be an array that I can pass on to a input_select entity using the input_select.set_options call.

I’ve tried many things without luck, so now I’m hoping for some guidance here. Thanks in advance!

[jeffcrum]

Play with your JSON and JSON Path here:
JSONPath Online Evaluator

[cklit]

Thanks!
Using *.title or $.*.title as filter on the evaluator gives the desired result. But unfortunately not in HA - I still just get the first item.

But I dont know if the rest sensor is even capable of using lists as attributes?

[jeffcrum]

It’ll be a couple days before I can play with this. Hopefully someone will come ahead of me and help.

[kbrown01]

Because why are you selecting only the title? Try just ‘$’ or the whole json. That said, you can always use JQ to parse the result into something you want.

[cklit]

Thanks for the tips! Part of it is because I don’t fully understand what the $ means in relation to the json filters.

No matter what I’ve tried, I’ve only successfully pulled out the title from the first item in the array.

Any pointers are welcome, until then, I’ve solved it in Nodered

[Chippie]

I have got the same problem.

This is the JSON:

{
  "producers": [
    {
      "type": "RTSP passive producer",
      "url": "rtsp://127.0.0.1:8554/5356e083d17d4e183d48bba419c8c030",
      "remote_addr": "127.0.0.1:43740",
      "user_agent": "ffmpeg/go2rtc",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=No Name\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2QAM6wVFKCgL/lQ,aO48sA==; profile-level-id=640033\r\na=control:streamid=0\r\nm=audio 0 RTP/AVP 97\r\nb=AS:64\r\na=rtpmap:97 opus/48000/2\r\na=control:streamid=1\r\nm=audio 0 RTP/AVP 98\r\nb=AS:69\r\na=rtpmap:98 MPEG4-GENERIC/16000/1\r\na=fmtp:98 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=140856E500\r\na=control:streamid=2\r\n",
      "medias": [
        "video, recvonly, H.264 High 5.1",
        "audio, recvonly, OPUS/48000/2",
        "audio, recvonly, MPEG4-GENERIC/16000"
      ],
      "receivers": [
        "96 H264, bytes=1945354847, senders=4",
        "97 OPUS/48000/2, bytes=279103127, senders=2",
        "98 MPEG4-GENERIC/16000, bytes=189433925, senders=2"
      ],
      "recv": 2453173155
    }
  ],
  "consumers": [
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://localhost:8554/backyard_live",
      "remote_addr": "127.0.0.1:43588",
      "user_agent": "FFmpeg Frigate/0.13.2-6476f8a",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.8.5\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2QAM6wVFKCgL/lQ,aO48sA==; profile-level-id=640033\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/16000\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=140856E500\r\na=control:trackID=1\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H264, bytes=1945354847, receivers=1",
        "97 MPEG4-GENERIC/16000, bytes=189433925, receivers=1"
      ],
      "send": 2162761924
    },
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://localhost:8554/backyard_live",
      "remote_addr": "127.0.0.1:43692",
      "user_agent": "FFmpeg Frigate/0.13.2-6476f8a",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.8.5\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2QAM6wVFKCgL/lQ,aO48sA==; profile-level-id=640033\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/16000\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=140856E500\r\na=control:trackID=1\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H264, bytes=1945354847, receivers=1",
        "97 MPEG4-GENERIC/16000, bytes=189433925, receivers=1"
      ],
      "send": 2162761924
    },
    {
      "type": "MSE/WebSocket active consumer",
      "remote_addr": "xx.xx.xx.xx, 172.30.32.1:35648",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:124.0) Gecko/20100101 Firefox/124.0",
      "medias": [
        "video, sendonly, H264",
        "audio, sendonly, MPEG4-GENERIC, PCMA, PCMU, L16, PCML, OPUS"
      ],
      "senders": [
        "96 H264, bytes=258187, receivers=1",
        "97 OPUS/48000/2, bytes=33167, receivers=1"
      ],
      "send": 303282
    },
    {
      "type": "WebRTC/WebSocket async passive consumer",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:124.0) Gecko/20100101 Firefox/124.0",
      "medias": [
        "video, sendonly, VP8, RTX, VP9, H264, ULPFEC, RED",
        "audio, sendonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000, TELEPHONE-EVENT/8000, L16, PCML"
      ],
      "senders": [
        "126 H264, bytes=257187, receivers=1",
        "109 OPUS/48000/2, bytes=32576, receivers=1"
      ],
      "send": 294912
    }
  ]
}

Using JSONPath Online Evaluator with consumers.*.remote_addr I got this list:
[
“127.0.0.1:43588”,
“127.0.0.1:43692”,
“xx.xx.xx.xx, 172.30.32.1:35648”
]

Using this rest:

rest:
  - authentication: basic
    scan_interval: 60
    resource: http://xx.xx.xx.xx:1984/api/streams?src=backyard_live
    sensor:
      - name: "Go2rtc consumers backyard_live"
        value_template: "{{ now() }}"
        json_attributes_path: "consumers.*"
        json_attributes:
          - remote_addr

and the attribute is only the first value in the list.

[kbrown01]

I think people get confused (maybe me also) but it is logical that there can only be one attribute with a specific name.

Like you cannot have this:

entity:
   attributes:
       remote_addr: addr1
       remote_addr: addr2

That would be an error of duplicate keys. You can have this:

entity:
   attributes:
       remote_addresses: 
             - addr1
             - addr2

Which is completely valid. Now sometimes the JSON provided to you is not perfect (like for instance you only care about the remote_addr but there are 1000 other things in the structure. For that, use JQ and parse the JSON to what you want (and only what you want). If you do not mind the “extra” information, then grab the parent as the “attribute” and not the item as you are only going to get one.

So in your case, just grab the consumers as the attribute and process later (deeper ) for the remote_addr or restructure the JSON using JQ in a command_line sensor to be how you want it.

Take this as an example:

- sensor: 
      scan_interval: 3600
      name: Russian River Flood Info
      command: "curl -s 'https://www.cnrfc.noaa.gov/graphicalRVF_csv.php?id=GUEC1' | jq --raw-input '{flooddata: [inputs | split(\",\") | {issued: .[0], valid: .[1] | strptime(\"%m/%d/%Y %I %p\") | strftime(\"%m/%d/%Y %I:%M %p\"), validts: .[1] | strptime(\"%m/%d/%Y %I %p\") | strftime(\"%Y-%d-%m %H:%M:%S\"), level: .[2] | tonumber, trend: .[3], status: .[4], mode: .[5] | gsub(\"[\\r]\"; \"\")  }]| .[2:999] }'"
      value_template: >
            {% set levels = namespace(lvl=[]) %}  
            {% for height in value_json.flooddata %}
              {% set levels.lvl = levels.lvl + [height.level] %}
            {% endfor %}
            {{levels.lvl | max }}
      json_attributes:
        - flooddata

This is grabbing a CSV file from the web (could be JSON but in this case is CSV) … parses the file through JQ to restructure the information and change some formatting, and sticks it all into one attribute (flooddata).

Here is another sample where JSON is the input for all apps available in a Vizio Television with all the content reorganized to how it should be accessed (and other content that is relevant and discards the rest):

- sensor:
      scan_interval: 36000
      unique_id: sensor.vizio_apps
      name: Vizio Apps
      command: "curl -s 'http://scfs.vizio.com/appservice/vizio_apps_prod.json' | jq '{\"apps\": [ .[] | {id: .\"id\", name: .\"name\", icon: .\"mobileAppInfo\".\"app_icon_image_url\", sort: .\"mobileAppInfo\".\"featured_sort\" } ]}' "
      value_template: "OK"
      json_attributes:
          - apps

NOTE: Those are command_line sensors so would need to be inserted using command_line: or in a split setup in a separate file and included:

command_line: !include command_line.yaml

[cklit]

Thank you for this thorough explanation, really appreciate it!

Will try to play with it later

[Troon]

This will pull in everything, and then you can extract what you need with template sensors. What data are you looking for?

rest:
  - authentication: basic
    scan_interval: 60
    resource: http://xx.xx.xx.xx:1984/api/streams?src=backyard_live
    sensor:
      - name: "Go2rtc consumers backyard_live"
        value_template: "{{ now() }}"
        json_attributes:
          - producers
          - consumers

Perhaps something like this in a template sensor:

{% set s = states['Go2rtc consumers backyard_live']['attributes'] %}
{{ s['producers']|map(attribute='remote_addr')|select('defined')|list +
   s['consumers']|map(attribute='remote_addr')|select('defined')|list }}

although you might need more work to deal with the comma-separated list of addresses in the MSE/WebSocket item.