Iterating through json

Configuration
1

Dear all,

I’m trying to create an MQTT template sensor to display the connected hosts on my home network in HA. I prepared a script that sends data (generated by nmap and converted to JSON using ) via MQTT in JSON format. This part is working fine.

However, in the Developer Tools - Template editor I can’t seem to extract the correct information from it: Since the ‘nmap’ output is not uniform for each host, I need to do some validation on them. Here is a sample JSON and my Jinja script (which is not working):


{%
set myData = 
{
    "host": [
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.5",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "64:64:4A:E2:DF:94",
                    "@addrtype": "mac"
                }
            ],
            "hostnames": null,
            "times": {
                "@srtt": "894",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.10",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:15:99:85:84:A5",
                    "@addrtype": "mac",
                    "@vendor": "Samsung Electronics"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "Samsung.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "664",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.82",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "B8:27:EB:DB:E3:34",
                    "@addrtype": "mac",
                    "@vendor": "Raspberry Pi Foundation"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "chopin.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "728",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.222",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:24:8C:66:7F:54",
                    "@addrtype": "mac",
                    "@vendor": "Asustek Computer"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "test.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "675",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "localhost-response",
                "@reason_ttl": "0"
            },
            "address": {
                "@addr": "192.168.1.111",
                "@addrtype": "ipv4"
            },
            "hostnames": {
                "hostname": {
                    "@name": "havm.local",
                    "@type": "PTR"
                }
            }
        }
    ]
}

%}

{% for host in myData.host %}

{% if host.hostnames.hostname is defined %}
{% set hostName = host.hostnames.hostname['@name'] %}
{% else %}
{% set hostName = "-" %}
{% endif %}

{% for address in host.address %}
  {% set a = address | first %}
  {{ a }}
{% endfor %}

{{ hostName }} - {{ ipv4Address | default ("***") }}

{% endfor %}

The specific problem is that most hosts have their ip address in a list format, i.e.:

            "address": [
                {
                    "@addr": "192.168.1.222",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:24:8C:66:7F:54",
                    "@addrtype": "mac",
                    "@vendor": "Asustek Computer"
                }
            ],

but the last one is just this much:

            "address": {
                "@addr": "192.168.1.111",
                "@addrtype": "ipv4"
            },

How can I test for, say, the type of myData.host[x].address?

Also: why does this work in itself (in Template editor):

{{ myData.host[3].address[0]['@addr'] }}
{{ myData.host[4].address['@addr'] }}

but not when iterating through them later on?

Any pointer to the right direction is much appreciated!!

2

Are you certain that the JSON payload you posted above is correct? Because I tested it in several online JSON validators and all of them reported errors with the payload’s structure.

Screenshot from 2022-07-04 13-08-43
Screenshot from 2022-07-04 13-08-431640×395 50.5 KB

Screenshot from 2022-07-04 13-11-24
Screenshot from 2022-07-04 13-11-24776×887 67.2 KB

Screenshot from 2022-07-04 13-12-26
Screenshot from 2022-07-04 13-12-26618×932 52.5 KB

3

There seems to be an issue here:

            "times": {
                "@srtt": "664",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },        }, // Here
                "@to": "100000"
            }
        },
        {
4

Well, yes, my mistake: It is a much larger JSON, and I stripped it down (evidently not carefully enough…) for easier handling here…

The correct one is here:

{
    "host": [
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.5",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "64:64:4A:E2:DF:94",
                    "@addrtype": "mac"
                }
            ],
            "hostnames": null,
            "times": {
                "@srtt": "894",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.10",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:15:99:85:84:A5",
                    "@addrtype": "mac",
                    "@vendor": "Samsung Electronics"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "Samsung.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "664",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.82",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "B8:27:EB:DB:E3:34",
                    "@addrtype": "mac",
                    "@vendor": "Raspberry Pi Foundation"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "chopin.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "728",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.222",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:24:8C:66:7F:54",
                    "@addrtype": "mac",
                    "@vendor": "Asustek Computer"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "test.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "675",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "localhost-response",
                "@reason_ttl": "0"
            },
            "address": {
                "@addr": "192.168.1.111",
                "@addrtype": "ipv4"
            },
            "hostnames": {
                "hostname": {
                    "@name": "havm.local",
                    "@type": "PTR"
                }
            }
        }
    ]
}

Also, I corrected it in the original post…

5

This worked with the latest sample data you provided. Try it and let me know if it needs further adjustments:

{% for x in myData.host %}
{{ x.hostnames.hostname['@name'] if x.hostnames is defined else '-' }} - {{ x.address[0]['@addr'] if x.address is not mapping else x.address['@addr'] }}
{% endfor  %}

EDIT

Correction. Bad copy-paste from my test example which contained value_json.host and I replaced it with myData to accomodate the author’s original example. However, it should be myData.host.

6

Thanks for looking into this.

I’m afraid, I’m getting the following error with your proposed script:

UndefinedError: 'str object' has no attribute 'address'

For reference (and to make sure I didn’t mess it up again), here is the full script from the Template designer:

{%
set myData = 
{
    "host": [
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.5",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "64:64:4A:E2:DF:94",
                    "@addrtype": "mac"
                }
            ],
            "hostnames": null,
            "times": {
                "@srtt": "894",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.10",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:15:99:85:84:A5",
                    "@addrtype": "mac",
                    "@vendor": "Samsung Electronics"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "Samsung.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "664",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.82",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "B8:27:EB:DB:E3:34",
                    "@addrtype": "mac",
                    "@vendor": "Raspberry Pi Foundation"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "chopin.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "728",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.222",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:24:8C:66:7F:54",
                    "@addrtype": "mac",
                    "@vendor": "Asustek Computer"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "test.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "675",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "localhost-response",
                "@reason_ttl": "0"
            },
            "address": {
                "@addr": "192.168.1.111",
                "@addrtype": "ipv4"
            },
            "hostnames": {
                "hostname": {
                    "@name": "havm.local",
                    "@type": "PTR"
                }
            }
        }
    ]
}

%}

{% for x in myData %}
{{ x.hostnames.hostname['@name'] if x.hostnames is defined else '-' }} - {{ x.address[0]['@addr'] if x.address is not mapping else x.address['@addr'] }}
{% endfor  %}
7

Also, if I remove the offending part, i.e. the ip address section, even the hostnames are not displayed correctly:

{% for x in myData %}
{{ x.hostnames.hostname['@name'] if x.hostnames is defined else '-' }} 
{% endfor  %}

This renders only a single -, doesn’t even iterate through all the items.

8

Found it!

This is the correct script:

{% for x in myData.host %}
{{ x.hostnames.hostname['@name'] if x.hostnames is defined else '-' }} - {{ x.address[0]['@addr'] if x.address is not mapping else x.address['@addr'] }}
{% endfor  %}

Note the .host in the first line.

Thanks for helping me!! I know I am not a Jinja guru, but I have created fairly complex scripts in the past, but I was not aware of the mapping keyword.
Thank you, much appreciated!!!

1 Like
9

There’s another test named iterable which is indicates if the value can be looped through with a for-loop (like a list or string).

1 Like
10

Awesome, good to know, thank you very much!!!