Iterating through json

Dear all,

I’m trying to create an MQTT template sensor to display the connected hosts on my home network in HA. I prepared a script that sends data (generated by nmap and converted to JSON using ) via MQTT in JSON format. This part is working fine.

However, in the Developer Tools - Template editor I can’t seem to extract the correct information from it: Since the ‘nmap’ output is not uniform for each host, I need to do some validation on them. Here is a sample JSON and my Jinja script (which is not working):


{%
set myData = 
{
    "host": [
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.5",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "64:64:4A:E2:DF:94",
                    "@addrtype": "mac"
                }
            ],
            "hostnames": null,
            "times": {
                "@srtt": "894",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.10",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:15:99:85:84:A5",
                    "@addrtype": "mac",
                    "@vendor": "Samsung Electronics"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "Samsung.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "664",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.82",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "B8:27:EB:DB:E3:34",
                    "@addrtype": "mac",
                    "@vendor": "Raspberry Pi Foundation"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "chopin.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "728",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.222",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:24:8C:66:7F:54",
                    "@addrtype": "mac",
                    "@vendor": "Asustek Computer"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "test.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "675",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "localhost-response",
                "@reason_ttl": "0"
            },
            "address": {
                "@addr": "192.168.1.111",
                "@addrtype": "ipv4"
            },
            "hostnames": {
                "hostname": {
                    "@name": "havm.local",
                    "@type": "PTR"
                }
            }
        }
    ]
}

%}

{% for host in myData.host %}

{% if host.hostnames.hostname is defined %}
{% set hostName = host.hostnames.hostname['@name'] %}
{% else %}
{% set hostName = "-" %}
{% endif %}

{% for address in host.address %}
  {% set a = address | first %}
  {{ a }}
{% endfor %}

{{ hostName }} - {{ ipv4Address | default ("***") }}

{% endfor %}   

The specific problem is that most hosts have their ip address in a list format, i.e.:

            "address": [
                {
                    "@addr": "192.168.1.222",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:24:8C:66:7F:54",
                    "@addrtype": "mac",
                    "@vendor": "Asustek Computer"
                }
            ],

but the last one is just this much:

            "address": {
                "@addr": "192.168.1.111",
                "@addrtype": "ipv4"
            },

How can I test for, say, the type of myData.host[x].address?

Also: why does this work in itself (in Template editor):

{{ myData.host[3].address[0]['@addr'] }}
{{ myData.host[4].address['@addr'] }}

but not when iterating through them later on?

Any pointer to the right direction is much appreciated!!

Are you certain that the JSON payload you posted above is correct? Because I tested it in several online JSON validators and all of them reported errors with the payload’s structure.

There seems to be an issue here:

            "times": {
                "@srtt": "664",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },        }, // Here
                "@to": "100000"
            }
        },
        {

Well, yes, my mistake: It is a much larger JSON, and I stripped it down (evidently not carefully enough…) for easier handling here…

The correct one is here:

{
    "host": [
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.5",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "64:64:4A:E2:DF:94",
                    "@addrtype": "mac"
                }
            ],
            "hostnames": null,
            "times": {
                "@srtt": "894",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.10",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:15:99:85:84:A5",
                    "@addrtype": "mac",
                    "@vendor": "Samsung Electronics"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "Samsung.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "664",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.82",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "B8:27:EB:DB:E3:34",
                    "@addrtype": "mac",
                    "@vendor": "Raspberry Pi Foundation"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "chopin.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "728",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.222",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:24:8C:66:7F:54",
                    "@addrtype": "mac",
                    "@vendor": "Asustek Computer"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "test.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "675",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "localhost-response",
                "@reason_ttl": "0"
            },
            "address": {
                "@addr": "192.168.1.111",
                "@addrtype": "ipv4"
            },
            "hostnames": {
                "hostname": {
                    "@name": "havm.local",
                    "@type": "PTR"
                }
            }
        }
    ]
}

Also, I corrected it in the original post…

This worked with the latest sample data you provided. Try it and let me know if it needs further adjustments:

{% for x in myData.host %}
{{ x.hostnames.hostname['@name'] if x.hostnames is defined else '-' }} - {{ x.address[0]['@addr'] if x.address is not mapping else x.address['@addr'] }}
{% endfor  %}

EDIT

Correction. Bad copy-paste from my test example which contained value_json.host and I replaced it with myData to accomodate the author’s original example. However, it should be myData.host.

Thanks for looking into this.

I’m afraid, I’m getting the following error with your proposed script:

UndefinedError: 'str object' has no attribute 'address'

For reference (and to make sure I didn’t mess it up again), here is the full script from the Template designer:

{%
set myData = 
{
    "host": [
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.5",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "64:64:4A:E2:DF:94",
                    "@addrtype": "mac"
                }
            ],
            "hostnames": null,
            "times": {
                "@srtt": "894",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.10",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:15:99:85:84:A5",
                    "@addrtype": "mac",
                    "@vendor": "Samsung Electronics"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "Samsung.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "664",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.82",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "B8:27:EB:DB:E3:34",
                    "@addrtype": "mac",
                    "@vendor": "Raspberry Pi Foundation"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "chopin.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "728",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "arp-response",
                "@reason_ttl": "0"
            },
            "address": [
                {
                    "@addr": "192.168.1.222",
                    "@addrtype": "ipv4"
                },
                {
                    "@addr": "00:24:8C:66:7F:54",
                    "@addrtype": "mac",
                    "@vendor": "Asustek Computer"
                }
            ],
            "hostnames": {
                "hostname": {
                    "@name": "test.local",
                    "@type": "PTR"
                }
            },
            "times": {
                "@srtt": "675",
                "@rttvar": "5000",
                "@to": "100000"
            }
        },
        {
            "status": {
                "@state": "up",
                "@reason": "localhost-response",
                "@reason_ttl": "0"
            },
            "address": {
                "@addr": "192.168.1.111",
                "@addrtype": "ipv4"
            },
            "hostnames": {
                "hostname": {
                    "@name": "havm.local",
                    "@type": "PTR"
                }
            }
        }
    ]
}

%}

{% for x in myData %}
{{ x.hostnames.hostname['@name'] if x.hostnames is defined else '-' }} - {{ x.address[0]['@addr'] if x.address is not mapping else x.address['@addr'] }}
{% endfor  %}

Also, if I remove the offending part, i.e. the ip address section, even the hostnames are not displayed correctly:

{% for x in myData %}
{{ x.hostnames.hostname['@name'] if x.hostnames is defined else '-' }} 
{% endfor  %}

This renders only a single -, doesn’t even iterate through all the items.

Found it!

This is the correct script:

{% for x in myData.host %}
{{ x.hostnames.hostname['@name'] if x.hostnames is defined else '-' }} - {{ x.address[0]['@addr'] if x.address is not mapping else x.address['@addr'] }}
{% endfor  %}

Note the .host in the first line.

Thanks for helping me!! I know I am not a Jinja guru, but I have created fairly complex scripts in the past, but I was not aware of the mapping keyword.
Thank you, much appreciated!!!

1 Like

There’s another test named iterable which is indicates if the value can be looped through with a for-loop (like a list or string).

1 Like

Awesome, good to know, thank you very much!!!