Efficient way to filter json

Configuration
,
1

I have a command_line sensor with attribute output of below…

{
    "f07903c4e39c4d2f8d4ea3ac900fb6bb": {
      "config": {
        "opt": "on"
        },
      "switch": {
        "start": "",
        "notified": ""
        }
    },
    "a31203c4e39c4d2f8d4ea3ac900fb6bb": {
       "config": {
        "opt": "off"
        },
      "switch": {
        "start": "",
        "notified": ""
        }
    },
    ...
}

Need to know if start is not empty. Currently loop through each entry to test. Attempted to find a solution using select, selectattr, …, having difficulty.

What is the most efficient way to filter this?

2

Hi GAHA,

Does this converter help you at all?
Templating - Home Assistant.

3

This will be true if any of the “start” keys have a value that is not an empty string:

{{ data.values() | map(attribute='switch.start') | select('ne', '') | list | count > 0 }}
1 Like
4

Absolutely. Not so much in this particular case though.

5

This helps, thanks! Some more context for what the objective is…

{
    "f07903c4e39c4d2f8d4ea3ac900fb6bb": {
      "config": {
        "notify": "4567776771",
        "opt": "on"
        },
      "switch": {
        "start": "",
        "notified": ""
        }
    },
    "a31203c4e39c4d2f8d4ea3ac900fb6bb": {
       "config": {
        "notify": "4565576771",
        "opt": "off"
        },
      "switch": {
        "start": "2024-08-11 10:24:56-05:00",
        "notified": ""
        }
    },
    ...
}

Need to check all the switch.start to see if start + 60 mins > now. If true, I need to get the value of config.notify to use in sending a notification.

6

This will give you a list of all the notify values that correspond to the start value being more recent than 60 minutes ago.

Note that I’m comparing datetimes as strings, which will only be valid if they are the exact same format and same timezone. If you need to compare as datetime objects you’ll need to loop through them.

{% set time_compare = (now().replace(microsecond=0) + timedelta(hours=-1)) | string %}
{{ data.values() | selectattr('switch.start','gt', time_compare) | map(attribute='config.notify') | list  }}
1 Like
7

This…

{% set time_compare = (now().replace(microsecond=0) + timedelta(hours=-1)) | string %}
{% set devices = state_attr('sensor.devices', 'devices') %}

Gives me this…

{
    "f07903c4e39c4d2f8d4ea3ac900fb6bb": {
      "config": {
        "notify": "4567776771",
        "opt": "on",
        "location": "1"
        },
      "switch1": {
        "start": "",
        "notified": ""
        },
      "switch2": {
        "start": "",
        "notified": ""
        }
    },
    "a31203c4e39c4d2f8d4ea3ac900fb6bb": {
       "config": {
        "notify": "4565576771",
        "opt": "off",
        "location": "2"
        },
      "switch1": {
        "start": "2024-08-11 10:24:56-05:00",
        "notified": ""
        },
      "switch2": {
        "start": "",
        "notified": ""
        }
    },
    ...
}

This…

{{ devices.values() | selectattr('switch1.start','gt', time_compare) | map(attribute='switch1') | rejectattr('notified', '!=', '') | list  }}

Gives me this when the notified key is empty.

[{'start': '2024-08-12 00:58:21-05:00', 'notified': ''}]

Problem is I need to do the test the opposite way ie selectattr(time_compare, 'gt', 'switch1.start') which doesn’t work.

This is how I do it in the automation loop…

- variables: 
    start: "{{ state_attr('sensor.devices', 'devices')[id]['switch1' if mode == 'sw1' else 'switch2']['start'] }}"
    cycle: "{{ ((now() | as_timestamp - as_datetime(start) | as_timestamp) / 60) | round(2) }}"
    notified: "false"
- choose:
    - conditions: "{{ cycle > 60 and not notified }}"
      sequence:

My objective is to check if "start": "2024-08-11 10:24:56-05:00" to now()is beyond an hour. I need to send a notification to "notify": "4567776771" if "notified": "" then set "notified": "true"

8

You’re going to have to stop moving the goalposts if you actually want a useful answer.

How many switch keys are there for each main entry? Does that number vary? What is the mode variable, and do you want something done with it?

Also, a single line template will return a single result. It could be a dict or list, but it’s a single result nonetheless. Setting the same variable to a value for each result requires looping through each result by definition.

Well which is it? start + 60 min > now is the same thing as start > now - 60 min which is the code I provided. now - 60 min > start is the opposite of what you requested the first time.

TL;DR: If you can define the variability of the incoming data, and you can provide a dict or list that you want as the template’s output, then I can try to help. I’m not going to keep playing along creating templates to solve problems that you don’t need solved.

1 Like
9

Totally agree.

2 switch keys. Does not vary.

mode maps sw1 and sw2 to switch1 and switch2

Objective…

  • send notification if start to now has been on for more than an hour and set notified key to true

Also, is it possible to get the main key from this? ie f07903c4e39c4d2f8d4ea3ac900fb6bb and a31203c4e39c4d2f8d4ea3ac900fb6bb

{{ devices.values() | selectattr('switch1.start','gt', time_compare) | list  }}
10

At the end of the day you’re going to need to use a loop since you don’t know how many results will come back requiring notifications. So I don’t see how a single line template is going to help you out.

If you still wanted a single line template anyway, my guess as to what you’d want this template to output (since you haven’t specified) would be a dictionary with each main item that matches as the key, and then the notify number as the value.

Here’s what I’m using as input to my template, since I don’t have your sensor and I am trying to include the various possibilities:

Click here to show template input 
{% set devices = 
{
    "f07903c4e39c4d2f8d4ea3ac900fb6bb": {
      "config": {
        "notify": "4567776771",
        "opt": "on",
        "location": "1"
        },
      "switch1": {
        "start": "",
        "notified": ""
        },
      "switch2": {
        "start": "",
        "notified": ""
        }
    },
    "a31203c4e39c4d2f8d4ea3ac900fb6bb": {
       "config": {
        "notify": "4565576771",
        "opt": "off",
        "location": "2"
        },
      "switch1": {
        "start": "2024-08-12 12:54:56-05:00",
        "notified": ""
        },
      "switch2": {
        "start": "",
        "notified": ""
        }
    },
    "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": {
       "config": {
        "notify": "1111111111",
        "opt": "off",
        "location": "2"
        },
      "switch1": {
        "start": "",
        "notified": ""
        },
      "switch2": {
        "start": "2024-08-12 13:04:56-05:00",
        "notified": ""
        }
    },
    "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb": {
       "config": {
        "notify": "2222222222",
        "opt": "off",
        "location": "2"
        },
      "switch1": {
        "start": "",
        "notified": ""
        },
      "switch2": {
        "start": "2024-08-12 12:24:56-05:00",
        "notified": ""
        }
    }
}
%}

{% set time_compare = (('2024-08-12 14:00:00-05:00' | as_datetime).replace(microsecond=0) + timedelta(hours=-1)) | string %}

Note that essentially it means now() is always assumed to be 2024-08-12 14:00:00-05:00

And if that is the input, this is what I’m assuming you’d want as the output:

{
    "a31203c4e39c4d2f8d4ea3ac900fb6bb": "4565576771",
    "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb": "2222222222"
}

But that result is not possible without looping anyway (at least not until the zip() PR is approved).

If you merely wanted a list of the relevant notifiers (which would be ['4565576771', '2222222222'] from the template input specified earlier) then that would be possible with this template:

{{ devices.values() | rejectattr('switch1.start','eq','') | selectattr('switch1.start','lt', time_compare) | map(attribute='config.notify') | list + 
  devices.values() | rejectattr('switch2.start','eq','') | selectattr('switch2.start','lt', time_compare) | map(attribute='config.notify') | list  }}

In the end I think what you should do is post your automation and ask for help simplifying it. I don’t expect the answer to be a single template, but there may be template improvements (or other logic improvements) that you can make.

11

Thanks. I’ll play around with it to try to get a better understanding of what’s going on.

Agreed. I’ll do that as I’m sure there are possible improvements.

12

A couple pointers to get yourself better at templating json:

  • Basics:
    • A dictionary is made up of keys and values like so: {"key1": "value1", "key2": "value2"}
    • A dictionary can contain strings, integers, lists, or other dictionaries
    • Keys must be unique, values don’t need to be
  • selectattr() will allow you to select only the keys which match the values specified
  • mydict.values() | list will return all the values of mydict in a list
  • mydict.keys() | list will return all the keys of mydict in a list
  • mydict.items() | list will return the dictionary in a list of key, value tuples
  • map(attribute='myattribute') will take a list of objects and return only the value of the key named myattribute for each of them. You will lose all the other data when doing this.
1 Like