Sort list by variable attribute/sort order in jinja

Hmm… seems I was to quick to announce success… upon careful checking of the results I found strange issue with sorting. It work fine for all attrubutes, but one Release Year. So, here is the final code of sensor (I know it could be done differenctly, as we discussed this here, but I found some other issues, which I will describe later):

template:
  - trigger:
    - platform: state
      entity_id: input_select.library
    - platform: state
      entity_id: input_select.sort_by
    - platform: state
      entity_id: input_number.movies_page
    - platform: state
      entity_id: input_boolean.sort_order
    sensor:
      - name: movies_list_fltr
        state: 'on'
        attributes:
          details: >     
            {% set attr = states('input_select.sort_by') %}
            {% set ord = is_state('input_boolean.sort_order', 'on') %}

            {% set movies_all = state_attr('sensor.movies_list', 'list') | sort(attribute=attr, reverse=ord) %}
            {% set movies_fltr = namespace(movies = '') %}
            {% set counter = namespace(i = 1) %}
            {% set json_string = '' %}
            {%- for movie in movies_all -%}
            {% if movie.Library == states('input_select.library') %}
              {% if counter.i >= (states('input_number.movies_page') | int * 100 - 99) and counter.i < (states('input_number.movies_page') | int * 100 + 1) %}
                {%- set imdbID = '{"imdbID":"' + movie.imdbID + '",' -%}
                {%- set Title = '"Original Title":"' + movie['Original Title'] + '",' -%}
                {%- set PLTitle = '"Polish Title":"' + movie['Polish Title'] + '",' -%}
                {%- set imdbLink = '"imdbLink":"' + movie['imdbLink'] + '",' -%}
                {%- set Rating = '"Rating":"' + movie['Rating'] + '",' -%}
                {%- set Genre = '"Genre":"' + movie['Genre'] + '",' -%}
                {%- set ReleaseYear = '"Release Year":"' + movie['ReleaseYear'] + '",' -%}
                {%- set PblDate = '"Publish Date":"' + movie['Publish Date'] + '",' -%}
                {%- set json_string = json_string+ imdbID + Title + PLTitle + imdbLink + Rating + Genre + ReleaseYear + PblDate +'},' -%}
                {% set movies_fltr.movies = movies_fltr.movies + json_string %}
              {% endif %}
            {% set counter.i = counter.i + 1 %}
            {% endif %}
            {% endfor %}
            {{ movies_fltr.movies }}

and definitions of corresponding helpers:

input_number:
  movies_page: # controls 100 elements page to be displayed
    name: Movies Page to Display
    icon: mdi:book-outline
    min: 1
    max: 1000

input_select:
  library: # selects library to limit number of objects to be parsed
    name: Movies Library
    options:
      - HD Movies
      - SD Movies
      - Music Movies
      - TV Shows

  sort_by: # controls by which attribute to sort the list
    name: Sort by
    options:
      - Publish Date
      - Release Year
      - Original Title
      - Polish Title
      - Rating

input_boolean:
  sort_order: # controls sort order (ascending/descending)
    name: Movies Attribute Sort Order
    icon: mdi:sort
    initial: off

And here is sample value of atribute being processed by this template:

details: 
- imdbID: aa0000001
  Original Title: David Garrett - Rock Symphonies
  Polish Title: "David Garrett: Na żywo, na świeżym powietrzu"
  imdbLink: http://www.imdb.com/title/aa0000001/
  Rating: N/A
  Genre: Music
  Release Year: '2010'
  Publish Date: '2012:07:27:11:06:02'
- imdbID: aa0000012
  Original Title: Jean Michel Jarre - Welcome to the other side
  Polish Title: Jean Michel Jarre - Welcome to the other side
  imdbLink: http://www.imdb.com/title/aa0000012/
  Rating: N/A
  Genre: Music
  Release Year: '2020'
  Publish Date: '2021:01:01:13:11:36'
- imdbID: aa0000004
  Original Title: Schiller - Atemlos
  Polish Title: Schiller - Atemlos Live
  imdbLink: http://www.imdb.com/title/aa0000004/
  Rating: N/A
  Genre: Music
  Release Year: '2010'
  Publish Date: '2017:07:02:09:24:46'
- imdbID: aa0000006
  Original Title: Schiller - Future
  Polish Title: Schiller - Future Live
  imdbLink: http://www.imdb.com/title/aa0000006/
  Rating: N/A
  Genre: Music
  Release Year: N/A
  Publish Date: '2017:07:02:12:54:56'
- imdbID: tt3812422
  Original Title: Schiller - Live in Berlin
  Polish Title: Schiller - Live in Berlin
  imdbLink: http://www.imdb.com/title/tt3812422/
  Rating: N/A
  Genre: Documentary
  Release Year: '2009'
  Publish Date: '2017:01:29:11:12:40'
- imdbID: aa0000011
  Original Title: Schiller - Live in Tehran
  Polish Title: Schiller - Live in Tehran
  imdbLink: http://www.imdb.com/title/aa0000011/
  Rating: N/A
  Genre: Music
  Release Year: N/A
  Publish Date: '2020:04:19:10:00:38'
- imdbID: aa0000008
  Original Title: Schiller - Sehnsucht
  Polish Title: Schiller - Sehnsucht Live
  imdbLink: http://www.imdb.com/title/aa0000008/
  Rating: N/A
  Genre: Music
  Release Year: N/A
  Publish Date: '2017:07:02:13:42:40'
- imdbID: aa0000010
  Original Title: Schiller - Sonne
  Polish Title: Schiller - Sonne Live
  imdbLink: http://www.imdb.com/title/aa0000010/
  Rating: N/A
  Genre: Music
  Release Year: N/A
  Publish Date: '2017:07:01:17:35:40'
- imdbID: aa0000009
  Original Title: Schiller - Symphonia
  Polish Title: Schiller - Symphonia Live
  imdbLink: http://www.imdb.com/title/aa0000009/
  Rating: N/A
  Genre: Music
  Release Year: N/A
  Publish Date: '2020:04:20:00:04:20'
- imdbID: tt3812422
  Original Title: Schiller - Tagtraum
  Polish Title: Schiller - Tagtraum
  imdbLink: http://www.imdb.com/title/tt3812422/
  Rating: N/A
  Genre: Documentary
  Release Year: '2009'
  Publish Date: '2021:03:21:12:05:37'
- imdbID: aa0000007
  Original Title: Schiller - Zeitreise
  Polish Title: Schiller - Zeitreise Live
  imdbLink: http://www.imdb.com/title/aa0000007/
  Rating: N/A
  Genre: Music
  Release Year: N/A
  Publish Date: '2018:05:31:18:28:29'
- imdbID: aa0000002
  Original Title: THePETEBOX
  Polish Title: THePETEBOX
  imdbLink: http://www.imdb.com/title/aa0000002/
  Rating: N/A
  Genre: Music
  Release Year: '2013'
  Publish Date: '2015:02:13:20:09:00'
- imdbID: tt0110758
  Original Title: Pink Floyd - Pulse
  Polish Title: Pink Floyd - Pulse
  imdbLink: http://www.imdb.com/title/tt0110758/
  Rating: '9.3'
  Genre: Documentary, Music
  Release Year: '1994'
  Publish Date: '2023:10:31:00:50:18'
- imdbID: tt2414166
  Original Title: Led Zeppelin - Celebration Day
  Polish Title: Led Zeppelin Celebration Day
  imdbLink: http://www.imdb.com/title/tt2414166/
  Rating: '9.0'
  Genre: Documentary, Music
  Release Year: '2012'
  Publish Date: '2012:12:10:23:40:32'
- imdbID: tt1132456
  Original Title: David Gilmour - Remember That Night
  Polish Title: David Gilmour - Remember That Night
  imdbLink: http://www.imdb.com/title/tt1132456/
  Rating: '8.9'
  Genre: Documentary, Music
  Release Year: '2007'
  Publish Date: '2016:08:08:20:53:24'
- imdbID: tt0271061
  Original Title: Metallica - S&M
  Polish Title: Metallica S&M
  imdbLink: http://www.imdb.com/title/tt0271061/
  Rating: '8.8'
  Genre: Documentary, Music
  Release Year: '2000'
  Publish Date: '2020:04:19:14:03:15'
- imdbID: tt7359702
  Original Title: David Gilmour - Live At Pompeii
  Polish Title: David Gilmour - Na żywo w Pompei
  imdbLink: http://www.imdb.com/title/tt7359702/
  Rating: '8.7'
  Genre: Documentary, Music
  Release Year: '2017'
  Publish Date: '2018:09:01:23:30:40'
- imdbID: tt0069090
  Original Title: Pink Floyd - Live at Pompeii
  Polish Title: Pink Floyd - Live at Pompeii
  imdbLink: http://www.imdb.com/title/tt0069090/
  Rating: '8.7'
  Genre: Documentary, Music
  Release Year: '1972'
  Publish Date: '2020:02:22:14:40:08'
- imdbID: tt2122272
  Original Title: Adele - Live At The Royal Albert Hall
  Polish Title: Adele - Na żywo w Royal Albert Hall
  imdbLink: http://www.imdb.com/title/tt2122272/
  Rating: '8.6'
  Genre: Documentary, Music
  Release Year: '2011'
  Publish Date: '2018:05:31:10:48:37'
- imdbID: tt1388331
  Original Title: Chris Botti - in Boston
  Polish Title: Chris Botti - in Boston
  imdbLink: http://www.imdb.com/title/tt1388331/
  Rating: '8.6'
  Genre: Music
  Release Year: '2009'
  Publish Date: '2018:05:05:17:38:13'
- imdbID: tt0244296
  Original Title: Roger Waters - The Wall Live in Berlin
  Polish Title: Roger Waters - The Wall na żywo w Berlinie
  imdbLink: http://www.imdb.com/title/tt0244296/
  Rating: '8.6'
  Genre: Documentary, Music
  Release Year: '1990'
  Publish Date: '2020:11:06:13:30:52'
- imdbID: tt2122472
  Original Title: Shakira - Live From Paris
  Polish Title: Shakira - Live From Paris
  imdbLink: http://www.imdb.com/title/tt2122472/
  Rating: '8.5'
  Genre: Documentary, Music
  Release Year: '2011'
  Publish Date: '2020:04:20:10:55:30'
- imdbID: tt0097181
  Original Title: Pink Floyd - Delicate Sound Of Thunder
  Polish Title: Pink Floyd - Delicate Sound Of Thunder
  imdbLink: http://www.imdb.com/title/tt0097181/
  Rating: '8.4'
  Genre: Music
  Release Year: '1989'
  Publish Date: '2022:05:14:09:12:05'
- imdbID: tt28814949
  Original Title: Taylor Swift - The Eras Tour
  Polish Title: Taylor Swift - The Eras Tour
  imdbLink: http://www.imdb.com/title/tt28814949/
  Rating: '8.3'
  Genre: Documentary, Music
  Release Year: '2023'
  Publish Date: '2023:12:13:21:31:12'
- imdbID: tt0084503
  Original Title: Pink Floyd - The Wall
  Polish Title: Ściana
  imdbLink: http://www.imdb.com/title/tt0084503/
  Rating: '8.0'
  Genre: Animation, Drama, Fantasy, Music, Musical
  Release Year: '1982'
  Publish Date: '2018:11:19:07:57:47'
- imdbID: tt0214952
  Original Title: Mike Oldfield - Tubular Bells III
  Polish Title: Mike Oldfield - Tubular Bells III
  imdbLink: http://www.imdb.com/title/tt0214952/
  Rating: '7.8'
  Genre: Documentary, Music
  Release Year: '1999'
  Publish Date: '2020:04:13:18:05:49'
- imdbID: tt0075617
  Original Title: ABBA The Movie
  Polish Title: ABBA. Film
  imdbLink: http://www.imdb.com/title/tt0075617/
  Rating: '6.4'
  Genre: Documentary, Comedy, Drama
  Release Year: '1977'
  Publish Date: '2023:07:23:15:44:05'
- imdbID: tt0978039
  Original Title: Loreena McKennitt -  Nights from the Alhambra
  Polish Title: Loreena McKennitt -  Nights from the Alhambra
  imdbLink: http://www.imdb.com/title/tt0978039/
  Rating: '6.4'
  Genre: Music
  Release Year: '2007'
  Publish Date: '2014:04:04:12:50:04'

Now, the problem is, that I can sort by all of the attributes defined in input_select.sort_by, with exception of Release Date. Not only applying this atrribute do not properly order values, but also sort order has no influence on sorting. I made some experiment in Developers tools, changing this parameters manually in template and I can see that some unpredictable sort order is applied only:


Trying to get it resolved, I tried to use your dictionary method to create the sensor, and apply int filter to 1Release Year1, to perhaps made sorting possible:

      - name: movies_fltr
        state: 'on'
        attributes:
          details: > 
            {% set attr = states('input_select.sort_by') %}
            {% set ord = is_state('input_boolean.sort_order', 'on') %}
            {% set lib = states('input_select.library') %}
            {% set movies_all = state_attr('sensor.movies_list', 'list') | sort(attribute=attr, reverse=ord) | selectattr('Library', 'in', lib) %}
            {% set movies_fltr = namespace(movies=[]) %}
            {% for m in movies_all %}
              {% set movies_fltr.movies = movies_fltr.movies + [dict(
                imdbID=m.imdbID,
                imdbLink=m.imdbLink,
                Rating=m.Rating,
                Genre=m.Genre,
                Original_Title=m['Original Title'],
                Polish_Title=m['Polish Title'],
                Release_Year=m.ReleaseYear|int(2000),
                Publish_Date=m['Publish Date']
                )] %}
            {% endfor %}
            {{ movies_fltr.movies }}

It creates almost identical output, the only difference is that Release Year is now integer, not a string. But it did not helped to resolve issue with sorting, still the same problem. Additionally issues I mentioned earlier:

  • is it possible using this method to create json keys with spaces in names? As I use these for filtering in UI it just looks better to have Original Title vs. OriginalTitle or Original-Title or Original_Title.
  • I found no way to apply ‘paging’… Using loop, as in my example I can easily filter objects into groups of 100 items for template output. This is quite serious issue, as in my library I have >6k movies and each of thes takes ~250 characters, leading to total size of unfiltered attribute being +1.5M characters. I know, I should probably use some database for this rather than json file, but this is what I already have and maintain for years :slight_smile:

Any suggestions to this filtering issue? probably something very simple and stupid, but I can’t find what’s wrong…