RESTful Sensor with complex return value_json

Spurious · January 27, 2023, 10:30pm

I have the following JSON response from a webservice:

{
	"success": "true",
	"dates_data": [
		{
			"zip": "8005",
			"date": [
				"19. Juni 2023",
				"05. Juni 2023",
				"15. Mai 2023",
				"29. April 2023",
				"17. April 2023",
				"03. April 2023",
				"13. März 2023",
				"27. Februar 2023",
				"13. Februar 2023",
				"30. Januar 2023"
			],
			"town": "Zürich"
		}
	],
	"_8400": "false",
	"msg": "."
}

The dates are oredered the other way around and they are in German with the format %d. %B %Y.
What I want now is to sort the array by date value ascending at take the first value.
The end result is basically like this: {{ value_json.dates_data[0].date[0] }}, I just need to have the date array sorted properly. Is there a way to accomplish this?

michaelblight · January 28, 2023, 3:11am

If you’re just wanting the last entry, you don’t need to sort - you can use “[-1]” as the negative sign for Jinja makes it work backwards from the end. If you need to convert this to a date object, you can use “strptime” as in:

{{ strptime( value_json.dates_data[0].date[-1], '%d. %B %Y') }}

Note that I tested this on an entity in Dev Tools (ie. not a REST sensor) and used English.

vingerha · January 28, 2023, 7:50am

If you want to sort the list in ‘date’ then the main challenge will be the German part. I cannot test this on my machine as pure English

{% set my_test_json = [
				"19. June 2023",
				"05. June 2023",
				"15. May 2023",
				"29. April 2023",
				"17. April 2023",
				"03. April 2023",
				"13. May 2023",
				"27. February 2023",
				"13. February 2023",
				"30. January 2023"
			] %}
{% set ns = namespace(input=[], output=[]) %}
{% for x in my_test_json %}
{% set ns.input = ns.input + [(strptime(x,"%d. %B %Y"))] %}
{% endfor %}
{% for x in (ns.input | list | sort) %}
{% set ns.output = ns.output + [x.strftime("%d %B %Y")] %}
{% endfor %}
{{ ns.output }}

will give you

[
  "30 January 2023",
  "13 February 2023",
  "27 February 2023",
  "03 April 2023",
  "17 April 2023",
  "29 April 2023",
  "13 May 2023",
  "15 May 2023",
  "05 June 2023",
  "19 June 2023"
]

If it it only reversing the order then …this may give you a lead

{% set my_test_json = [
                "19. Juni 2023",
				"05. Juni 2023",
				"15. Mai 2023",
				"29. April 2023",
				"17. April 2023",
				"03. April 2023",
				"13. März 2023",
				"27. Februar 2023",
				"13. Februar 2023",
				"30. Januar 2023"
			] %}
{% for x in my_test_json | list | reverse %}
{{ x }}
{% endfor %}

gives you below, you can use a similar approach as above ton construct a array if you want

30. Januar 2023

13. Februar 2023

27. Februar 2023

13. März 2023

03. April 2023

17. April 2023

29. April 2023

15. Mai 2023

05. Juni 2023

19. Juni 2023

Spurious · January 28, 2023, 7:50am

Excellent hint with [-1], I was not aware. So, this is at least a starting point. I still need to convert it. I will try to play around with it some more. But I receive the following error when I test it in the developer tools template sandbox:
ValueError: Template error: strptime got invalid input '30. Januar 2023' when rendering template '{{ strptime( states('sensor.mr_green'), '%d. %B %Y') }}' but no default was specified

michaelblight · January 28, 2023, 8:00am

Maybe you’re not getting German. Try this:

{{ strptime('20. January 2023', '%d. %B %Y', 'English failed') }}
{{ strptime('20. Januar 2023', '%d. %B %Y', 'German failed') }}

For me, English works but German fails - which is expected here in Australia.

Spurious · January 28, 2023, 8:02am

Yes, sorry, it fails for the language reasons. I am not an expert in the Jinja stuff.
I think I am not too far off with my current formula:

{% set months = ['Januar', 'Februar', 'März', 'April', 'Mai', 'Juni', 'Juli', 'August', 'September', 'Oktober', 'November', 'Dezember'] %}
{% set intmonth = months.index('Januar') + 1 %} {{intmonth}}
{{ strptime( states('sensor.mr_green'), '%d. %m %Y') }}

I now need to replace the month with the numeric value and it should work, I think.

vingerha · January 28, 2023, 9:44am

Interesting topic this conversion, have had it in the past which led to something like this
Maybe one of the jinja experts will tear this down to something more efficient

{% set months = ['Januar', 'Februar', 'März', 'April', 'Mai', 'Juni', 'Juli', 'August', 'September', 'Oktober', 'November', 'Dezember'] %}
{% set x = ("19. Februar 2023").split(" ")[0].replace('.','') %}
{% set y = ("19. Februar 2023").split(" ")[1]  %}
{% set z = ("19. Februar 2023").split(" ")[2]  %}
{% set inmonth = months.index(y) + 1 %} 
{% set date = x ~ '-'  ~ inmonth ~  '-' ~ z  %}
{{strptime(date, "%d-%m-%Y")}}

Spurious · January 28, 2023, 10:09am

This looks crazy, I will see if it works. Thanks a lot!

Spurious · January 28, 2023, 10:15am

And I made it, thanks a lot for the help!!

{% set months = ['Januar', 'Februar', 'März', 'April', 'Mai', 'Juni', 'Juli', 'August', 'September', 'Oktober', 'November', 'Dezember'] %}
{% set month = states('sensor.mr_green').split(" ")[1]  %}
{% set intmonth = months.index(month) + 1 %}
{{ strptime( states('sensor.mr_green').replace(month, intmonth|string), '%d. %m %Y') }}

vingerha · January 28, 2023, 10:31am

Nice job !
My example was needed in another solution where I had to build up the date from scratch…the ‘replace’ would have reduced that too!

Spurious · January 29, 2023, 8:12pm

Maybe I get another helping hand with my YAML. I am trying to get this into a conditional card. The template card that I omitted at the end works fine standalone. But with the following, it does not work and I cannot understand the error message:

type: vertical-stack
cards:
  - show_current: true
    show_forecast: true
    type: weather-forecast
    entity: weather.wohnung_michele_jonas
    name: Wetter
  - type: conditional
    conditions:
      - entity: sensor.mr_green
        state_not: unknown
      - entity: {% set days_left = (states('sensor.mr_green') | as_datetime - states('sensor.date')|as_datetime).days %} {% if days_left <3 %} true {% else %} false {% endif %}
        state_not: 'true'
    card:
      type: custom:mushroom-template-card
     ...

The error message reads:

missed comma between flow collection entries (12:18)

  9 |     conditions:
 10 |       - entity: sensor.mr_green
 11 |         state_not: unknown
 12 |       - entity: {% set days_left = (states('sens ...
-----------------------^
 13 |         state_not: 'true'
 14 |     card:

vingerha · January 30, 2023, 11:21am

I am not able to verify this till Thu but…
I think the issue is that you are trying to put a template instead of the entity which is most cards is not possible. Also not sure why the ‘entity’ needs to be templated, it is usually the state…
You can try to circumvent this by installing card_templater (hacs) but no guarantee

Spurious · January 30, 2023, 11:42am

Ok, I thought I could use a template as well. A workaround as such would be to create a template sensor and use it instead, right?

Troon · January 30, 2023, 12:06pm

Shorter, not sure about efficiency. The bottom two lines do the heavy lifting — the first bit is just for the template editor. I assume strptime needs English month name abbreviations, which is what my replace statements achieve:

{% set value_json = {
	"success": "true",
	"dates_data": [
		{
			"zip": "8005",
			"date": [
				"19. Juni 2023",
				"05. Juni 2023",
				"15. Mai 2023",
				"29. April 2023",
				"17. April 2023",
				"03. April 2023",
				"13. März 2023",
				"27. Februar 2023",
				"13. Februar 2023",
				"30. Januar 2023"
			],
			"town": "Zürich"
		}
	],
	"_8400": "false",
	"msg": "."
} %}
{% set dc = (value_json['dates_data'][0]['date'][-1]|regex_findall("^(\d\d)\.\ ([A-Z]\w\w).*(\d{4})$"))[0] %}
{{ strptime(dc[0]~dc[1]|replace('ä','a')|replace('i','y')|replace('k','c')|replace('z','c')~dc[2],"%d%b%Y",default="unknown") }}

Spurious · January 30, 2023, 12:29pm

Thank you very much, I thought about the letter replace route as well. The last step to a one-liner would be reducing the second regex to three letters. But this is good enough, thanks a lot!

Troon · January 30, 2023, 1:01pm

Shorter, and without the replaces — adaptable for any language with unique three-letter month abbreviations:

{% set d,m,y=(value_json['dates_data'][0]['date'][-1]|regex_findall("^(\d\d)\.\ ([A-Z]\w\w).*(\d{4})$"))[0] %}
{{ as_datetime('%s-%0.2d-%s'%(y,'JanFebMärAprMaiJunJulAugSepOktNovDez'.index(m)//3+1,d)) }}

Alternatively, a one-liner for German only, with @Spurious’s sensor name in place of the value_json:

{{ strptime(states('sensor.mr_green').translate({228:97,105:121,107:99,122:99})|regex_replace('^(\d\d)\.\ (\w{3}).*(\d{4})$','\\1\\2\\3'),'%d%b%Y') }}

vingerha · January 30, 2023, 1:17pm

I am slightly (?) loosing it… donot get me wrong, it is great that it can be this short but …wow

Troon · January 30, 2023, 1:24pm

I know the first one looks obtuse and a bit code-golfy, but it’s not that hard (and not as fun as my recent favourite solution). The first line pulls out a list (tuple, strictly) like ("30","Jan","2023") from the string with a regex match.

The second line formats that into a yyyy-mm-dd string, with a lookup to get the month number out of the German abbreviations, then turns it into a datetime object.

The one-liner template is a compacted version of my first one, using the not-very-well-documented translate function instead of the sequence of replaces.

Yes, that is the solution, not just a workaround. Template binary sensor, and note that they are 'on' or 'off' in condition tests.

vingerha · January 30, 2023, 1:50pm

… was already wanting to ask about your avatar name and golf … thanks for the ‘links’

Troon · January 30, 2023, 1:55pm

Unrelated, as it happens.