Another user has been using my old rest sensor set up for getting alert info from the NWS.
They posted a few days ago that their templates were no longer working correctly post-v118 so I told them I’d take a look. But for the life of me I can’t figure out what’s going on.
Here is the template sensor:
- platform: template
sensors:
nws_test_alert_event_filtered:
friendly_name: NWS Test Alert Event
value_template: >
{% set update = states.sensor.time.state %}
{% if states('sensor.nws_test_alert_event_raw') == 'unavailable' or states('sensor.nws_test_alert_event_raw') == 'unknown' %}
{{ states('sensor.nws_test_alert_event_filtered') }}
{% else %}
{{ states('sensor.nws_test_alert_event_raw') }}
{% endif %}
This is actually the latest iteration of the template but they have all ended in the same end result.
The above template returns the correct result in the template editor but when using it in the actual sensor config it doesn’t return the same (correct) result.
Which I saw before but just thought it was a glitch in the editor/browser.
I have to say the new template renderer has some crappy (un)intended consequences.
It makes everybody need to be a python expert to know exactly what to expect when using these kinds of obscure datatypes.
I still say that my suggestion of defaulting to a string return type (like it did before) unless it was specified otherwise would have made this kind of thing less confusing.
Thanks for the info.
EDIT:
Actually, now that I think about it I’m still not sure why it acted as it did.
Of course you’re right that it was because of the None in the template but I thought that by default all states are supposed to be strings and that new template renderer was only supposed to return the native python types from attributes since those are the only things that can have a data type other than string?
Once I changed the None in the upstream sensor to “No Active Alerts” everything works as expected.
Still…I don’t see why the template sensor didn’t see the state of the raw sensor as a string other than a python data type.
it acted the way it did because you had multiple outputs in the template editor. It resolves the entire template in the template editor. So the entire thing was seen as a string, not each output. So when you delete the ‘extra’ outputs that you had, it shows what your template will actually render as.
Because of what I stated above. It resolves EVERYTHING in your template. It does not resolve output by output.
The “it” i was talking about there wasn’t the result in the template editor. That part worked fine both times based on the screenshots (until after the edit - then it showed nothing at all - so I guess that was the “true” result - but it’s hard to know that based on the feedback on the screen - which was nothing at all.)
I was referring to the results of the template sensor. That’s the one that didn’t work as expected because of the data type thing.
Yes, “it” in your statement is referring to the template editor. I was referring to the results of the template sensor.
According to all the docs that I’ve seen supposedly all states are always strings and will never have any other data type assigned.
the “filtered” sensor was using the state of the “raw” sensor as it’s return value. The state of the “raw” sensor should have been a string (“None”) and not a python data type (None) so the result should have been a string (“None”) not “unknown”.
But apparently it doesn’t work the way the docs/release notes describe it as working. So that needs to be addressed and/or clarified. I could do it but I obviously don’t know enough about native python data types (who would know that None was a data type unless you are a python programmer?) so I don’t know how effective I would be.
I guess I could submit a bug report but again I’m not entirely sure that this isn’t the expected result. Are all states really strings?
I’m not trying to beat a dead horse and I do appreciate the help but I think the docs need to be clarified and I’m just trying to understand where the disconnect in my understanding is since I definitely have a misunderstanding based on something I saw somewhere here.
Yes, so None is what is resolved as the python type which ends up being empty (in the template editor), which ultimately gets converted to unknown (on your sensor).
That’s still true. Your state is ‘unknown’. The template editor does not tell you what your state will be, it’s a tool that shows you the output of the template. States always need a state of some sort, that gets set after the resolution. Basically template gets calculated → template resolves type → special cases occur here. I wouldn’t bet on this being the case for long either. I’m 75% sure that typing will be skipped on states for template sensors in the future. I remember hearing about that. Not sure when it will happen.
Yes they are. Again, the state of your sensor is unknown. You yourself verified that when you accessed it with states.sensor.nws_test_alert_event_filtered.state. That returned a string, ‘unknown’.
I think the disconnect is that you think the type you get in the template editor is the type that sensor returns. It is not. Your input is any type, the output is always a string. The docs are correct on this.
Since the state of the “raw” sensor is “None” then the “{% if …%}” is not satisfied so that part is skipped. So the template returns the “{% else %}” portion. The value of that portion of the template is a string value of “None”.
The template, is for some unknown (to me) reason, converting that state (the string “None”) into a None python data type and returning the result as “unknown”.
I think the disconnect is that input to the “filtered” sensor is a string “None” (from the state of the “raw” sensor) and the “filtered” template is converting it to a different data type. Which I thought (from the description/forum discussion of the release notes) would only happen for attributes that can have any data type.
It seems that the new template renderer automatically assumes that there is a data type (other than string) for every template input no matter the source (state or attribute). So if the value “looks like” a data type other than a string (even tho by definition it is a string since it is a state) then it auto converts it to the data type that it “thinks” it is even tho it’s clearly not in context (because it’s a state and all states are strings by default, always and forever, so saith the docs…amen ).
Where ever templates are, typing occurs. And yes, template sensors convert that back to a string. Is it redundant? Yes. But as I said before, this is already being discussed based on a plethora of issues that stemmed out of the template change.
It’s complicated enough when it’s done in using attributes but using states just adds to it.
I’ll say again I wish they wouldn’t have instantly slapped down my suggestion to use an explicit return type option in the templates. It would have eliminated a lot of the confusion over this.
Well, I just went and looked at the issue that sparked the discussion and there has been no movement for the past month. I wouldn’t hold your breath then. So drop that 75% down to 25%.
I get the reticence in jumping right in to every issue that is brought up but you would think (or at least I would think…and I’ve said this before) that something that has an effect on automations working correctly/as expected would be pretty high on the list of things to get working. as far as I understand it automations are kind of important to at least a couple of home automation systems. I mean it’s pretty much right there in the name of the concept.
Instead the developers spend time/resources integrating the 2000th piece of Iot gear that may be 100 other people might use and let the important stuff to that needs to get fixed for basic languish in limbo.
Well you can always adjust things… Yeah you have to do more work to account for it, but its not like this is a breaking issue. In your case, if you want to keep the word none, do this:
{{"{}\x00".format('None') }}
or if you want to format a number to a specific number of sig figs:
{{ "{:.3f}\x00".format(states('sensor.xyz') | float) }}
^
|
Number of sig figs
By the definition of “breaking changes” in HA this kind of thing and the discussion of the rounding and any number of the other “unintended consequences” of this change meets that definition - things used to work one way but now they don’t with no change to your config.
I’ve never seen either of those forms of the .format function. I’ve seen a similar one to the numeric function ( without the “\x00” portion) but never have seen the alphabetic format used at all. I’ll have to look at that and see what’s going on with all those notations.
but since the template returns an assumed data type from the string in the template (that is no longer a string) as a result of the template rendering wouldn’t it still do the conversion on whatever you have formatted inside the template?
for example, consider the following two templates:
{% if true %}
{{ "None" }}
{% endif %}
-> returns a None data type since it does the conversion after the template is rendered before the result is returned.
{% if true %}
{{"{}\x00".format('None') }}
{% endif %}
-> wouldn't this also return a None data type for the same reason as above - the conversion is done after the template is rendered ( resulting in "None" ) but before the result is returned?
I really have no idea but based on this discussion I would think both of those templates would result in the same returned value.
I think that they ended up making things harder for the majority of people who will likely never use the benefits of the new rendering system and helped a few others who, even they use them a few times, will also be inconvenienced by the changes in many other respects.
‘\x00’ is an invisible string character. Having it in your template output ensures that the value will be a string when it’s typed. You can do it many ways:
I whole heartedly disagree. This makes so many things possible. I’ve already seen many beginners utilize this by templating the entire data section. And that’s just the tip of the iceberg. MQTT publishing is easier as well. Pretty much every aspect of this has gotten easier except for template sensors. Not to mention that most blueprints wouldn’t even work without this.
Well, THAT would have been a good thing to have in the docs or even at least been brought up in the many discussions where people have said “there is no way to force a string return type from a template…”.
How does that (\x00) work where numbers are concerned as in your example above for rounding?
'{:.3f}\x00'.format('1.3245' | float)
\ /
--------- This is the output of the format
the '1.3245' gets converted to a float and
chopped to 1.324 and placed inside the {}.
).