Trouble setting up a rest sensor

Configuration
1

I’m trying to set up a rest sensor and getting some unexplained results.

Here is the code:

- platform: rest
  resource: https://api.weather.gov/alerts/active?zone=INZ009
  name: NWS Weather Alert
  value_template: "{{ value_json['features'][0]['properties'] }}"
  json_attributes:
    - event
    - description
    - instructions
    - expires
  headers:
    Accept: application/ld+json
  scan_interval: 30

This is the example curl command from the template creator on the NWS website

curl -X GET "https://api.weather.gov/alerts/active/zone/INZ009" -H "accept: application/ld+json"

If I put the above in the command line this is the result I get:

ex
ex960×122 7.22 KB

and in a chrome browser on windows:

ex2

So the website is returning data correctly.

But when I try to run the sensor I get the following errors in the log and the sensor shows as ‘unknown’:

2018-10-02 06:32:19 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: <HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
 
You don't have permission to access "http&#58;&#47;&#47;api&#46;weather&#46;gov&#47;alerts&#47;active&#63;" on this server.<P>
Reference&#32;&#35;18&#46;150bf748&#46;1538476339&#46;1dfed2c
</BODY>
</HTML>
, template: {{ value_json['features'][0]['properties'] }})

I can’t figure out why the results between the two formats are different.

How can I troubleshoot this?

Another issue that I need to figure out is how to handle when there is no “properties” attribute but I can probably come up with something after I get the “access denied” problem figured out.

Unless the template is what’s causing the problem. Then I need help with why that would be.

2

The main issue is you need to add a User-Agent header. The value can be anything. This is mentioned in their documentation. (It says you need UserAgent, but that didn’t work. So I used curl’s -v option to find what it used and discovered it was User-Agent. That seems to work.)

Next you have to decide on the output format you want (based on the value of Accept.) You used ‘application/ld+json’ with curl and HA, but you didn’t specify with chrome, so you can see the output was different in chrome. These are also documented.

Lastly you need to make sure your value_template and/or json_attributes match the format you specified. E.g., with ‘application/ld+json’ none of the top-level keys you specified exist in the output.

Anyway, at least do this:

- platform: rest
  resource: https://api.weather.gov/alerts/active?zone=INZ009
  name: NWS Weather Alert
  value_template: "{{ value_json.title }}"
  headers:
    User-Agent: homeassistant
    Accept: application/ld+json

Adjust value_template and/or json_attributes as necessary.

3

Thanks for the help on this.

I totally skimmed right over the requirement for authentication. And I realized the results are different based on which “accept” value i select but as long as it returns a fixed json result with the proper header values that I can assign an attribute to I didn’t see any difference between the two. Maybe I’ll have to re-visit those two json options.

the issue I’m going to likely have is that when there are no alerts for my area I get the minimum results above. However, if there are alerts then the result is a lot more extensive and contains all of the attribute values I want. I’ve read that there is a 255 character limit for the sensor value and I’m sure I’ll easily reach that limit if I don’t filter the results somewhat. If I use the value_template to filter the results to the section I want then pull the attributes from there I’m hoping I can pull out what I’ll need.

I just need to figure out how I want to handle the attribute values not existing. I haven’t even started trying to work that out yet.

I’ll definitely give the header a try and see what I get as a result then move forward from there.

Thanks again. I’m sure I’ll have more questions as I work through this. But hopefully not! :grinning:

4

Of course you’ll want to use value_json to get at the attributes. But if the returned result is not valid JSON then value_json won’t exist. Also, if the attributes you’re looking for don’t exist, even when it’s valid JSON, you can’t just try to access them. But you can test for these conditions. So, for example, let’s say you ultimately want value_json.features[0].properties, you can do something like this:

{% if value_json is defined and 'features' in value_json and
      value_json.features|length >= 1 and
      'properties' in value_json.features[0] %}
  {{ value_json.features[0].properties }}
{% else %}
  unknown
{% endif %}
5

Funny enough that was the one thing I have left that I needed help with. The documentation on the sensor in respect to attributes is pretty sparse. I’m not getting anywhere with it.

To clarify would I put that test in the ‘json_attributes:’ definition? Like this:

json_attributes: >
  {% if value_json is defined and 'features' in value_json and value_json.features|length >= 1 and 'properties' in value_json.features[0] %}
    {{ value_json.features[0].properties }}
  {% else %}
    unknown
  {% endif %}

Or is there some sort of template that is necessary that I then access via the json_attribute section?

And can I then create a list of the attributes that I want to parse out using the template above by adding - in front of each template to create a new attribute?

I’m sorry that I have so many questions. Finding accessible documentation on this stuff is next to impossible unless you already know what you are looking for.

Right now as far as I can test this code works for both cases of “features []” and “features [“some_json_code…”]”:

- platform: rest
  resource: https://api.weather.gov/alerts/active?zone=LMZ522
  name: NWS Alert Event
  value_template: >
    {% if value_json.features[0] is defined %}
      {{ value_json['features'][0]['properties'].event }}
    {% else %}
      None
    {% endif %}
  #json_attributes:
  #  - event
  #  - description
  #  - instructions
  #  - expires
  headers:
    User-Agent: Homeassistant
    Accept: application/geo+json
  scan_interval: 30
6

json_attributes only let’s you list top-level keys in the returned JSON. So if you get, say:

{"a": "b", "c": {"d": "e"}}

you can only list a and c, but not d, or even c.d or c['d']. And, of course, they will only “work” if you get valid JSON, and that JSON contains those keys (although if it’s valid JSON, but those keys don’t exist, I think the attributes will still be created, but they’ll have a value of None. But don’t quote me on that. :slight_smile:)

And let’s say with the above JSON, if you do list a & c, then the sensor will have attributes named a & c, and their values will be Python versions of the JSON. So a's value would be the string 'b', and c's value would be the dictionary {'d': 'e'}. And you could access them like {{ state_attr(ENTITY_ID, 'a') }} and {{ state_attr(ENTITY_ID, 'c').d }}.

value_template you use to extract the value from the returned results (using value to access the raw characters of the response, or value_json if the results were valid JSON) that you want to become the state string. And, yes, that is limited to 256 (or 255???) characters.

Bottom line, given the varying results from this website this may be more than the RESTful sensor can (at least easily) do for you, especially if you want to retrieve multiple values.

To answer some of your specific questions:

No, you can’t use a template with json_attributes. As mentioned above, it simply takes a list of JSON top-level keys.

In your solution so far, know that it will fail if the returned results are not valid JSON, or if it is but it doesn’t have the top-level key named 'features'. That’s why I provided a much more explicit set of tests in my suggested template above.

7

Every test I have run so far using the geo+json format has returned the “features” key (either empty or populated) so I don’t think that’s a concern.

It’s unfortunate that the attributes only work with top level keys. I saw a couple of github issues that others raised a while back but never really got whether they were accepted or rejected. It looks like they got rejected. :frowning_face:

I don’t really need the other attributes but it would have been nice to have the extra info on hand.

I could do it by creating more sensors and using the keys as the value but then I get into having to poll the website more trying to get the sensor info and I’m sure they wouldn’t appreciate that too much.

Maybe I’ll try to look into different ways of doing it but for now as long as I have the event name I can trigger a notification from that and it will be fine.

While we’re here do you have any other suggestions for a better way on how I might go about pulling the info into HA?

8

I’m not sure about parsing it, but the NWS does have rss feeds that you could pull in with this custom component

or possibly feedreader to do something with it

9

Awesome. Thanks! I saw the RSS feed too but I didn’t know of anything to do with it.

I’ll look into it.

10

Maybe I wasn’t very clear. Sorry about that. I found a zone with active alerts as an example, so let’s say the returned JSON had these (partial set of) values:

{
    "type": "FeatureCollection",
    "features": [
        {
            "properties": {
                "event": "Flood Warning",
                "description": "River XYZ...",
                "instructions": "Run for the hills!",
                "expires": "2018-10-04T10:23:00-05:00"
            }
        }
    ],
    "title": "Current watches, warnings, and advisories for Some, Place",
    "updated": "2018-10-04T01:25:20+00:00"
}

Then if you did this:

- platform: rest
  resource: https://api.weather.gov/alerts/active?zone=INZ009
  name: NWS Weather Alert
  value_template: "{{ value_json.title }}"
  json_attributes:
    - features
    - updated
  headers:
    User-Agent: homeassistant

Then sensor.nws_weather_alert’s state would be 'Current watches, warnings, and advisories for Some, Place', and it would have two attributes: features and updated. The features attribute would be a list of dictionaries: [{'properties': {'event': 'Flood Warning', 'description': 'River XYZ...', 'instructions': 'Run for the hills!', 'expires': '2018-10-04T10:23:00-05:00'}}]. And the updated attribute would be the string '2018-10-04T01:25:20+00:00'.

So, e.g., let’s say you wanted the event. You could use a template like this:

{{ state_attr('sensor.nws_weather_alert', 'features')[0].properties.event }}

Of course you may need to add checking to make sure the attribute and keys exist as we previously discussed.

1 Like
11

I don’t think I was very clear either so I apologize as well. But I’ve already got a working code (above) that returns the event name. So i can use that for automating alerts.

I was just disappointed that I couldn’t extract the nested keys to extract as attributes. It would have made the notifications more informative. I would have liked to pulled the description and the expiration time.

I think the problem I would run into if I used the entire “features” key as an attribute as you have above I would easily run up against the 255 character limit. (…Unless I’m misunderstanding and there is no issue with an attribute being more than 255 characters…? :thinking: )

the way I have it in my code I don’t have any attributes but the state of the sensor would be “Flood Warning”. And that is the most important information to be able to trigger an alert off of.

playing with the RSS solution above looks promising as well.

Again, thanks for the help. I really appreciate it. :grinning:

12

The string size limit is only for the state string, not for attributes. But, of course, the larger the attributes are, the more space they will take up in the database, so you may still not want to grab the entire features key.

13

Well that’s good to know. I think I can work with that information and the code you posted above to get what I need.

Knowing that actually solves a couple of problems.

That also allows me to grab the alert and if the other sensor I set up to grab the number of active alerts is greater than one then I can pull the second or subsequent alert info out of the attribute too.

Thanks again!

14

I found a location that has two active alerts and I just tried the following:

- platform: rest
  resource: https://api.weather.gov/alerts/active?zone=WYC037
  name: NWS Alert Event 2
  value_template: >
    {% if value_json.features[0] is defined %}
      {{ value_json['features'][0]['properties'].event }}
    {% else %}
      None
    {% endif %}
  json_attributes:
    - features
    - updated
  headers:
    User-Agent: Homeassistant
    Accept: application/geo+json
  scan_interval: 30

and it didn’t end up with the attributes results I was expecting:

ex2
ex2836×122 9.93 KB

Any idea why that shows up like that?

Are the keys below features still available but they just don’t show up in the list of attributes?

I haven’t tried using the attribute in a template to see what happens yet. I should have some time later to try that.

15

Try the following on the Templates page:

{{ state_attr('sensor.nws_alert_event_2', 'features') }}

It could be that the States page will limit the size of a row if the attributes are too big.

16

yes that must be it. I just got a chance to try the template and everything shows up there.

17

@pnbruckner

Now that I’ve had a chance to watch my sensors I’ve created for a bit I have another question I can’t figure out a solution to.

Here is the sensor:

- platform: rest
  resource: https://api.weather.gov/alerts/active/count
  name: NWS Alert Count
  value_template: >
    {% if value_json.zones.INZ009 is defined %}
      {{ value_json.zones.INZ009 }}
    {% elif value_json.zones.INC033 is defined %}
      {{ value_json.zones.INC033 }}
    {% else %}
      0
    {% endif %}
  headers:
    User-Agent: Homeassistant
    Accept: application/ld+json
  scan_interval: 60

For the most part the sensor works but occasionally I get “unavailable” for the sensor state.

From my value template I would think there could only be three possibilities for the state - either the actual count returned by the website for one of those two zones or 0 if those zones are’t defined.

The problem crops up when the website doesn’t return any json value and the website reports an error something like “NGINX ERROR - the website is unavailable. Please try again later.”

I tried the following thinking that it might catch the error and just feed the state back to itself but it doesn’t work either.

{% if value_json is defined %}
  {% if value_json.zones.INZ009 is defined %}
    {{ value_json.zones.INZ009 }}
  {% elif value_json.zones.INC033 is defined %}
    {{ value_json.zones.INC033 }}
  {% else %}
    0
  {% endif %}
{% else %}
  {{states.sensor.nws_alert_count.state}}
{% endif %}

Is there any way to filter the results to eliminate the “unavailable” result?

18

No, not directly. The way the REST Sensor works, if the query fails the state will be unavailable. The only thing I can think of is to create a template sensor and use that to filter the results from sensor.nws_alert_count.

19

I kind of figured that was the case.

It was pretty straight forward to come up with a template filter but I was hoping to have something a little cleaner.

Thanks again.

20

I guess it depends on what you do with sensor.nws_alert_count. If you just display it, then I would think seeing unavailable sometimes wouldn’t be a huge issue. If you use its value in other sensors, automations or scripts, then you could always handle the unavailable state there. E.g., if you ever did something like states('sensor.nws_alert_count')|int, you could easily change that to states('sensor.nws_alert_count')|int(0) which would result in a value of zero if the state was ever unavailable. (Actually, turns out the int filter will default to zero anyway if the “input” can’t be converted to a number, so you wouldn’t even have to explicitly state a default value of 0.)