New sensor for alerts from GeoRSS feeds

Share your Projects!
13

If set to true the sensor will publish georss incidents retrieved from the external service directly onto the HA event bus. You can then write an automation that does not get triggered by changes of the sensor, but just needs to listen to those events.
This approach comes with the advantage that the fired event contains all the information you may want to send yourself in a notification. Also, you can process each incident separately, and for example use conditions to further filter.

Correct, the sensor’s state just represents the number of active incidents.

With the above mentioned HA event bus approach, I see two options:

  1. Create a sensor that retrieves all earthquakes, regardless of magnitude. The action gets triggered by events fired by the sensor; you create a condition that interrogates the event data where you should find the magnitude as an attribute. Then you can forward all relevant information in a notification.
  2. Create a sensor that retrieves all earthquakes but filters by magnitude, so that it will only send events to the HA event bus that are relevant for notifications. The action’s trigger will still be the same but you won’t need a condition since the events are already all the ones you want to send yourself a notification for.

I don’t have a configuration example like this at hand at the moment, but I can probably post one tonight.

14

And here comes an example of an event listener automation:

automation:
  - alias: Notify
    trigger:
      platform: event
      event_type: nsw_fire_service
    action:
      service: persistent_notification.create
      data_template:
        message: '{{ trigger.event }}'
        title: "NSW Fire Service"

There was recently a bug fix that made it into version 0.57 and fixed that event_data is really optional. So make sure you use 0.57 or later.
The value for event_type must match whatever your sensor generated from the sensor name. Please have a look at your logfile to see what type of events are being generated:

INFO:homeassistant.core:Bus:Handling <Event nsw_fire_service[L]: ...>

The above automation simply generates a notification box in the UI. For some reason I can’t explain right now, the message from the action is not displayed, and instead if just says “notifying”.

15 
2017-11-12 12:19:24 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.geo_rss_events
2017-11-12 12:19:24 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] latitude=41.9376, longitude=12.4557, url=http://info.terremoti.ingv.it/feed/atom/all_week, radius=200.0
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] 491 entri(es) available in feed http://info.terremoti.ingv.it/feed/atom/all_week
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (42.2572, 13.5737): 98.85116202219044 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 10:23:24 UTC -  Magnitude(ML) 1.6  -  NW Fagnano Alto (AQ)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=10, tm_min=23, tm_sec=24, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 98.85116202219044, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17623641', 'magnitude': '1.6'} does not match filter OrderedDict([('attribute', 'magnitude'), ('regexp', '([0-9]{2,}|[2-9]{1})\\.\\d{1}')])
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (42.8062, 13.0782): 109.28564019806099 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 09:52:23 UTC -  Magnitude(ML) 1.2  -  2 km NW Norcia (PG)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=9, tm_min=52, tm_sec=23, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 109.28564019806099, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17623351', 'magnitude': '1.2'} does not match filter OrderedDict([('attribute', 'magnitude'), ('regexp', '([0-9]{2,}|[2-9]{1})\\.\\d{1}')])
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (42.842, 13.1918): 117.33504609388073 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 09:39:02 UTC -  Magnitude(ML) 1.6  -  7 km SE Castelsantangelo sul Nera (MC)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=9, tm_min=39, tm_sec=2, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 117.33504609388073, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17623251', 'magnitude': '1.6'} does not match filter OrderedDict([('attribute', 'magnitude'), ('regexp', '([0-9]{2,}|[2-9]{1})\\.\\d{1}')])
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (42.8167, 13.15): 113.17108167555972 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 09:26:12 UTC -  Magnitude(ML) 1.3  -  5 km NE Norcia (PG)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=9, tm_min=26, tm_sec=12, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 113.17108167555972, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17623181', 'magnitude': '1.3'} does not match filter OrderedDict([('attribute', 'magnitude'), ('regexp', '([0-9]{2,}|[2-9]{1})\\.\\d{1}')])
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (42.7957, 13.1262): 110.17486426602294 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 09:09:24 UTC -  Magnitude(ML) 1.0  -  3 km E Norcia (PG)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=9, tm_min=9, tm_sec=24, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 110.17486426602294, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17623121', 'magnitude': '1.0'} does not match filter OrderedDict([('attribute', 'magnitude'), ('regexp', '([0-9]{2,}|[2-9]{1})\\.\\d{1}')])
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (43.0298, 13.0358): 130.43021111886017 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 08:59:26 UTC -  Magnitude(ML) 1.1  -  2 km SW Pieve Torina (MC)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=8, tm_min=59, tm_sec=26, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 130.43021111886017, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17622951', 'magnitude': '1.1'} does not match filter OrderedDict([('attribute', 'magnitude'), ('regexp', '([0-9]{2,}|[2-9]{1})\\.\\d{1}')])
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (42.8207, 13.1237): 112.4852197854774 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 08:48:52 UTC -  Magnitude(ML) 1.1  -  4 km NE Norcia (PG)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=8, tm_min=48, tm_sec=52, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 112.4852197854774, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17622911', 'magnitude': '1.1'} does not match filter OrderedDict([('attribute', 'magnitude'), ('regexp', '([0-9]{2,}|[2-9]{1})\\.\\d{1}')])
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (37.759, 14.819): 506.4934995129277 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (38.4352, 15.4398): 464.6068052752786 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (43.0122, 13.0): 127.55560744491149 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 07:29:54 UTC -  Magnitude(ML) 0.9  -  2 km N Monte Cavallo (MC)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=7, tm_min=29, tm_sec=54, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 127.55560744491149, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17622391', 'magnitude': '0.9'} does not match filter OrderedDict([('attribute', 'magnitude'), ('regexp', '([0-9]{2,}|[2-9]{1})\\.\\d{1}')])
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (42.8052, 13.1325): 111.34638502865567 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Keeping event {'category': None, 'title': '2017-11-12 06:45:06 UTC -  Magnitude(ML) 2.6  -  3 km E Norcia (PG)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=6, tm_min=45, tm_sec=6, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 111.34638502865567, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17622041', 'magnitude': '2.6'}
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (43.6207, 10.9992): 221.7034245046353 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (42.8132, 13.1318): 112.08772374482871 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 05:45:54 UTC -  Magnitude(Md) 1.1  -  4 km NE Norcia (PG)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=5, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 112.08772374482871, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?eventId=17621851', 'magnitude': '1.1'} does not match filter OrderedDict([('attribute', 'magnitude'), ('regexp', '([0-9]{2,}|[2-9]{1})\\.\\d{1}')])
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Distance from [41.9376, 12.4557] to (42.9158, 13.0432): 118.97948658899703 km
2017-11-12 12:19:31 DEBUG (SyncWorker_7) [custom_components.sensor.geo_rss_events] Event {'category': None, 'title': '2017-11-12 04:55:49 UTC -  Magnitude(ML) 0.8  -  4 km N Preci (PG)', 'date_published': None, 'date_updated': time.struct_time(tm_year=2017, tm_mon=11, tm_mday=12, tm_hour=4, tm_min=55, tm_sec=49, tm_wday=6, tm_yday=316, tm_isdst=0), 'distance': 118.97948658899703, 'id': 'smi:webservices.ingv.it/fdsnws/event/1/query?

,

16

can you help in automation that tells if a erarthquake is registered (above X magnitude)

17

I have additional problem, how to have less logs from your component, its making the log too big. Few days and is 2 GByte!

18

No worries, I’ll see what I can do:

First of all, have you activated publishing data to the event bus?
Please add publish_events: true to your existing sensor:

sensor:
  - platform: geo_rss_events
    ...
    publish_events: true

The log you posted does not contain any of the data we’re looking for. It’s not the sensor itself that logs the event, but HA core. If you used the earthquake example from above, the sensor’s name is “Terremoti” which would be turned into the event type terremoti.

INFO:homeassistant.core:Bus:Handling <Event terremoti[L]: ...>

Now to the automation: Have you set up the sensor to only include earthquakes with a magnitude of 3 or higher? If not, I would recommend doing that because it makes the automation easier as it won’t require a condition.

  - alias: 'Attenzione Terremoto'
    initial_state: 'on'
    trigger:
      - platform: event
        event_type: terremoti
    action:
      - service: notify.claudio_pushbullet
        data_template:
          message: 'Attenzione Terremoto: {{ trigger.event }}'
      - service: notify.claudio_zanzito
        data_template:
          message: 'Attenzione Terremoto'

You are already using the data_template approach in your action which allows you to extract more details from the event that triggered the automation. I don’t have a good example for a nicer message yet, but basically you can access all details of the event such as title, date, distance, and custom attributes.

19

Well, it appears that you log at debug level. Have a look at the logger documentation which explains how you can set log levels for different components. Looking at your log file, I think you want to set custom_components.sensor.geo_rss_events to info.

20

how you do this?

21

As explained in a previous post, you will have to create a regular expression that matches the desired criterion. Unfortunately it’s not as easy as comparing two numbers with a “greater than” condition. Maybe in a future version this could be enhanced to allow for generating typed custom attributes and corresponding typed custom filters.

Anyway, in the meantime you would need to add the following bit to your sensor configuration. The custom attribute definition extracts a new attribute magnitude from the title of the georss feed. The custom filter then matches the magnitude against the provided regular expression which looks for anything larger than 3.0.

    custom_attributes:
      - name: magnitude
        source: title
        regexp: '.*Magnitude\(M.+\) (?P<custom_attribute>\d+.\d+) .*'
    custom_filters:
      - attribute: magnitude
        regexp: '([0-9]{2,}|[3-9]{1})\.\d{1}'

To test if your regular expression works you could use a service like pythex:

regexp-tester
regexp-tester888×529 13.7 KB

22

Hi @exxamalte,
Are you planning to update your existing sensor with this new version?
It has more features and could handle some sources, e.g. Copernicus, it nicer way as far as I understood.

23

Yes, absolutely. Unfortunately I still haven’t found the time to write more thorough unit tests for the new functionality. I’ll try my best to get this done within the next week or so.

Have you tried the development version? Do you have any feedback for me - does it work as expected, is it relatively easy to configure the regular expressions, is there still anything missing for the GeoRSS sources you are interested in?

24

I’ve tried to replace the original code with a new one, however, faced with permission issue.
Can I install it as a customer component?
I definitely would be happy to provide you with my feedback.

1 Like
25

That should be possible. However, because the default HA installation already contains a component with the same name, it would probably be best to rename the new file in <config_dir>/custom_components/sensor/, e.g. to geo_rss_events_new.py. Then accordingly you would use platform: geo_rss_incidents_new in your configuration.

26

FYI - pull request raised which contains the following enhancements:

  • Define custom attributes from data retrieved from the GeoRSS feed. This allows for extracting additional information from the available data and for example use it for filtering, sorting or publishing to the event bus.
  • Define custom filters on attribute values. This allows for more fine granular filtering of retrieved events.
  • Sort events in info dialogue view by attribute value (ascending/descending).
  • Publish events retrieved from GeoRSS feed onto Home Assistant event bus.
2 Likes
27

Great!

I am still testing your new component and stuck with regexp for custom_filters to be able to sort events with activation date after certain date e.g. 2017-01-01:

  - platform: geo_rss_events_new
    name: European Emergency
    url: http://emergency.copernicus.eu/mapping/activations-rapid/feed
    radius: 250
    unit_of_measurement: 'Incidents'
    custom_attributes:
      - name: date
        source: description
        regexp: '.Activation\ .*\d{4}\-\d{2}\-\d{2}\s+\d{2}:\d{2}:\d{2}'
    sort_by: 'date'
    sort_reverse: true
    custom_filters:
      - attribute: date
        regexp: '?????'
28

The custom attribute regexp definition must contain a named group called custom_attribute. The date filter is simple if you are after a specific date. The regexp does not support dynamic values like a template.
Please see below for a working example:

  - platform: geo_rss_events_new
    name: European Emergency
    url: http://emergency.copernicus.eu/mapping/activations-rapid/feed
    radius: 250
    unit_of_measurement: 'Incidents'
    custom_attributes:
      - name: date
        source: description
        regexp: '.Activation\ .*(?P<custom_attribute>\d{4}\-\d{2}\-\d{2})\s+\d{2}:\d{2}:\d{2}'
    sort_by: 'date'
    sort_reverse: true
    custom_filters:
      - attribute: date
        regexp: '2017-.*'
1 Like
29

Thank you very much for help and clarification.
It would be good to have this explanation in the doc for the sensor when it will be updated since it is not so obvious for lamers like me.

30

Is there any idea why I get this error:

2017-12-12 20:47:57 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up platform geo_rss_events_new
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity_component.py", line 171, in _async_setup_platform
    SLOW_SETUP_MAX_WAIT, loop=self.hass.loop)
  File "/usr/lib/python3.5/asyncio/tasks.py", line 400, in wait_for
    return fut.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result
    raise self._exception
  File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/homeassistant/.homeassistant/custom_components/sensor/geo_rss_events_new.py", line 120, in setup_platform
    data.update()
  File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/util/__init__.py", line 306, in wrapper
    result = method(*args, **kwargs)
  File "/home/homeassistant/.homeassistant/custom_components/sensor/geo_rss_events_new.py", line 290, in update
    events = self.filter_entries(feed_data)
  File "/home/homeassistant/.homeassistant/custom_components/sensor/geo_rss_events_new.py", line 309, in filter_entries
    event = self.create_event(entry, distance)
  File "/home/homeassistant/.homeassistant/custom_components/sensor/geo_rss_events_new.py", line 350, in create_event
    CONF_CUSTOM_FILTERS_ATTRIBUTE]])
  File "/srv/homeassistant/lib/python3.5/re.py", line 163, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or bytes-like object
31

Yes, you are absolutely right. I should have posted a link to the corresponding documentation change.

The error happens in the code that tries to apply the custom filter.
In the quoted configuration snippet in your previous post, an asterisk (*) seems to have gone missing. The regexp should be '2017-.*'.
Also, I can see that you are not using the latest version of the change. The latest version which is part of the PR contains a few minor changes and bugfixes.
Looking at the error message you sent there, I should probably add a few more test cases and some error handling code to better cater for errors in the regular expression.

Just a quick heads-up: The code in the PR is otherwise fully functional, but I may need to do some more refactorings to fit the new functionality into the HA framework. Stay tuned…

32

Thanks, update to the latest version from the PR and adding of missing closing parenthesis in the end of regexp solved my issue.

1 Like