Alert2 - a new alerting component

Rudd-O · November 30, 2024, 10:44pm

So cool! Thanks!

tman98 · December 1, 2024, 3:04pm

That’s fantastic, thanks so much. While you are in there, a small but very powerful addition would also be to be able to add an optional tag to an alert and for the UI card optionally specify a set of tags for that card to display and if none are specified show all (similar to the Scheduler Card / Component which has an implementation of this for both including/excluding tags in the card: GitHub - nielsfaber/scheduler-card: HA Lovelace card for control of scheduler entities). This allows you to have different cards on dashboards with different alerts shown which is very useful as the system grows (and would be big for me).

Very happy to test the changes and many thanks for your hard work!

redstone99 · December 2, 2024, 12:08am

I just released Alert2 UI v1.5.2. It should fix the bug with the slider and the 1min refresh and adds a unittest. After upgrading, you can click on the “Alerts” heading at the top of the Lovelace card to show the loaded version and verify it’s v1.5.2.

@tman98, can you see if this release fixes the issue you see with an alert firing and not showing up promptly in the UI? They should appear within a second or two. I didn’t find a smoking gun that would explain the behavior you’re seeing, though I fixed a few issues that could plausibly be related. If you see the issue again, let me know the details and what you see in the console. There’s an assumption I make in the code, that HA groups together contemporaneous entity updates when sending it to the U. I could dig into verifying that assumption if you run into the issue again. It could explain things.

This UI release also reorders the alerts in the Lovelace card. The new ordering lists unack’d alerts first, then the ack’d alerts. And within each section, it’s ordered most recent first. @tman98 or others, let me know if you have thoughts. I can add bright colors highlighting active alerts, but I wanted to put this release out there without that so we can see if the new ordering alone is enough.

Feedback welcome!
-Josh

woodersayer · December 2, 2024, 7:15am

Adding more optional arguments would be exactly it! I think something along the lines of a data object that is then passed into the notifier would be perfect since that’s how most additional fields are handled in the notifiers I’ve seen. Appreciate the response!

redstone99 · December 2, 2024, 12:19pm

K, will look into adding optional args to report(), maybe I can bundle that with the pattern change I’m working on.

@tman98, I forgot to respond that adding ability to restrict UI card to certain entities, via tags or otherwise, seems like a good idea. It may be a while before I have time to look into it. To make sure I’m understanding, is this so, e.g., if you have multiple HA instances feeding via HA remote into one global instance, that in the global instance you can have multiple alert2 UI cards, either one per HA instance or grouped together?

-J

tman98 · December 2, 2024, 3:06pm

@redstone99 Great - already downloaded and testing.

Initially, one thing I saw is if I snooze or ack an alert it doesn’t drop down into the Acked, snoozed or disabled section (if its off). I’d expect that if it’s on it would stay in the top section but if it’s off it would drop down. If I reload it is in the right section.

I think the easiest visible change would actually be just to change the icon and icon color (and can be consistent with other HA look/feel). Let me come back with a few thoughts on that though to be more coherent on it.

One of the use for tags would be what you are saying - I could group alerts from different local instances within the master instance. But we have another use case with my home install, we want to manage a todo list for the family and alert on some critical items if not done. But I also am using alert2 for my system monitoring. I would like those separate on my dashboards, I keep system monitoring on its own page for me and for the family I have a dashboard set up they can see. I’d like only alerts for the family (like the garbage wasn’t taken out) on that dashboard and my system dashboard to have system alerts. Tags would allow me to do just this.

redstone99 · December 2, 2024, 4:19pm

@tman98: The ack issue you just saw and the fired alerts not showing up in the UI issue happens only to alerts forwarded via the remote_homeassistant component, is that right? If so, I think I figured out what’s going on.

Alert2 UI relies on observing sensor.alert2_change_count increment as a signal to update the UI promptly. If it doesn’t see that sensor increment, it won’t update the UI (until the 60s timer elapses). So any alert2 entity forwarded via remote_homeassistant won’t cause the sensor to increment, causing the various UI issues you’re seeing.

sensor.alert2_change_count is a performance optimization. The way Lovelace works is that there is a variable, hass that contains the state of every entity. Each Lovelace card gets notified each time something in hass changes, without knowing what specifically changed. So a naive implementation of Alert2 UI would be, on every change to hass, to scan every entity to see if any alert2 entities changed. I was trying to avoid all that computation by letting Alert2 UI just look for changes to that change_count entity.

It sounds like I need to abandon the change_count optimization. I can either do the brute force approach detailed above, or, I could keep a set of known alert2 entities, so at least my brute force scan is limited to those entities. I’m not super excited about the n^2 complexity feel of it, though.

EDIT: Actually, perhaps a better solution is to add an optional config parameter, ui_watch_entities. If you’re using remote_homeassistant, you would set that parameter to the list of entity names corresponding to the sensor.alert2_change_count entities you forward from your remote HA instances. That config parameter could probably a template if you want.

Thoughts?
Josh

teachingbirds · December 4, 2024, 2:15pm

I am trying to snooze from an automation like this:

action: alert2.notification_control
data:
  enable: "on"
  snooze_until: "{{ (now() + timedelta(minutes=30)).strftime('%Y-%m-%d %H:%M:%S') }}"
target:
  entity_id: alert2.test_test

The template evaluates to for example 2024-12-04 15:34:48 and this seems to align with what the action takes, but I get an error for: “Notification control call has snooze time specified without timezone info: 2024-12-04 15:34:48 for test_test”

tman98 · December 4, 2024, 2:34pm

Hi Josh,

Thanks for the UI updates on grouping, it’s really getting there and I’m definitely finding it easier to know what is actually a problem to worry about (and using ack to move out of my “worry” group once I’m good with the alert)! So far I think it generally makes sense, but I’ll see after more detailed use.

Here’s my suggestion on making the alerts pop more in the UI. I suggest using some of the existing HA color/icons, e.g. from the Problem class to keep consistency with the HA look/feel:

For unacked alerts, I think the filled exclamation point in red (similar to the Problem class ‘on’ color). Perhaps the circle red exclamation like the Problem class:

For acked or snoozed condition alerts that are still active, an outline icon in orange (still thinking which specific icon would be best),. The following are just colors, I think I like orange better as it indicates more of an issue whereas yellow denotes a switch on in HA generally:

For all past alerts (e.g. acked or disabled and not presently ‘on’), I think the checkbox in blue standard HA color.

I think the pallete above would match HA themes well and give indications as to which alerts are in which state.

Regarding your change_count optimization. I’m not sure if I’m following the EDIT but I worry that adds a degree of complexity where right now things generally work without needing special config. I’m not yet deep into lovelace programming so wasn’t aware every card got notified on every change - how do other cards generally handle this I wonder as it seems like it might affect a lot of the more complex cards (e.g. auto-entities)? Perhaps there is a pattern in use by those cards?

EDIT: A simple quick less invasive change may be to do your performance optimized scan more frequently, every 5 or 10 seconds. I don’t mind the UI doesn’t immediately react, but 60 seconds is a bit long and creates uncertainty if things are working

EDIT2: Some revisions and pictures added to the icon/color thoughts

redstone99 · December 4, 2024, 2:49pm

Hi @teachingbirds - the snooze request you make should still work despite the error. I’ll get rid of the error entirely - it’s more of a debug check from when I was developing the UI code.
J

teachingbirds · December 4, 2024, 2:55pm

Ah, thank you! I did see that my alert was actually snoozed but I also tried without the templating so I thought that was why.

redstone99 · December 5, 2024, 2:05pm

I just released UI v1.5.3. @tman98 , can you see if it fixes the UI updating issues you were seeing?

Thanks for pointing me at auto-entities. They do brute force scans of all entities, with some throttling. So I did the same thing and removed the dependency on change_count. So it should now work fine with remote_homeassistant. I throttle the scan to at most once per second.

@woodersayer, I took a look at Battery Notes. Since, conceptually, a low battery is a state that persists over time (until the battery is replaced) rather than a momentary event, I think an Alert2 condition alert is a better match than alert2.report() (which fires an event). Battery Notes creates an entity for each device with a boolean attribute battery_low that you can alert off of.

Once I implement patterns, would that be adequate for your use case? If not, what additional functionality would condition alerts need for you? Here are two possible ways of expressing a pattern alert for low batteries. Each create a series of alert entities, one per device. I’m not sure how to avoid the regex/map complexity :

- domain: battery
  # Explicitly list the battery notes device names
  pattern: [ "device1", "device2", ... ]
  name: "{{ patternItem }}_is_low"
  condition: "{{ state_attr('sensor.'+patternItem+'_battery_plus', 'battery_low') }}"
  # or using a wild-card matching battery-notes entity names
  pattern: "{{ states.sensor|selectattr('entity_id', 'match',
                 'sensor.(.*)_battery_plus')
               |map(attribute='entity_id')|list }}"
  name: "{{ patternItem|
            regex_replace('sensor.(.*)_battery_plus','\\1')  }}_is_low"
  condition: "{{ state_attr(patternItem, 'battery_low') }}"

In the meantime, I’ll look into adding that extra data arg to report(). Can you give me an example of how you’d use it?

-Josh

tman98 · December 9, 2024, 5:54pm

Thanks @redstone99 - I’ve downloaded the new version and have been testing it, will report back after a few days.

I do have a new issue I’m seeing which is a few alerts I snoozed never unsnooze. Note this was on a local instance not across remote home assistant:

And looking at the details for the low temperature alert, the snooze time period has passed, a while ago.

This has persisted across restarts of home assistant, so something on the server side looks like it’s not picking up these old alerts and correcting their state. The only server side log entries related to the low temperature alert are these:

2024-12-09 12:16:27.354 DEBUG (MainThread) [custom_components.alert2.entities] reminder_check for temperature_low_temperature: need_reminder=0 0 0 False

2024-12-09 12:16:27.387 DEBUG (MainThread) [custom_components.alert2.entities] reminder_check for temperature_low_temperature_condition: need_reminder=0 0 0 False

2024-12-09 12:17:19.825 DEBUG (MainThread) [custom_components.alert2.entities] Result cb for name=temperature_low_temperature_condition, self.entity_id=alert2.temperature_low_temperature_condition entity_id=None

EDIT/UPDATE: So far the UI component seems to be updating quickly and on changes on the master remote homeassistant instance, will continue to monitor

redstone99 · December 10, 2024, 12:44am

@tman98 , thanks for the detailed bug info! Made it easy to find the bug. I’m about to release support for patterns, so was going to include the fix for the snooze bug with that.
-Josh

woodersayer · December 10, 2024, 6:54am

@redstone99 woops, late reply. I think what you’ve laid out for matching would work well! The reason I was looking at the eventing approach was to try and make this generic without having to implement a specific alert for each device. What you’ve shared seems similar to using a selector as one would with PromQL and AlertManager which is pretty ideal. I personally think just using regex will be best as there’s ample documentation and functionality is well known. Over all I think the approach you suggested would work well.

In the meantime, I’ll look into adding that extra dataarg to report(). Can you give me an example of how you’d use it?

I feel the most common would be quick and dirty alerting in automations. I could foresee some of this being useful for listening to events but pattern matching would solve that. I think for me this would be a “nice to have” but I’d rank pattern matching and hot reloading the yaml as higher priority for me since most use cases would be captured.

Appreciating the work you’re doing for the community!

redstone99 · December 11, 2024, 3:02pm

I’m pleased to release Alert2 v1.6 with support for generator patterns!

First, @teachingbirds , this release fixes the spurious alert error you saw around snoozing, and @tman98 this release fixes the bug you observed where Alert2 would not notice a snooze interval expiring if the alert was not firing.

I added a section to the docs for generator patterns that has all the details. Here’s an example, alerting on low battery for all battery_plus entities:

alert2:
  alerts:
    - generator_name: low_bat
      generator: "{{ states.sensor
                    |entity_regex('sensor.(.*)_battery_plus')
                    |list }}"
      domain: battery
      name: "{{ genElem }}_is_low"
      condition: "{{ state_attr(genEntityId, 'battery_low') }}"

In the above example, entity_regex() is a filter function available in generator templates that selects entities matching the regex. An alert is created for each match. Any config template will have availabe the variables genEntityId, which evaluates to the entity_id that matched, and genElem, which evaluates to the first group in the regex. For generators, domain and name take teamplates.

So in the above example, a sensor named sensor.dev1_battery_plus, will result in an alert, alert2.battery_dev1_is_low that fires when the battery_low attribute of the sensor turns on. (genElem will be “dev1”)

Lastly, a sensor, sensor.alert2generator.low_bat is created whose state is the number of alerts created by the generator.

There’s a lot more detail in the docs.
Give it a try and let me know what you think. I’m happy to change it around.

-Josh

tman98 · December 11, 2024, 5:49pm

AWESOME! This is killer, so appreciated! I’ll start trying it out over the weekend.

One thing that might be good to show in the docs is also an expand statement, I found it’s really useful to organize the sensors you care about into a group and then expand the group in the alert. For example I found it useful to put all temperature sensors I care about low temperatures for into a single “minimum” sensor group (I don’t have to then name them something specific in their name to match on, and also the minimum gives me a single quick glance at the lowest temperature of the group so can quickly also identify a problem that way too).

Now a few comments on my initial usage:

Entity id list for generator based on examples

Following your example, I tried this generator using expand instead of entity_regex:

alert2:
  alerts:
    - generator_name: low_temperature
      generator: "{{ expand('sensor.low_temperature')|list }}"
      domain: temperature
      name: "{{ genElem }}_is_low"
      condition: "{{ states(genEntityId)|int < 45}}"
      message: "{{ state_attr(genEntityId,'friendly_name') }} has low temperature of {{ states(genEntityId)|int }}"

This does not work, which is because the above is not producing a list of entitiy_ids.

This list of entity_ids does appear to work (caveated by the next comment below):

      generator: "{{ expand('sensor.low_temperature')|map(attribute='entity_id')|list }}"

So I think two comments to your docs:

Clarify that entity_regex is actually outputting a list of entity_ids (and not a list of entity objects)
I think providing an example that does not rely upon entity_regex but requires a map(attribute='entity_id') filter (like my expand example above) would be great.

Log and alert2_error errors

Now using the map filter as above, I see the following snippets in my logs:

2024-12-11 12:31:36.795 DEBUG (MainThread) [custom_components.alert2.entities] alert2generator_low_temperature generator update called with list result=['sensor.cpu_temperature', 'sensor.up_sense_temperature']
2024-12-11 12:31:36.797 INFO (MainThread) [custom_components.alert2.entities] Generator alert2generator_low_temperature created new alert entity alert2.temperature_sensor.up_sense_temperature_is_low
2024-12-11 12:31:36.799 DEBUG (MainThread) [custom_components.alert2.entities] reminder_check for temperature_sensor.up_sense_temperature_is_low: need_reminder=False 0 False False
2024-12-11 12:31:39.029 DEBUG (MainThread) [custom_components.alert2.entities] Result cb for name=temperature_sensor.up_sense_temperature_is_low, self.entity_id=alert2.temperature_sensor_up_sense_temperature_is_low entity_id=None

2024-12-11 12:31:39.029 ERROR (MainThread) [custom_components.alert2.util] Err reported: {'domain': 'alert2', 'name': 'error', 'message': "temperature_sensor.up_sense_temperature_is_low template Template&lt;template=({{ states(genEntityId)|int &lt; 45}}) renders=8&gt;: UndefinedError: 'genEntityId' is undefined"}

2024-12-11 12:31:39.062 DEBUG (MainThread) [custom_components.alert2.util] create_task called for domain alert2, <Task pending name='Task-3207' coro=<AlertBase._notify.<locals>.foo() running at /config/custom_components/alert2/entities.py:821> cb=[set.remove(), create_task_int.<locals>.<lambda>() at /config/custom_components/alert2/util.py:51]>
2024-12-11 12:31:39.062 DEBUG (MainThread) [custom_components.alert2.entities] reminder_check for alert2_error: need_reminder=False 0 0 False
2024-12-11 12:31:39.084 DEBUG (MainThread) [custom_components.alert2.entities]   _updateBuckets: secsSinceLastAdvance=0.024291 secsLeft=179.975709,  [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2024-12-11 12:31:39.084 DEBUG (MainThread) [custom_components.alert2.entities] remainingSecs: acumm=1 ret0 [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2024-12-11 12:31:39.085 DEBUG (MainThread) [custom_components.alert2.entities] can_notify_now alert2_error, snooze_remaining_secs=0 max_limit_remaining_secs=0 normal_remaining_secs=0 remaining_secs=0 freas=good-to-go-should-never-see
2024-12-11 12:31:39.085 DEBUG (MainThread) [custom_components.alert2.entities] in _notify, remaining_secs=0 good-to-go-should-never-see, ms=<custom_components.alert2.entities.MovingSum object at 0xffff734f24b0>
2024-12-11 12:31:39.085 DEBUG (MainThread) [custom_components.alert2.entities]   _updateBuckets: secsSinceLastAdvance=0.024291 secsLeft=179.975709,  [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2024-12-11 12:31:39.085 DEBUG (MainThread) [custom_components.alert2.entities] reportFire: [2, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2024-12-11 12:31:39.085 DEBUG (MainThread) [custom_components.alert2.entities]   _updateBuckets: secsSinceLastAdvance=0.024291 secsLeft=179.975709,  [2, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2024-12-11 12:31:39.085 DEBUG (MainThread) [custom_components.alert2.entities] remainingSecs: acumm=2 ret0 [2, 0, 0, 0, 0, 0, 0, 0, 0, 0]

2024-12-11 12:31:39.086 WARNING (MainThread) [custom_components.alert2.entities] _notify msg=Alert2 alert2_error: temperature_sensor.up_sense_temperature_is_low template Template&lt;template=({{ states(genEntityId)|int &lt; 45}}) renders=8&gt;: UndefinedError: 'genEntityId' is undefined
2024-12-11 12:31:39.086 WARNING (MainThread) [custom_components.alert2.entities] Notifying ['all_devices']: Alert2 alert2_error: temperature_sensor.up_sense_temperature_is_low template Template&lt;template=({{ states(genEntityId)|int &lt; 45}}) renders=8&gt;: UndefinedError: 'genEntityId' is undefined

The generator is correctly creating multiple alerts for the various entity_ids that were expanded from my group (I’m just showing one above), so that’s good.
genEntityId goes not appear to be correctly getting rendered at the alert creation time for condition
#2 causes the alert to fire it appears, and the notify call does not seem to have message rendered, likely due to genEntityId again not being found

redstone99 · December 11, 2024, 7:03pm

Hi @tman98 ,
Thanks for the quick try. The way generator works is a bit complicated and I welcome feedback on making it clearer or more intuitive. A few quick notes:

As far as I can tell, any template in HA when rendered produces a string. Alert2 does a literal_eval on that string and then examines the result.
If the result is a list of strings, then genElem will be each member of the list. genEntityId will be undefined.

So if your generator template produces “[ ‘sensor.dev1’, ‘sensor.dev2’ ]”, then genElem will be each of those entity ids, and genEntityId will be undefined.

That’s why expand()|map(attribute="entity_id") results in genElem being each entry in the list of entity_ids

expand()|list I imagine produces a list of objects, which HA templating renders to a list of weird strings like “<object __repr__>” and so genElem will be those each of those weird strings.
entity_regex is doing some fancy footwork. It’s producing a list of dictionaries. So in the battery_plus example, the entity_regex is producing something like:
```
'[ { "genElem": "dev1", "genEntityId":"sensor.dev1_battery_plus"}, 
   {"genElem": "dev2", ...} ]'
```
And Alert2 knows to interpret a dictionary as a set of variables to define. The purpose of this fanciness is for convenience in the rest of your alert spec, so you don’t need to extract “dev1” from “sensor.dev1_battery_plus”, or alternatively, convert “dev1” into “sensor.dev1_battery_plus”.

Essentially, what’s going on is that filters & templates generally can return a list of only one thing. And as I was writing generator specs for my own alerting, I got tired of the verbosity of writing “|regex_sub”, the string conversion and writing the regex pattern multiple times in each generator spec.

I can absolutely put an expand() example in the docs.

Thoughts? Got an idea of a better way to structure the generator?

EDIT: I realize another example might help. The above battery_plus generator can be written without using entity_regex as follows. Note genElem is the variable that takes on each entity_id (since generator is returning a list of strings, rather than the list of dictionaries that entity_regex returns)

alert2:
  alerts:
    - generator_name: low_bat
      generator: "{{ states.sensor
                   |selectattr('entity_id', 'match',
                             'sensor..*_battery_plus')
                   |map(attribute='entity_id')
                   |list }}"
      domain: battery
      name: "{{ genElem |regex_replace(
             'sensor.(.*)_battery_plus','\\1')  }}_is_low"
      condition: "{{ state_attr(genElem, 'battery_low') }}"

-Josh

tman98 · December 11, 2024, 11:22pm

EDIT: rewrite as I understood more about the implementation.

I think I understand more of what you’re doing. You’re basically letting generator take a list and you don’t really care what the list is, you just allow that list’s values to pass straight down to the generated iterations.

That itself makes sense, until you create the special filter that then has a special variable name called genEntityId. That implies to me as the reader of the documentation (especially as the documentation focuses on the filter and the variables, but without calling out the filter’s special dictionary output) that genEntityId will always have the entity_id no matter what (and not that it might actually be null and genElement has the entity_id if that’s how my list was constructed).

I see value in generator taking a list and then passing the list to the generated iteration. I’d make the variables more straightforward in that case and the documentation clear:

Have genElement return the list item for the current iteration without translation. So if the original list was a list of strings, it is the current string. If it’s a list of dictionaries, it’s the current dictionary as a whole and not just one key/value pair’s value. There’s no hidden/magic translation happening then between the generator line and genElement. I.e. this functions just like a foreach loop and it’s iteration variable.
Clarify in the docs that entity_regex outputs a dictionary specifically. Allow the regex to have multiple ( ) matches and not just 1. Make it’s dictionary elements be:

'[ { "entity_id":"sensor.dev1_battery_plus", "regex_match": ["dev1", "the next ( ) match, etc"] } ] - e.g. regex matches are a list

So you’d get values in the iteration like:

genElement - if a single list of strings was produced by the generator template, would just be a string

genElement['key'] - if a dictionary was produced by the generator template, you select which key you want.

genElement['key'] with predefined keys - if a dictionary was produced by the generator template via entity_regex, you select which key you want via the dictionary created by entity_regex (and not special variables names):

genElement('entity_id') - returns entity_id as the key created by entity_regex.

genElement['regex_match'][]0] - returns the specific regex match position from entity_regex

redstone99 · December 12, 2024, 10:38pm

@tman98, nice job identifying the conceptual problem with my approach. Your proposal is conceptually cleaner. One concern I have is that getting the entity_id when using states|selectattr|map is just genElement, but if using states|entity_regex, it becomes the more cumbersome genElement('entity_id'). It’d be nice if getting the entity_id was the same in both cases.

What about one of the following possibilities:

Alert2 checks the generated list. If it’s entity_ids or the result from entity_regex, then the variable genEntityId is defined. Using entity_regex also defines genGroups. And maybe genRaw is defined as you suggest for genElement as a fallback, in case you’re generating a mixture of entity_ids and other things.
Add an extra boolean config parameter generating_entity_ids, to explicitly tell the system what you’re generating. If it’s true, then genEntityId is defined. Otherwise, genElement is defined.

Btw, my interest in entity_regex is partly conciseness and partly so I can write the entity-matching regex just once in an alert generator spec and so hopefully reduce mistakes.

Thoughts? Also, @woodersayer or others, feel free to chime in if you have an opinion.
Josh