Alert2 - a new alerting component

pav · September 26, 2024, 1:06pm

Hi Josh, you keep surprising me, not only for paying such close attention to my ramblings , but also for proposing well thought-out changes/improvements that surpass my own suggestions.
Just to say that the combination of collapsing the internal entities into a single one, and the addition of the parameter governing its creation would leave me wishless (at least for now, haha).

Btw, talking about the alerts : I must admit that sometimes they leave me bewildered as to their origin and/or meaning. E.g. I had one the other day “Err reported: {‘domain’: ‘alert2’, ‘name’: ‘assert’, ‘message’: ‘Printer_Powered, not notifying but remaining time is 0’}”. Cryptic to me, as I have no clue as to the what or why …

redstone99 · September 27, 2024, 10:39pm

Hi Paul, I just released Alert2 v1.3.5. It consolidates all the internal alert entities to a single one, alert2.error, and provides an option to omit even that entity. The weird alert you saw was an unnecessary notification - I fixed the bug. I also somewhat improved the message for alert2.error to request that people file an issue or notify the Alert2 maintainer. My goal is that any alert should be clear and informative, so let me know if you see any confusing things going forward.
-Josh

redstone99 · October 5, 2024, 7:50pm

And just released v1.3.6. The main change is to add unittests (yay!) and also allow specifying literal entity names in condition and threshold value parameters. So in an alert, instead of writing:

condition: "{{ states('binary_sensor.my_sensor') }}"

you can now alternatively just write:

condition: binary_sensor.my_sensor

and similarly with the value field of a threshold.

EDIT: I updated the latest release to 1.3.7.1 to support the new HA 2024.10.
HA 2024.10 changes how messages to notifiers are interpreted, making it no longer necessary to wrap messages in “{% raw %}”.

-Josh

redstone99 · October 28, 2024, 6:37pm

v1.4 is out with support for template notifiers! You can now dynamically specify who gets notified and how. I updated the docs with details and examples.

Also, I opened up the discussions section of the github repo, which might be a good place for discussing specific feature requests. Or here.

Feedback or suggestions welcome!
-Josh

tman98 · November 7, 2024, 12:56pm

Wow, @redstone99 this is an incredible project. Thank you for your contributions!

Quick question, do I need to restart Home Assistant after any edit to configuration.yaml or can I just reload yaml files?

On a related note I am finding maintaining alerts to be difficult through YAML (especially as a HA restart takes a long time in my setup). It would be amazing if there were a UI for creating alerts as maintaining a lot I’m finding to be a challenge and these are the types of things you add/edit/change a lot. A simple YAML error causes my entire HA setup to fail on startup and twice I’ve had to revert to backups (separate problem I’m trying to chase down where if HA seems to not load components in the right order because of a config error it resets dashboards).

Also testing the alert is something I’m struggling with when using templates as I can’t easily get some triggers/conditions to occur.

It would be incredible to have a UI to manage alerts (and hopefully minimize problems with formatting/errors in alert templates for example) and I think would open HA up as an overall product unbelievably. Something like the scheduler card would be really the best.

I’m not sure how to be able to better test alerts without causing the triggers/conditions they are dependent on to fire, because I want to make sure they are right especially if some of the conditions are a “must alert” situation, I don’t want an untested error to prevent the alert from firing.

redstone99 · November 7, 2024, 7:51pm

Hi @tman98 , thanks for the feedback! Yes, I believe that modifying configuration.yaml requires an HA restart. I’ve tried to make Alert2 so that a bad Alert2 config will never prevent HA from starting and also so that a bad single alert config will not prevent other alerts from loading successfully. I think the original Alert integration requires the whole config to be correct.

The ability to configure alerts through the UI would be nice. I’d help someone else do the development of this. Doing it well is probably a decent spot of work.

How to test alerts is a great question. Today I suppose you copy an alert and replace the sensor reference with true to see what happens. One could imagine an interface that let’s you test “what happens if…” without changing your alerts. It would mock the sensors referenced in an alert with values you specify so you could see what happens.

Two other threads on the alert UI topic below (though neither seems to be active).

tman98 · November 11, 2024, 10:39pm

Hi - I agree on the UI being some work. I have a few ideas I’ve been tossing around.

In the meantime, I’ve been getting a few errors reported to the alert2_error alert. I routninely get a number of these errors on a HA restart:

exception {'message': 'Task exception was never retrieved', 'exception': ServiceNotFound('service_not_found'), 'future': <Task finished name='Task-527' coro=<ServiceRegistry.async_call() done, defined at /usr/src/homeassistant/homeassistant/core.py:2697> exception=ServiceNotFound('service_not_found')>}

The second is this error, which I periodically get when a threshold alert starts to exceed its bound:

Internal error. Please report to Alert2 maintainers (github.com/redstone99/hass-alert2). Details: system_memory_high turning on but already have delayed wait set

The relevant alert is looking at the system monitor’s memory usage:

    - domain: system
      name: memory_high
      friendly_name: Memory high
      threshold:
        value: sensor.system_monitor_memory_usage
        maximum: 90
        hysteresis: 5
      delay_on_secs: 120
      message: "{{ states('sensor.system_monitor_memory_usage') }}"

redstone99 · November 12, 2024, 12:59am

Hi, thanks for reporting the issues! I confirmed the bug with thresholds and delay_on. Fix coming soon. The “Task exception”/“service not found” one is a bit trickier though I have an idea.

Can you take a look at the logs and see if there is a message from Alert2 right before the exception happens? There should be a log line right before reporting which notifier service Alert2 was trying to async_call on. Might be helpful if you could send some log lines before that as well, just in case.
EDIT: I think I know the issue. My guess is in the logs you’ll see a message about persistent_notification.

Regarding UI alert2 config support, I’d love to hear your thoughts. I’ve started laying some groundwork on the backend as I mull over how it might work and look at how the Scheduler card works. Though let’s first get these bugs sorted out.

-J

tman98 · November 12, 2024, 4:24pm

Thanks much! On the threshold error it appears I received that error today and never got the actual alert so it looks like it’s preventing the alert itself from correctly firing, looking forward to that fix!

For the task exception was never retrieved, I found a few logs. Unfortuantely I cannot for the life of me figure out how to copy out of the terminal (there’s a github issue tracking this), so here is a screenshot:

There are then a bunch of logs of this same info being repeated over and over
Note the actions “notify.lovelace_popup” (a custom one of my own in pyscript) or “notify.mobile_app_6453” (a standard mobile device with the companion app) are being reported as not found. Both exist and work later on when alerts are actually fired.

However when I look later in my log I see the pyscript that creates lovelace_notify being run later, and then I see “Setting up mobile_app” occuring in the log later as well.

Perhaps alert2 is trying to configure/initialize something with notifiers that have not yet been set up in the sytem yet?

redstone99 · November 12, 2024, 5:56pm

Hi, I just released v1.4.1 that hopefully will fix the issues you encountered. Can you give it a shot?

It adds a dependency on notify during startup which should ensure persistent_notification is available. The notify component changed somewhat recently to defer full initialization of all notifiers, which means it can take some time after startup for all notifiers to be available. So Alert2 has a notion of “deferred” notifications during startup. I also now catch the ServiceNotFound exception, which I don’t think should ever occur anymore, but if it does, it will log an error in the logs.

Thanks,
Josh

tman98 · November 12, 2024, 7:30pm

Wow amazing that was incredible turnaround and with unit tests! So glad you have a solid test base, makes me more confident in the codebase as well. I’m installing and will report after a day or two!!

tman98 · November 12, 2024, 8:37pm

Hi @redstone99, unfortunately the startup issue isn’t yet resolved, here’s the log from startup of alert2 (below). Subsequently in the log, my two notifiers notify.lovelace_notify and notify.mobile_app_6453 are set up. So it does not appear that we’re delaying until after all notifiers are established.

For the mobile one, that is a standard mobile app notificiation so seems like we should definitely be delayed until after that, so it seems like alert2 is starting before registered notifiers (but perhaps after the notifier system and persistent_notification themselves are set up).

For my custom one written in pyscript (lovelace_popup), it occurs to me alert2 wouldn’t actually know to wait as the system wouldn’t know a pyscript has a notifier. I’m not sure how I can force myself to be in the startup chain ahead of alert2 without modifying the manifest (which would be overwritten on an upgrade).

Given that it seems there can be notifiers being set up during HA startup that alert2 may not be aware are being set up until an alert fails during startup, I’m not sure the utility of trying to send alerts to notifiers that are not yet set up during startup. Even with error checking they will still get lost and never reported to yet-to-be-setup notifiers, so perhaps that phase needs a little retooling, probably some logging of the alert and either queuing up the alerts to send only once HA starts or the notifiers are available or something else.

I’m also not entirely sure why a throttle interval alert is the very first alert that appears to be being thrown in the logs before any other alerts.

2024-11-12 15:16:10.883 INFO (MainThread) [homeassistant.setup] Setting up alert2

2024-11-12 15:16:10.886 INFO (MainThread) [custom_components.alert2] Setting up Alert2

2024-11-12 15:16:10.894 WARNING (MainThread) [custom_components.alert2] _notify msg=[Throttling ending] Alert2 alert2_error: Did not fire during throttled interval

2024-11-12 15:16:10.896 WARNING (MainThread) [custom_components.alert2] Notifying ['all_devices']: [Throttling ending] Alert2 alert2_error: Did not fire during throttled interval

2024-11-12 15:16:11.192 INFO (MainThread) [homeassistant.setup] Setup of domain alert2 took 0.31 seconds

2024-11-12 15:16:11.258 INFO (MainThread) [homeassistant.setup] Setting up shopping_list

2024-11-12 15:16:11.258 INFO (MainThread) [homeassistant.setup] Setup of domain shopping_list took 0.00 seconds

2024-11-12 15:16:11.289 ERROR (MainThread) [custom_components.alert2] Exception {'message': 'Task exception was never retrieved', 'exception': ServiceNotFound('service_not_found'), 'future': <Task finished name='Task-1996' coro=<ServiceRegistry.async_call() done, defined at /usr/src/homeassistant/homeassistant/core.py:2697> exception=ServiceNotFound('service_not_found')>}

2024-11-12 15:16:11.290 ERROR (MainThread) [custom_components.alert2] Err reported: {'domain': 'alert2', 'name': 'error', 'message': "exception {'message': 'Task exception was never retrieved', 'exception': ServiceNotFound('service_not_found'), 'future': &lt;Task finished name='Task-1996' coro=&lt;ServiceRegistry.async_call() done, defined at /usr/src/homeassistant/homeassistant/core.py:2697&gt; exception=ServiceNotFound('service_not_found')&gt;}"}

2024-11-12 15:16:11.291 WARNING (MainThread) [custom_components.alert2] _notify msg=Alert2 alert2_error: exception {'message': 'Task exception was never retrieved', 'exception': ServiceNotFound('service_not_found'), 'future': &lt;Task finished name='Task-1996' coro=&lt;ServiceRegistry.async_call() done, defined at /usr/src/homeassistant/homeassistant/core.py:2697&gt; exception=ServiceNotFound('service_not_found')&gt;}

2024-11-12 15:16:11.292 WARNING (MainThread) [custom_components.alert2] Notifying ['all_devices']: Alert2 alert2_error: exception {'message': 'Task exception was never retrieved', 'exception': ServiceNotFound('service_not_found'), 'future': &lt;Task finished name='Task-1996' coro=&lt;ServiceRegistry.async_call() done, defined at /usr/src/homeassistant/homeassistant/core.py:2697&gt; exception=ServiceNotFound('service_not_found')&gt;}

2024-11-12 15:16:11.295 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved (None)

Traceback (most recent call last):

File "/usr/src/homeassistant/homeassistant/core.py", line 2735, in async_call

raise ServiceNotFound(domain, service) from None

homeassistant.exceptions.ServiceNotFound: Action notify.lovelace_popup not found

2024-11-12 15:16:11.309 ERROR (MainThread) [custom_components.alert2] Exception {'message': 'Task exception was never retrieved', 'exception': ServiceNotFound('service_not_found'), 'future': <Task finished name='Task-1995' coro=<ServiceRegistry.async_call() done, defined at /usr/src/homeassistant/homeassistant/core.py:2697> exception=ServiceNotFound('service_not_found')>}

2024-11-12 15:16:11.309 ERROR (MainThread) [custom_components.alert2] Err reported: {'domain': 'alert2', 'name': 'error', 'message': "exception {'message': 'Task exception was never retrieved', 'exception': ServiceNotFound('service_not_found'), 'future': &lt;Task finished name='Task-1995' coro=&lt;ServiceRegistry.async_call() done, defined at /usr/src/homeassistant/homeassistant/core.py:2697&gt; exception=ServiceNotFound('service_not_found')&gt;}"}

2024-11-12 15:16:11.310 WARNING (MainThread) [custom_components.alert2] _notify msg=Alert2 alert2_error: exception {'message': 'Task exception was never retrieved', 'exception': ServiceNotFound('service_not_found'), 'future': &lt;Task finished name='Task-1995' coro=&lt;ServiceRegistry.async_call() done, defined at /usr/src/homeassistant/homeassistant/core.py:2697&gt; exception=ServiceNotFound('service_not_found')&gt;}

2024-11-12 15:16:11.311 WARNING (MainThread) [custom_components.alert2] Notifying ['all_devices']: Alert2 alert2_error: exception {'message': 'Task exception was never retrieved', 'exception': ServiceNotFound('service_not_found'), 'future': &lt;Task finished name='Task-1995' coro=&lt;ServiceRegistry.async_call() done, defined at /usr/src/homeassistant/homeassistant/core.py:2697&gt; exception=ServiceNotFound('service_not_found')&gt;}

2024-11-12 15:16:11.312 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved (None)

Traceback (most recent call last):

File "/usr/src/homeassistant/homeassistant/core.py", line 2735, in async_call

raise ServiceNotFound(domain, service) from None

homeassistant.exceptions.ServiceNotFound: Action notify.mobile_app_6453 not found

redstone99 · November 12, 2024, 9:52pm

Hi @tman98, sorry the fix didn’t work and thanks for sending logs. I created an issue to track this: Notifying "notify groups" during startup cause ServiceNotFound exceptions · Issue #3 · redstone99/hass-alert2 · GitHub.

HA initializes notifiers asynchronously. So I believe that HA can say it has fully started while there are still notifiers that have not finished initializing. For this reason, Alert2 has a mechanism to queue up and defer notifications during startup if the notifiers don’t yet exist.

However, Alert2 isn’t (yet) smart enough to check inside “notify groups”. So I think the following could be happening:

Due to the earlier threshold bug, Alert2 tried to send you lots of notifications and throttling started. That throttling state persists across restarts.
HA restarts. The notify group all_devices initializes. However the member notifiers like lovelace_notify have not yet initialized
Alert2 wants to notify you that throttling of alert2.error has ended.
Alert2 goes to notify all_devices, sees that the notifier exists and invokes the services.async_call to notify.
The notify group all_devices then invokes services.async_call to notify each member.
The members don’t exist, so ServiceNotFound is thrown, which is attributed to alert2 since it originally initiated the call.

If I’m right about this issue with “notify groups”, one answer is to make Alert2 detect “notify groups” and expand them itself, so it can check and defer any not-yet-existing notifiers.

As a work-around, Alert2 already has the ability to notify multiple notifiers itself. One could create an entity called sensor.all_devices whose state would be the list of notifiers. Then you could tell Alert2 to notify that entity name.

EDIT: Actually, I realize that unless early_start is specified, generally alerts wait till HA has fully started before triggering. The alert2.error is an exception to this rule. Making it conform would also avoid the immediate issue.

I’ll look into notify groups and see what’s involved in expanding them.
I’ll copy this message into the Github issue. You can reply there or here.
I appreciate you help making Alert2 better! Nothing beats real world mileage.
Cheers,
Josh

tman98 · November 12, 2024, 11:11pm

Great! And more than happy to be a real world production user!

tman98 · November 14, 2024, 6:33pm

Hi Josh, I may have hit another bug. I had an alert go off yesterday that I snoozed for a few hours. It is still indicating snoozed today after the time has passed:

And in the details of the alert:

Notifications

Status: snoozed until 2024-11-14T01:39:20.713000+00:00

Which (as long as Ive gotten my UTC conversions right ) is a time yesterday so it looks like the snooze time passed without the snooze ending. Happy to provide any logs you’d like.

Thanks!

tman98 · November 14, 2024, 10:05pm

I feel like I’m throwing a lot at you :). I had an alert go off earlier today that I snoozed. This one is if there is motion on my indoor motion sensors. It suddenly went off now so that was surprising. However when reading it it’s something delayed from a few hours ago, possibly after the snooze period ended, but no alert should have fired now as the occupancy ended over 3 hours ago.

2024-11-14 17:01:03.896 WARNING (MainThread) [custom_components.alert2] _notify msg=House occupancy away fired 4x (most recently 3h ago): turned off 3h ago after being on for 54s

2024-11-14 17:01:03.897 WARNING (MainThread) [custom_components.alert2] Notifying ['all_devices_critical']: House occupancy away fired 4x (most recently 3h ago): turned off 3h ago after being on for 54s

redstone99 · November 14, 2024, 10:33pm

Hi - I’m glad for the feedback, keep it coming! I will be less online till this Sunday, so sorry in advance for delays responding.

Re the notification exceptions: I figured out what’s going on. It’s partly due to limitation/issue with the group integration. I’m testing a workaround that should avoid the problem. It’s probably ready to release now. Will release and explain on Sunday or sooner.

Re the snooze not turning off when it should. I’ll look into it:

Did HA happen to restart within 15 minutes of when you last snoozed the alert?
If it happens again, can you go to the bottom of the “more info” pop-up for the alert, click on “Attributes” and see if the value for “Notification control” matches the time you expect the snooze to end?
If you set an alert to snooze for a minute, does it unsnooze after a minute for you?

EDIT: I noticed that Alert2UI does not update the more-info notification-control “Status”. So if you updated the snooze time, the “status” field would still show the old time until you closed the popup. Is it possible that’s what you observed? In the upcoming release, I changed it so the “status” field auto-updates.

Re the unexpected notification after snooze ends. The intention was that, if any alerts fired during the snooze period, to send a notification summarizing what happened when the snooze period ends. I can see that being unexpected and maybe undesired, especially at the “critical” notification level. I see two options to improve this:

Add a config flag to suppress any such summary notifications
Add a config flag to specify the notifier to use for such summaries.

Any preference or thoughts?

Cheers,
Josh

tman98 · November 16, 2024, 4:27pm

Thanks - it is possible that HA did restart within 15 minutes of when I snoozed the alert as I was doing some work with HA. I’m not certain but it’s possible. I’ll keep my eye on it to see if it occurs again.

Regarding the notification after the snooze ends, that’s an interesting feature you have thought of (the summary). It was counter to what i expected as behavior in this case beacuse this was a critical alert notifier for occupancy sensors I had snoozed when someone was at the house. Getting an alert at the end of the snooze period that occupancy had gone off was surprising and seemed like an intruder was in the house.

I actually suggest both of your options. My feeling is the summary is more of an “opt-in” type feature so perhaps only have it sent if you have selected to have it sent (and an optional separate notifier is great - for example a less critical one would be useful). I think the messaging might want to include something about summary in the text because even when reading the message I thought the alert had just fired.

Perhaps something like: "House occupancy away summary at end of snooze period: alert fired 4x during snooze (most recently 3h ago): turned off 3h ago after being on for 54s

tman98 · November 18, 2024, 10:34pm

Hi Josh - I think there is something going on with snooze. I had two alerts go off today, and I snoozed both. One ended at the end of the snooze time no problem, the other appears to have passed the snooze time. It’s now 5:32 pm and the snooze was supposed to end at 5:03 pm, which the notification control shows as seen below. I’ve refreshed the browser so I think this is a server side error and not a UI caching problem. I also never got the “unexpected” snooze ending notification we were discussing for this alert.

I’ve tried enabling custom_components.alert2 debug logging, not sure if that will turn anything up?

redstone99 · November 18, 2024, 11:29pm

Hi @tman98, I just released Alert2 v1.5 along with a corresponding update to Alert2 UI. This release should help with some of the issues you encountered.

Alert2 biggest changes:

Add config options notifier_startup_grace_secs and defer_startup_notifications to better support YAML notify groups.
Add summary_notifier config option so you can use separate notifiers for summary messages (eg snooze ending)
Add unack service call

Alert2 UI biggest changes:

Make more-info popup dynamically update “Status” field and “attributes”.
Clicking on “Alerts” header of Lovelace card shows Alert2 & Alert2-UI version.

And I updated all docs.
Regarding recent issues you’ve faced:

Notifier exceptions

I believe those notification exceptions you saw were not coming from Alert2 per-se, but from the “notify group” code. YAML notify groups have an issue where they throw an exception if a member notifier does not exist. And Alert2 has no good way to tell during HA startup if a notifier is a notify group with a missing member. So…

I created a config option defer_startup_notifications. You can list notifiers here, eg notify groups, and Alert2 will defer trying to notify them for notifier_startup_grace_secs during HA startup. You can also just set defer_startup_notifications to true and Alert2 will defer all notifications during startup. For non-group notifiers, Alert2 automatically defers notifying during startup, so this is probably only makes sense for YAML notify groups.

EDIT: I forgot to mention, you could also just use Alert2 native “groups” aka creating an entity with a list of notifiers and using that. The docs have an example. It’d avoid the issues I mention above.

Surprising summary notifications

Sorry for the alarming and confusing summary notification on your high-priority alert. I created summary_notifier to control where those go. You can specify a notifier here, set to True to use the regular notifier, or set to False to not send summaries. So to be clear, normal notifiers are used for an alert firing, a condition alert stopping firing and reminders that a condition alert is still on. summary_notifier is used for other notifications, which at the moment are to detail alert action that happened while an alert was snoozedor throttled.

Snooze not appearing to unsnooze

The v1.5 release makes the Status and attributes sections of the more-info dialog dynamically update. That should help the more-info popup show current info. I’m not sure why the alert2 lovelace card you screenshotted is showing stale info. Are there any errors in the javascript console log (shift+ctrl+ I ) ?

When the snooze period for an alert ends, there is a log (INFO) line written containing the alert name and “snooze has expired. Reenabling notifications”. Can you look in your logs and see if you see those at the times you expect? If you haven’t already, you may need to set the default log level to “info” in your configuration.yaml, as below.

logger:
  default: info

And if you want gobs of debug info, you can set debugging output for the alert2 like this in your configuration.yaml:

logger:
  default: info
  logs:
    custom_components.alert2: debug

Oh and lastly, when you update, you can click on the “Alerts” header of the Lovelace card to verify you’re running the current versions.

Thanks for looking into the snooze issue and trying things out!
-Josh