Hi @tman98 , thanks for the feedback! Yes, I believe that modifying configuration.yaml requires an HA restart. I’ve tried to make Alert2 so that a bad Alert2 config will never prevent HA from starting and also so that a bad single alert config will not prevent other alerts from loading successfully. I think the original Alert integration requires the whole config to be correct.
The ability to configure alerts through the UI would be nice. I’d help someone else do the development of this. Doing it well is probably a decent spot of work.
How to test alerts is a great question. Today I suppose you copy an alert and replace the sensor reference with true to see what happens. One could imagine an interface that let’s you test “what happens if…” without changing your alerts. It would mock the sensors referenced in an alert with values you specify so you could see what happens.
Two other threads on the alert UI topic below (though neither seems to be active).