It could just be a coincidence, @bkbartk. Can you give me step by step instructions on how to repro this for you? I’d like to see if I can repro it and maybe then it’s more than just a You Thing. I may be doing the same steps as you, but I am using Vivaldi and also I could be doing things not the same. 
I made a small video (in vivaldi because it was easier to hide the address bar)
on beta4, not beta5
Here I notice a small delay between save and the warning,
I do not notice that on other browsers.
I’m seeing the same thing, however the 2nd window disappears if I don’t touch anything. Seems like it’s just a normal bug. I’d wager it existed before beta as well. I.e. try what you did again, but before clicking save on the second window, wait a second.
it indeed disappears,
and there are plenty of workarounds for this,
but accidentally creating an automation twice could be really ugly.
But I think I should report this as a normal bug indeed.
Yep, include the video, it’s easily reproducible with that.
AND WE’VE REACHED THE END!
2025.5 is now released. Please head to the release blog thread to continue feedback regarding this release.
Thanks, y’all! 
Is this the solution though?
It is because I say it is.