Came here to make this same request. I have the Alexa service connected, but I expose very little to that service. Every time I add a new entity, I receive an Alexa notification that it’s now connected. I have to go disconnect that entity and then remove the unavailable entity in the Alexa app. This is very annoying behavior that I would love an option to change. Hope this topic gets noticed.
edit: after some more searching I found this which worked for me.