I have the same problem. It’s extremely tedious.
It started with HomePod version 17.4.
I think the problem is that the HA scenes are displayed as switches in Apple Home. And maybe HA is not reporting the current status of the switch correctly.
Or, as always, Apple wants to make integration with everything else worse…
If I had known HA earlier, I wouldn’t have bought any HomePods 