I even tried to remove both the button and switch from HA but the button still managed to turn on the switch. I guess there is some kind of signal interference going on.
I have a new finding.
The Hue button will turn on the Aqara switch whenever it is connected to the switch in the Zigbee mesh network.
If I force the Hue button to connect to another node, e.g. the coordinator, the problem goes away. However, after a while, the Hue button will automatically find a new (optimized) route and connect to the Aqara switch again. Then the issue comes back. Not sure whether this is because of Aqara’s non-standard implementation of their Zigbee devices.