For those who came for this solution, I disabled both Wi-Fi and Bluetooth as instructed on my Yellow. But then, after a reboot, my ZHA failed due to missing /dev/tty/AMA1. Maybe disabling BT on CM4 somehow also disables the Gecko Module on the Yellow @xpal. So maybe the best approach for HA Yellow is only disabling the Wifi, which I found decreased the interference to ZHA quite a lot.