I have now added my FoH switches through zigbee2mqtt through a SkyConnect, which has significantly improved the delay and made me able to make dimming a light more smooth as it sends a “release” action.
I’ve set it up using this blueprint: Zigbee2MQTT enOcean PTM 215Z (Friends of Hue) switch