ZHA device binding works only one way

Hello,

I have two Philips Hue Bulbs, and one in-wall switch Lonsonho QS-Zigbee-S05-LN. I connected bulbs in a way, that they have permanent power and the switch does not control output, but only works as a ZigBee switch. I would like to set up binding so these devices are connected directly because I have a feeling that raspberry pi with running HA is the weakest part of the ZigBee network and I would like to have working lights even if raspberry is stuck.
I use ZHA to connect them to HA.
I created a new ZigBee group in ZHA configuration/groups where I added both bulbs. Then I went to switch settings/clusters and select bind device where I selected this group. I also tried push bind groups but there are only Ota and Time Clusters in the bind group select box.
In the end, it works, but only halfway. When I “turn on” the in-wall switch, it only turns this switch on.
But when I turn on or off that bulbs group, it toggles also in-wall switch, which is correct I think.

Does anyone know, what could be wrong?
Could it be something with device signature and in_clusters and out_clusters? In in-clusters, I can see that there is 0x0006 which should be the OnOff cluster if I get it correctly, but in out-clusters, there is 0x000a and 0x0019 which is Time and Ota (which I can see in that bind group select box). Should 0x0006 be also there in out-clusters? Is there any way how can I modify this? Or does that mean that the in-wall switch does not support this?

Thank you