How many Zigbee Router devices have you added to ZHA and how close are they to the Zigbee Coordinator (Zigbee radio adapter built-into Yellow) and how close are the Zigbee Router devices to each other?
I bet the main root cause to much of your problems is that you have not added enough Zigbee Router devices to your ZHA Zigbee network.
Highly recommend that you beging reading this and start by adding a few “known good” Zigbee Router devices (also refered to as Zigbee repeaters or Zigbee range extenders) as well as doing your mest to make sure that you move the Zigbee Coordinator (in your case the Yellow) away from any sources of EMF/EMI/RMI interference:
For starters it is extremely important to understrand that Zigbee is not only very sensitive to EMF/EMI/RMI interference but due to poor radio propagation and low-power radios these devices use they totally depend on mesh networking technology (Zigbee network mesh topology that is automativcally built when adding mains-powered Zigbee products that act as Zigbee Router devices) in order for all messages send to reach its intended target without them having to be re-sent over and over, meaning you always need many Zigbee Routers as for robustness Zigbee devices need to have multiple paths to reach the Zigbee Coordinator.
This is why you should never begin by just adding battery-powered devices to a new Zigbee network.
Firmware upgrade is probably a also good idea and when doing that do change away from RCP Multi-PAN (multiprotocol) so move to Zigbee NCP only firmware if you have not already, see → About firmware options – Home Assistant Yellow (same applies if to use SkyConnect Home Assistant SkyConnect)
If all those actions do not help then the suggested next step would be to change Zigbee channel in ZHA.
Also read these which further explain why you should have loads of Zigbee Router devices and more: