This sounds like RF interference to me.
Maybe the dongle is not on an extension cable and away from other RF sources, or you are on a channel that is busy with WIFI interference, or something like that.
I have 3 zigbee networks, 2 in one house, and I don’t see any of that. The Home Assistant Cookbook - Index