Now that you figured out what caused it, it would make sense to switch everything over to Z2M, no?
There’s no need to keep running ZHA just for these switches if it gives the same behaviour. The blueprint @allard77 linked to has a link to the Z2M version for these switches Z2M - TS004F Tuya 4-Gang Switch blueprint