Last time I had this issue was when encryption was released and passwords were depreciated. I had to delete all 42 of my ESP devices from home assistant. Restart Home Assistant (important!) and then add them all back again. This resolved all connection issues.
If you do this you will loose any customisations to devices you have made using the Device GUI. You will not lose any history as the recreated entity_id’s will be the same. You may have issues if you use device triggers, conditions or actions (don’t use them).
So I resolved this. The issue was that my Home Assistant VM had the wrong subnet mask configured /24 instead of /22. Not quite sure why this was causing it to intermittently work but at least it appears to be solved now.