I ran HA on an rpi3 for a while. The limiting factor is not the CPU (ok, stuff takes a while) but actually the RAM.
1GB is filled up extremely fast when running AddOns.
The default config for HA is to only have 200MB of swap size. That also runs out quick and when that happens you go into weird unresponsive states which sometimes catch themselves, sometimes they don’t.
The solution is to increase swap size. Sadly there is no easy config for this, but there is a hack.
Try this, it worked wonders for me!