Unable to connect to Home Assistant over SSL 50% of time

I have a hass.io installation that I have setup to use a registered domain name and I have setup Let’s Encrypt for it. My problem is that it works about 50% of the time and the other 50% I get “Unable to connect to Home Assistant” errors. Everything points to the domain name, SSL, DNS working all the time. If I ping the domain name it always returns the correct IP. It is a DDNS registered name, but the IP has not changed in months. I have another non SSL website that is port forwarded in the same manner on the same router and it always works. Even when the SSL domain name access to the home assistant doesn’t work I can still connect to it via the Local IP or the external IP, of course I get a certificate error either way, as expected.

At this point I am leaning toward a problem with Let’s Encrypt, but I’m just guessing. This is my first experience with Let’s Encrypt.

Any ideas about why I can’t connect 50% of the time?

Timeout problem?
What’ s about the cpu-load?