Ssl error, ha cloud

I am trying to use HA cloud to connect to my HA yellow but it is very unreliable.
Sometimes I can get in after 3 tries, or it can take 10 tries, and sometimes it does not work at all. I have tried restarting the app, turning on and off flight mode in between tries but haven found any way making it work consistently.
Setup process was logging in to nabu casa account, turning on remote control and adding the url to my android app. Using a pixel 4a running android 11 with a custom rom, /e/os 1.5

Error message:
failed to perform ssl handshake, please ensure your certificate is valid.

Cloud connection status in HA is connected and remote control is enabled.

log: ha_companion_ssl_error -

Any suggestions in dialing in the issue would be greatly appritiated.