Nabu casa remote UI connection not always available

I am on 0.103.5 and I have the same issue:

2020-01-08 12:29:01 ERROR (MainThread) [hass_nabucasa.remote] Can't handle request-connection without backend
2020-01-08 12:29:01 ERROR (MainThread) [hass_nabucasa.iot] Error handling message
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/hass_nabucasa/iot.py", line 90, in _async_handle_handler_message
    result = await handler(self.cloud, message["payload"])
  File "/usr/local/lib/python3.7/site-packages/hass_nabucasa/iot.py", line 143, in async_handle_remote_sni
    await cloud.remote.handle_connection_requests(caller_ip)
  File "/usr/local/lib/python3.7/site-packages/hass_nabucasa/remote.py", line 216, in handle_connection_requests
    raise RemoteNotConnected()
hass_nabucasa.remote.RemoteNotConnected