Only allow refresh token when accessing HA via Home Assistant Cloud?

I have subscribed to Home Assistant Cloud. I know that good passwords should be used by all users, but as a way of defence-in-depth I’d like to have the option to disallow password based authentication when logging in remotely via the cloud; only refresh token logins should be allowed in that case. Is it possible to set things up this way?

I know there’s the option to only allow certain users to login via the local network and not via Home Assistant Cloud, but if I understand things correctly this would also block refresh token based authentication from remote - which is not what I would like.