Home assistant not getting galaxy watch to connect to wifi when used

So I have recently decided to try and work out the kinks I am having with some things in HA (Wow, there are a lot!)
Anyway, the biggest one right now seems to be the HA app for WearOS. I can get it to work in 1 of 2 ways.

  1. Turn off bluetooth. This forces the watch to stay connected to Wifi (Which is the problem)
  2. Go into the settings, go into connections, go to wifi, and wait for it to connect. Then operate my watch as fast as I can back to Home Assistant for WatchOS and hopefully I will be able to press 1-2 things before the wifi disconnects again (but usually not)
    Any ideas?
    It is a Samsung Galaxy Watch 4.