HTTP GUI access from ethernet port and WiFi

We have HA running on a Pi4 using the supplied image. All working fine. The Pi is connected using the Ethernet plug to a separate network containing some IoT devices which it can communicate with. If I connect my laptop to the network I can access the GUI with no problem. It is also connected using WiFi to another network. Both networks show up green with the nmcli con show command. If I disconnect the Ethernet cable I can logon to the GUI using the address on the WiFi based network. But when I reconnect the Ethernet cable access is no longer possible and I have to use the address on the Ethernet network to connect to the GUI again. All is done using IPv4 addresses. In the configuration file I’ve tried adding the hhtp: server_host: command with the address on the WiFi based network and done a restart. It worked for some time but now it does not work anymore.
The problem is that I have to use a VPN to get remote access the WiFi based network for configuration of the HA system. The cabled network is not under our control so we cannot access that by using a VPN connection.
So why does it stop working on the WiFi network and is there any way to force it to use the WiFi network for the GUI?