Multiple Networks Issue

I just setup two separate networks for my home with two separate routers. I’ve got my main network which is connected to the internet and I’ve got my IOT network which is not connected to the internet. I’ve got Home Assistant running in docker on Ubuntu. When I connect the computer to both of these networks through ethernet Home Assistant errors out when trying to reach out to the internet. I’m able to ping and use wget for the same websites Home Assistant tries to hit but whenever Home Assistant tries to reach out to the internet either for app push notifications or dark sky weather, I get either a timeout error or a host unreachable error.