Local devices and services using external url

I’ve been pulling my hair out trying to figure out why my local devices are all trying to use my external url. I’ve got home assistant in docker and all my IOT devices on the same vlan. The external URL is a domain I own port forwarded to my home-assistant hub and has a SSL cert managed by a proxy server. My internal url is a non secure regular local IP address. Most of my devices want to use the address of my external url which works ,but the return address is the vlan router which in this case is . This sort of works expect when the traffic gets heavy the signals start getting mixed up and I get weird errors like authentication errors, or incorrect data type errors. My internal url works just fine it’s just the device always choose the external url.

My question is how do integrations and devices decide which url to try to report back to? How can I make my local devices use my local url instead of my external url when they don’t have that configuration option?