Just to add a solution that worked for me in order to put a router between HA and my ESP. Here is the config of the ESPHome that is behind the router:
wifi:
ssid: xxxx
password: yyyyy
fast_connect: true
use_address: 192.168.0.144 #this is the WAN IP of the router.
manual_ip:
static_ip: 192.168.4.144
gateway: 192.168.4.1
subnet: 255.255.255.0
On the router, you need to configure port forwarding for 6053 and 8266 ports.
When integrating the ESPHome device in HA, I use the WAN IP of the router.
I am using this w/o problems for 2 years now.
Hope this helps.