I don’t use the configurator, I usually just edit the yaml config files right on the machine running home assistant using the nautilus file explorer in Ubuntu. The machine has a keyboard and monitor. I can still ssh into it or access it on my windows laptop through a samba share. Ssh and samba are running on the “bare metal” of the ubuntu machine.
I do have zwavejs2mqtt and zigbee2mqtt’s config pages accessible through iframe pages through home assistant, and yes those need to be configured with a reverse proxy for external access. Home Assistant won’t proxy them for you. For a reverse proxy I use the Swag container which combines nginx, duckdns, letsencrypt (for secure ssl certs/access) and fail2ban. It sounds like you already know how to deal with reverse proxies, but if you’re interested in SWAG I set that up using the instructions from this post
Another option instead of or in addition to a reverse proxy is a VPN. Then when the VPN is connected all the local http addresses would work just like you’re in the network even though you are away. I have wireguard for that and wrote up this guide about installing it in a container