A firewall is the correct solution for this though… Using a host-based firewall (Linux iptables or firewalld) is the best way to “restrict” access to ports locally on your device. You can simply allow the 443 from any host and only allow 8123 from localhost (127.0.0.1) or a specific remote proxy machine. Additionally, I would watch out if using any third-party plugins, as they sometimes open additional ports on the HA device (Node-Red, Samba, or SSH for example).
My router only NAPTs IPv4. All devices have their own public v6 address that is wide open to the internet.
If your router is a typical home gateway device (Linksys, Netgear, D-Link, etc) it should still be providing SPI/firewall coverage for the v6 addresses. Just because a device has a routable v6 address assigned to it, you shouldn’t be able to reach everything on that device from the outside. If this isn’t the case… I would be a lot more worried about everything else on your HA box being potentially exposed to the public. Typically you need to “allow” the port access from the public interface (not address) on the gateway to the IPv6 of the inside device, unlike the port-forwarding used by NAT’ed clients. Now, I have been using a pfSense appliance for years, so I don’t know exactly how modern SOHO gateways handle this.