Are you not able to connect via the iOS app at all or only when you’re on your home network? If you can connect with Wi-Fi turned off on your iDevice then you know that you’ve got port forwarding set up correctly, and if a computer on you network (? other than the one hosting the server) can connect to your DuckDNS address then hairpin NAT isn’t an issue.
To be honest I’m not entirely clear why there are separate internal & external URL entries in the companion app- I’ve left my internal URL field blank and it works fine. Now that I think about it, maybe that’s the issue… the internal URL entry is probably looking for a local IP address.
Edit: so try leaving internal URL blank. It won’t work if you internet goes down, but without custom DNS routing in your router or another “split-brain” setup (which I haven’t figured out yet) it’s always going to be that way once you’ve secured HA with a certificate.
Thanks for the reply. I’ll try leaving it blank and see what happens.
Also, I assume this changes things, but I can’t connect internally on any device now.
Was anyone able to figure this out without using something like NAT loopback? I had been successfully using the “Internal URL” and “External URL” settings within the app for years, but I recently bought a new phone and figured it would be easy to just put the settings into my new app and everything would work just as before. This was a mistake on my part.
I can connect to my server successfully as long as I am not connected to my local WiFi network (for example, if I turn WiFi off and use my cellular network). The app will connect using my External URL, and my router is configured with port forwarding to my Home Assistant server.
When I’m on my local network (for example, via WiFi), the app correctly detects this and attempts to use the “Internal URL” I have configured. But I get a certificate warning (NSURLErrorDomain 1202) and it is ultimately unable to connect. Note that if I point a web browser at the Internal URL I have configured, it works fine (after I clear the certificate warning).
The certificate warning makes sense because the Internal URL I am using is a 192.168.x.y IP address. But I want to stress that using different External and Internal URLs was working just fine until I went to set up the app on a new phone. I did not change any other settings.
You could set up your own dns server, either on your router, or as addon for HA…
You could use hairpin NAT, if your router supports it…
You can set up HA’s addon NGinx as reverse proxy
There are so may ways to do this…
While I truly appreciate the help and the suggestions, I think the proposed solutions kind of miss the point. Setting up a DNS server might fix the issue, but this is now another semi-complicated component to maintain (and creates a new attack vector). My router does not support hairpin NAT and I don’t have a desire to buy a new one and set up my entire network again just to fix this one issue. Setting up a reverse proxy (via something like NGinx) is, again, another component to maintain in order to solve a very specific and bizarre issue (and is, again, a new potential attack vector).
All of these ideas represent new and additional components to fix something that, as recently as last night, was working properly using components/options that are already built into the app, seeming for this express purpose. The app has the ability to detect what network I’m on based on the SSID I am currently using, and it actively switches (seamlessly!) from using the External URL to using the Internal URL. I have to imagine the developers created this functionality specifically to solve the issue described here. I have no idea why it suddenly stopped working, but I would assume there’s a way to make it work again without purchasing new hardware or installing additional components or workarounds. Perhaps it’s simply a new bug?
Interestingly, the simplest solution right now would be to simply stop using the app altogether when I’m on my local network. I can access my server using the Internal URL via my phone’s browser (Safari, Chrome, etc) without any problems – it seems it’s just the app that can’t. But I would obviously prefer if the app worked as intended.
No, this is how https is supposed to work since …forever…
I used to setup my own dns using the dns server of my router (so my internal url was https://ha.mydomain.net:8123 and resolved to local address 192.168.1.2 while my external https://ha.mysomain.net with port forward 443 to 8123).
Recently I configured hairpin nat on my router.
And the main reason people use NGinx is exactly to overcome your issue….
Understood. So how was it working for the last several years without hairpin NAT or a reverse proxy?
What changed? Why was I able to access the server with my app over https via my Internal URL yesterday, but not today? Is the overarching issue simply that the app will not allow me to bypass the certificate mismatch error (while the browser will)? And perhaps I had the app on my old phone for so long that I had been preserving some buried configuration flag through various app updates that allowed me to bypass the certificate error, but this flag can no longer be set on new installs?
The iOS app is not able to ignore certificate errors; adding logic to do so would be possible but also cause extended battery issues, particularly on the Apple Watch. My hunch is you were not actually using an internal URL before (e.g. location privacy prevented it, the SSID list was incorrect, etc.) and the external worked, but I don’t know how that would be possible without your router cooperating with NAT support.
Very good question, to which i do not know the answer…
Maybe you did not use https, but http?
Or maybe the phone was really old and still allowed https without valid certificate?
But i really don’t have the answer, it should not have been possible……
@ngrusz1, if you go back and look at my original post and the eventual solution, it was to go out and get an up-to-date router. If your router is old enough that it won’t do NAT loopback/haripin/whatever then it’s probably lagging in other areas as well.
{sigh} so, there’s a lot going on here. The good news is it looks like I was able to figure out the issue, although I don’t understand how it happened.
To address @fmon and the router issue: I have an Asus RT-AC68R. It is a relatively modern router and continues to be officially supported by Asus (their most recent firmware update was a few weeks ago, and I applied it to my router around that time). There is also an active open-source third party firmware (Asuswrt-Merlin) that still supports my router model. I mention this because it turns out that lack of configurable NAT loopback support is apparently a known issue for basically all Asus routers. The feature used to exist, but at some point it was actually removed (in both the official Asus firmware and the Merlin firmware). Googling the issue gets a lot of hits, and a good example can be found at this forum post here. Ironically, at the end of this post is a Home Assistant user pleading with the posters for a solution because they could not access their HA instance from within their local network.
In any event, my router isn’t too old for this feature, and it is not lacking in other areas. It is extremely odd that Asus removed support for NAT loopback, but it seems as though every modern Asus router will have this issue. It’s also a bit ridiculous that the only solution for this limited issue – that only impacts my use of the Home Assistant mobile app within my local network and nothing else – would be to purchase new hardware. I expect that kind of solution from Microsoft or Apple, not Home Assistant. And again, the mobile app seems to have configuration options for this exact issue, and at first glance it certainly appeared (see below) as though the issue was due to the app not allowing me to ignore the certificate warning when using my Internal URL.
While doing some additional searching this evening, I attempted to access my HA External URL from my PC. I was expecting nothing (similar to what I was getting with my mobile app). However, I actually got something: a very simple “403: Forbidden” error. This told me I actually was reaching my HA server, via my External URL, while inside my local network, and for whatever reason HA was rejecting my traffic. Some Googling got me to this forum post about the forbidden error – somehow HA had banned by router’s IP, preventing me from using my External IP within my local network. Initially, it did not seem like the mobile app was reporting the 403 error – I would get a certificate warning and then nothing would happen. But it looks like the 403 error may actually have been there…it’s just that my app is in dark mode, so it was displaying dark gray text on a black background, making it nearly impossible to see (once I suspected it might be there, I was able to see it by highlighting the text).
I checked my ip_bans.yaml file and the IP of my router was listed. I removed it and restarted the HA service, and voila! – success. My mobile app is working again (using the External URL). Based on the various forum posts regarding the sudden 403 error, I will probably disable the IP ban feature if it happens again, as I don’t know why my router IP was banned.
So in my case, perhaps a suggestion to the mobile app devs about an update that ensures error messages will not be displayed in dark text on a dark background might be the best “solution” for now
It still seems odd, however, that the Internal URL feature does not seem to work here. Is it because it will not allow the user to ignore a certificate error? Something else?
Take a look at chapter 4.2.2 DHCP Server (p60)
There you have to option to define a DNS under step 3:
In the Domain Name text box, enter a domain name for the wireless router.
You can use this box to specify your external (duckdns?) dns name (using used.dns as example)
Next step is to make an IP reservation (step 8); it only seems to lack a field for the name, but it should be able to lookup the local device by name (the name you gave your HA, as example I used ‘my’ below)
Once everything is configured, you should be able to use:
The whole idea is that when ‘outside’ your phone will lookup a DNS from your provider, resolving my.used.dns to your external internet address. Your router will forward port 443 to 8123 (which is NAT)
However, when you are inside, you want your router to resolve dns my.used.dns to use your local network for the domain used.dns and resolve the local address ‘my’ to the local IP address of HA (192.168.x.y)
This used to be my temporary ‘fix’, until I had to time to figure out how my hairpin NAT worked (Im using an EdgeRouterX)
I smell what you’re cooking there. It looks like I already had my domain name set in the DHCP configuration. This might explain how my configuration seems to have worked before my HA instance banned my router’s IP. And this configuration makes it appear as though the traffic hitting HA from a system inside my local network is coming from the router’s IP and not its own IP (when using my domain name). This is probably where the lack of NAT loopback is hurting me, because when it banned my router’s IP than meant everything inside my network that tried to reach HA (via my domain name) would be refused since it would appear to be coming from the router’s IP.
It appears that this is working again, and hopefully Asus will decide to add NAT loopback to their routers again.
I have a newish ASUS also- at some point in the past the NAT loopback control was removed and now it’s always on. So that’s why you didn’t have an issue before you got banned.
Edit: I actually posted this above in Jan of '20 ha
Just for fun, in case anyone runs into this issue again: it looks like a recent firmware update to some Asus routers has added a feature that can cause this problem to reoccur. The firmware update adds a new option under the WAN configuration called “Forward local domain queries to upstream DNS” and it is turned off by default. If you update your firmware and the connection issue described here occurs, you may need to turn this option on.
