So I actually specifically asked about those when I joined. The reason the fallback is used on NXDOMAIN and REFUSED is because of this
It seems there was a rash of bugs at one point and the root cause was that self-hosted DNS server software had a habit of returning NXDOMAIN or REFUSED for AAAA requests when a domain only resolves on A requests. This was extremely problematic because github.com and ghcr.io don’t resolve on AAAA requests, they only have an ipv4 address. So DNS servers that did this basically broke all of home assistant. Every alpine based container (i.e. nearly every image we ship) thought github.com and ghcr.io didn’t exist anymore and was completely unable to check for updates or pull most of our images (among other things but this was the most obvious).
This was common enough that it was actually the reason the fallback was added. SERVFAIL is kind of expected once in a while with a local dns server, people kind of get what happened when they see that in the log. But random NXDOMAIN responses for sites they know exist and did not block in their software? That is confusing.
That being said the plan actually is to add the ability to disable the fallback. It’s the next step after this PR. Once supervisor is able to detect this confusing situation and inform users about it then we’re comfortable giving a way to disable the fallback. We just want to make sure users with a DNS server that is going to create problems are aware of it first so we don’t get a rash of bugs again.