Data collection. It allows the devs to know how many installations of Home Assistant there is, where they are and when they’re online. The more often they report, the more precise the data.
IMO, what we need to be able to change:
“checkonline” URL to a different URL
frequency of the checks
disable the feature entirely
Disabling the feature might not be easily possible since there might be other components that rely on it, but the two others should be rather easy to implement.
If it is too problematic to implement quickly in the UI, a simpler way to do it would be through configuration.yaml file.
So what can we do with this?
Just block it in Pihole is not the way I want to go.
But what possibilities do we have?
Does anyone of you have a solution?
Well… One (crappy) solution is to use the dnsmasq addon to block the request on Home Assistant itself.
The issue is technically still there, but the DNS requests are blocked on the pi instead of the network’s DNS.
In case you want to try it out:
Install the official Dnsmasq addon from HA’s settings
In the addon’s configuration, add your network’s DNS server (e.g. Pi-Hole) in the upstream servers and remove the ones you don’t need
Still in the addon’s config, add the following in the Hosts section:
I block internet access to most of my IoT devices including HA, what i’m seeing is that once blocked HA spams my network every second, and i end up with around 15K entries in my PiHole logs every 24 hours. If i unblock it then it goes back to checking every 5 mins.
Wouldn’t blocking with dnsmasq cause it to spam itself? I see this also breaks updates, which I’m actually okay with.
EDIT:
Your solution worked great at blocking the checkonline spam from going out of HA but for whatever reason MQTT broker fails to connect now, Z2M can find the server just not HA even though its hosted on HA.
and beyond (from a dev’s view), I do not see the point why 15+ requests within one second are necessary for that although there is a working internet connection.
In addition to this, HA claims to be privacy-focussed which actually was one of the reasons why I chose it:
Open source home automation that puts local control and privacy first.
Wanting to know 24/7 to some minutes when someone is online seems to contradict that. At this point, I miss the personal control over it. I would prefer to have an option to control the interval or even to disable it “on board” instead of having to tinker with my home IT. (Nevertheless thanks to @syhlx and @johnsnow for their efforts! I wish there was an easy solution.)
I have taken a look at the code, since then, and it seems that the issue is likely to from come the fact that, since all HA components are isolated from each other, they are all make their own connectivity checks.
So, every component makes one request, but they all do it roughly at the same time.
Another solution to limit the large amount of DNS requests, would be to follow the same procedure as in my last post without adding the Hosts section. The Dnsmasq addon should cache the requests and ask your network’s DNS only once every 5 min, instead of 15x.
Note that this would only work if you are not blocking the requests with your local DNS (e.g. Pi-Hole).
It, definitely, will.
You can temporarly change the DNS in HA’s settings to check for updates.
It might have an issue with the IP of the Dnsmasq addon, try putting the IP of your HA (e.g. 192.168.x.x) as the DNS in HA’s settings instead of 127.0.0.1
It’s because the Home Assistant maintainers know better than us lowlife plebeians. /s
Could you clarify how tens of requests every five minutes for a version check adds to any meaningful data collection? If they wanted to know how many installs were out there, a single request would be sufficient, no? Reading more of the recent posts after this, I see this point was already made, but I wanted to reiterate it slightly.
You’d have to make some deep changes to the supervisor in order to stop the floods. The code is awkward and jumps around between a bunch of different modules which call the update checker function. As you’ve already mentioned.
I guess another alternative would be to not use any addons and disable all components; then we would magically stop all the DNS spam. But that’s an absolutely backwards way of addressing the issue.
I already have 2 DNS servers running on my network. One being Pi-Hole, which already extends the caching of records. The problem isn’t necessarily caching related, more that there are many many requests made in quick succession every five minutes. It’s traffic that’s flowing across my network that is useless and serves absolutely no purpose other than to clog up the tubes (the internet isn’t a big truck, it’s a series of tubes!);
additionally, it’s traffic that we as end users cannot control directly, we have no knobs or dials to turn to change how this is happening. That contradicts the whole privacy and control thing that is championed as one of the core values of the project.
Open source home automation that puts local control and privacy first.
HAHAHAHAHAHAHAHAHAHAHAHA
That’s an absolute knee-slapper of a joke.
Remember when they forced the banner about the home assistance conference into our settings pages? That was some great privacy and control we had right there.
This indeed worked MQTT is now working however, the dnsmasq logs show another address that I do not recognize, is it MQTT or perhaps the docker layer?
dnsmasq[221]: query[A] checkonline.home-assistant.io from 172.30.32.1
dnsmasq[221]: config checkonline.home-assistant.io is 0.0.0.0
dnsmasq[221]: query[NS] . from 172.30.32.1
dnsmasq[221]: forwarded . to 1.1.1.1
Yep so 172.30.32.1 seems like its the docker IP.
dnsmasq[216]: query[AAAA] cas.codenotary.com from 172.30.32.1
dnsmasq[216]: forwarded cas.codenotary.com to 1.1.1.1
dnsmasq[216]: query[A] cas.codenotary.com from 172.30.32.1
dnsmasq[216]: forwarded cas.codenotary.com to 1.1.1.1
so I’ve added another rule, which may not be required
the other odd thing is dnsmasq prefers 1.1.1.1 over my pihole iP on 198.168.x.x, so for now I’ve set my pihole ip and no backup. If pihole ever crashes then HA will have issues.
The forced connectivity check is the data collection. The multiple parallel requests is just bad design.
The supervisor has only one connectivity check in its code + the NetworkManager of its container.
Even with no addon installed, Home Assistant OS splits all HA components into different docker containers, which is a good thing for a number of reasons, but they all make the same requests on their own, which is far from ideal.
The idea of caching the request directly in HA is just to avoid sending a ton of useless requests across the network. They are handled on the same host without involving other network components.
I completely agree.
A better approach to this whole situation might be to add a (configurable) “network” component to HA to handle all the connectivity checks and network connectivity in general, so that we don’t have a lot of different components doing requests on their own without any control upon them.
My apologies, I didn’t mean to come off as combative with my comments. I’m just a tad frustrated with HA and how this is being handled.
But, wholly agreed with what you said and have already said in this thread.
Though, except the bit about running a third DNS server; yeah, it’d stay local on the machine, but would still eat some amount of CPU cycles, which, we both know is out of our control as it stands. I don’t find that type of shim really acceptable or a good workaround, because it’s a large compromise in some ways, yaknow?
CPU & RAM, which is rather scarce as well. But, I did say it was a “crappy” solution (to a problem that should not exist in the first place…)
It is simply resquesting the list of Root Servers. Pretty much all DNS servers do this.
Short of out right blocking port 53 (DNS) with a firewall or pointing to a non-existing IP for your DNS, I don’t think you can prevent these requests with Dnsmasq or, by extension, Pi-Hole.
I have ipv6 enabled, and it only seems to block the “internal” dnsmasq the ipv6 (AAAA) queries and the ipv4 (A) queries get through anyway, what should I change in the code to include those ipv6 queries too?
Dnsmasq config:
Update: I changed the network settings to my DHCP assigned HA IP address and now the queries are being blocked apart from the “.”. I don’t know what are those…
From Pi-hole:
Now this is my top domain haha a “dot”. Not funny at all Home Assistant supervisor programmers…This is certainly not a fix just a bad method to avoid your DATA COLLECTION!
Something else that you can do, is to disable IPv6 on your Pi.
It won’t magically make the problem disappear, but it lower the amount of DNS requests by more than half, with no side effect.
To do so:
On your Raspberry Pi’s SD Card, there is a file called “cmdline.txt”. It is on the first partition called “hassos-boot”.
Simply add “ipv6.disable=1” at the end of the line and save file. Do NOT change anything else in that file!
In the end the file should look something like this:
How can you reach this file if you are running a Virtual Machine (OVA) version of Home Assistant?
EDIT: found another solution without taking out the non existent SD card from the server, there is an option in the network settings to turn IPv6 off entirely.
I decided to migrate my HASS OS to a single LXC container on Proxmox, with Node-Red and ZwaveJS2MQTT. Took my 2 hours to migrate everything, no more weird DNS lookups and I actually feel better about everything is compartmentalized now. Proxmox on ZFS provides snapshots, restoring backups is a breeze.
I’m certain not everyone will want to take this path, but in case you were thinking about it it’s pretty easy to achieve.
