While I won’t deter anybody from using Docker, I would like to share my view and experience. It’s definitely a convenient and flexible option in many ways.
I’ve been running HA Core since I started (about 5 years now).
I find a Python upgrade not too hard. I simply do a Python altinstall from sources, which I can do without shutting down HA. Tearing down and setting up the new venv takes seconds (during which HA needs to be down, of course). Of course, to keep up with your Python versions, you need to install and upgrade certain dev packages and headers. None of that’s an issue, but at some point the host must be upgraded too (this is true for Docker too, so no real difference there).
I find the above much less of a hassle than having to fiddle with paths to access things outside of the container (a highly debatable point, I know, but that’s my preference). Philosophically it also really bothers me, because that then means things aren’t really self-contained anymore, right?
Also, most people will run Docker as root. I’m not doing that, personally. There are some things to consider (and many more posts on this topic) if you don’t want to do that. This matters to me, because I’m not simply accessing attached storage (like a NAS or USB device), plus those devices may not be available or plugged in when you start the container. I have much better control over permissions too (e.g. in the udev rules).
In the end, YMMV.