If you are looking for a battle tested virtualisation solution that works consider Proxmox. There are other virtualisation platforms available like VirtualBox, VMWare, Hyper-V, etc but for open-source, linux standards based (KVM, LXC), enterprise grade with lots of community support look no further than Proxmox.
If stability is your thing, Proxmox has to be. Proxmox is Debian-based and therefore has one of the most compatible platforms out there. Plus you can go true datacentre grade with high availability and https://ceph.com.
Lastly you can run openmediavault, casaos, trueNAS, etc in a VM under Proxmox, as well as full Windows 10/11 Pro, MacOS and Android x86 VMs.
I run the Home Assistant container install in docker over Ubuntu on a Dell Optiplex. You lose the Home Assistant addons this way, but many addons are available as full programs you can manage and install yourself through Docker. I basically manage all the docker containers, including Home Assistant, through Portainer, which has a nice UI. Besides Home Assistant, I have Node Red, zwavejsUI(formally zwavejs2matt), zigbee2mqtt, wireguard, and swag/nginx all running as their own docker containers. On the bare metal Ubuntu machine, I have an attached hard drive I use as a network NAS and also run a Shinobi NVR system. That’s a lot of software on one machine, but docker is more light weight then a VM. Below is CPU and memory usage from the “top” command on the machine with everything running and you can see they are very low.
I tried a VM and noticed it used more CPU. The way a VM works is it needs to basically stand up another instance of the OS, where docker does this more through virtualization and sharing the CPU resources across the containers in a better way. I wrote up about this before on this post too which includes an article on VM vs container (ie docker).
I’m sure the VM/proxmox way will work too (there are many ways to run things), and I’m not all that familiar with Proxmox (I think it works slightly different from a traditional VM) but I definitely have had success with my docker setup, which I would say is more efficient in using the machine’s resources, and wouldn’t change it at this point.
Proxmox can run ANY OS or app out there. It is a proper type-1 hypervisor which is also capable of paravirtualisation (i.e. it can run under another hypervisor or another hypervisor can run under Proxmox assuming it supports paravirtualisation).
Did I mention you can run Proxmox inside Proxmox? Why? Check my Proxmox High Availability posts to get an idea. For example test a Proxmox change in an isolated instance, then commit to production or not.
Docker is no issue under Proxmox and you can run one or multiple docker instances as KVMs or LXCs. LXC are system containers, i.e. KVMs lacking the kernel - sharing the Debian-Proxmox kernel (Proxmox suggests running docker in a VM, i.e. KVM and I also agree with this).
If you plan to run multiple KVMs/LXCs you need plenty of RAM (suggest 32G to 64G) and no more than 16 cores, unless you will be running CPU heavy apps, in which case even 32 cores might not be enough.
I also suggest you run at least 2TB NVMe mirrored and you setup Proxmox over ZFS.
jchh
(my username is my initials (and my first name isn't John))
7
I run Docker on DietPi (a lightweight Debian OS) on my 2GB RPI 4. I also run Mosquitto, Z2M, Prometheus and AdGuard Home In Docker.
A lot of people are obsessed with Docker, which is fantastic if you only want to run things under Linux (that is Linux which shares the same kernel as the OS docker in running under).
Where VMs and type-1 hypervisors are useful is to be able to run any OS you want (FreeBSD, Unix, Windows, MacOS, Android, etc). Also networking is easier under a hypervisor instead of Docker.
For testing hypervisors are superlative. Why? Because with a single click you can copy an entire VM or System container. You can also run a mini data centre, something you cannot do with Docker.
You can try OVS to provide full SDN functionality.
To me Docker has it place, but it cannot replace a type-1 hypervisor, as Docker is for apps and Hypervisors are for systems and platforms.
jchh
(my username is my initials (and my first name isn't John))
9
I’m not obsessed with Docker, but it does also run under Windows and OSX.
I agree that docker is best if you only want to run things under linux (everything I run is on linux). A docker Home Assistant install really only works on docker over linux, because host networking mode (which Home Assistant requires) isn’t supported with docker on Windows or mac. If you need to use another OS the VM is the better bet at that point.
Thank you all for your replies!
As I don´t want to lose the ease of Addons I will use proxmox for HAOS and run a small VM with Docker for Radarr, Sonarr and so on.
To hopefully provide a broader perspective of the power of hypervisors you might want to consider the following. You have the either/or option or all options at the same time:
Run one Docker instance (KVM/LXC) with all Docker apps (Radarr, Sonarr, etc) under the same instance
Run multiple Docker instances (KVMs/LXCs) with only one app per docker instance
Run an LXC/KVM instance for every app (Radarr, Sonarr, etc), not using Docker, just an app install under whichever OS you decide to use
Run a single LXC/KVM instance with all apps (Radarr, Sonarr, etc) installed in the same instance
Use any combination of the above
If you change your mind in the future and want to split apps running in one instance, in two separate instances, just copy the VM, change the IP address if static, and delete the app or apps you do not want from that instance. Simple as that, taking no longer than a couple of minutes. Similar logic applies if you want to combine apps in a single instance.
Also for upgrades, test your upgrade by just copying the KVM/LXC, test, and if satisfied, make it your production instance; keep the old as a backup. You can move forward and backward like this in an instant (a few seconds is all it takes).
2 Likes
jchh
(my username is my initials (and my first name isn't John))
13
I’m been running running HA in* Docker on OSX for several years - works just fine. It’s true that network_mode: host is not possible so I do have to specify the ports. Here is my docker-compose…
Docker running over windows or mac is not listed as a supported install option for a reason - Installation - Home Assistant
If you don’t use any of the integrations that rely on host networking, just opening port 8123 will work fine. However, any integration that uses SSDP (list of them at this link) Simple Service Discovery Protocol (SSDP) - Home Assistant likely won’t work without host networking, and therefore couldn’t be used on a docker over windows/mac install. I believe there are others as well.
jchh
(my username is my initials (and my first name isn't John))
15
I cannot argue with you with it not being supported, but SSDP does work (for me). I have Philips Hue, Samsung TV & Logitech Harmony all discovered and running well.
I can’t explain it, but it does work. Maybe it only works by luck rather than intent?
Lol, probably a lot of things work by luck. Could depend on the machine’s network configuration, but I’ve read many posts where not having host networking mode gave other people trouble.
But like the old saying goes, if everything works for you, “if it ain’t broken don’t fix it”
1 Like
jchh
(my username is my initials (and my first name isn't John))
17
…but you do have me thinking. I’d rather fix address it now before it goes wrong which, of course, will be when I am 1000 miles away on holiday or something.
I do use VMs for other things so happy to try it for HA if it makes it more robust. Thanks for sharing the fact that Docker wasn’t supported on Mac - I didn’t realise that before now (I just knew that it worked).
