Running HassOS as an LXD/LXC virtual machine

[mjmccans]

@thiscantbeserious Unfortunately I don’t think my findings will be directly helpful for you as I was not encountering the same error, but they may help point you in the right direction. The issue with mDNS, as I recall, is that qemu needs to have the interface have trustGuestRxFilters=‘yes’ set for mDNS traffic to flow through. I was not able to find a way to do that for LXD VMs as part of the VM’s configuration, but what I was able to do was to tell systemd to change the settings of the network interfaces when they are brought up so that all multicast traffic flows through. On my system, which is a Ubuntu Focal host, I created a file called /etc/systemd/network/lxd-vm-multicast.network with the following content:

[Match]
Name=mac*
[Link]
AllMulticast=true

This allowed mDNS traffic to flow into the VM, which is what I needed to get the Chromecast integration working (and is likely required for others). The approach I took may be a bit blunt, but it works for my use case as Home Assistant is the only VM that I run and it has been working well for me ever since. I am always open to constructive criticism and I am by no means an expert on this topic, so happy to get any feedback about how I could have done this better if you or anyone else has any better ideas. It took me a lot of Googling around to figure this out, so hopefully some of the keywords above help you out (I made some progress searching “trustGuestRxFilters”).

[andynbaker (Andy)]

I’m sure you guys know this and have decided to manage your own VM host using Ubuntu (I think?), but proxmox makes this sooooooooooo easy. Like easy enough for someone like me who had no idea what they are doing to stand up a HAOS VM, set up regular automated backups, and host a few other services on the machine in docker running in an lxc. It’s just made for this sort of thing. All in, my host runs the same services with less CPU utilization compared to what I was doing before (core in docker on Ubuntu desktop).

[LarsNorgaard (Lars Nørgaard)]

Has anyone got this working with macvlan under lxc?

As soon as the script gets to network-manager setup, it kills the connection and it won’t come online again.

Power usage and performance in LXD/LXC is way better compared to a VM, that’s why I want to run this in LXC

[mjmccans]

I did have Home Assistant Core working under lxc with macvlan so it can function. However, I take it from your comment you are talking about Home Assistant Supervised in a container and that is something that I never did try so I cannot provide any help with your network-manager issue. I too would much prefer to run Home Assistant under LXD/LXC instead of a VM for performance and resource usage reasons, but I wanted to run a supported installation so I decided to go the VM route. I completely understand why the development team wants to limit which installations are supported, and I am very happy that they are still supporting Supervised on Debian 10. However, in my ideal world I would be able to run Home Assistant Supervised in an LXD container as a supported configuration.

[deviantintegral (Andrew Berry)]

I’d echo everything above about running in lxc. The single qemu VM I have for Home Assistant is easily the biggest use of CPU even when “idle”, as compared to all the other lxc containers I run on the host.

I’m a little confused about https://github.com/home-assistant/supervised-installer - it references qemu for all the machine types, and so I thought that meant it would set up a VM. However, it looks like the installer script pulls docker containers with qemu in their name?

Is there any downside to running the docker install directly inside an lxc container? Does that mean you lose access to the supervisor addons?

[Delurk]

Hi,

I try to launch a vm from the qcow2 image but nothing happen after the vm starting.

@mjmccans can you share your way to run it ?

[mjmccans]

When running the qcow2 image on my machine there was no output as well, but the VM was actually running. In addition, it did not show an IP address when running ‘lxc list’ because the lxd-agent is not running in the VM which means that the IP address is not reported back. In fact, it took me an embarrassing amount of time to realize that the VM was actually working despite giving no output. That being said, I did follow the instructions at the following link and was able to connect to the VM through virt-viewer on a Windows machine I had around and was able to get access to the machine that way: https://discuss.linuxcontainers.org/t/vga-console-connect/8814/5

Your experience may be different than mine, but you may want to try to look at your DHCP server or otherwise check out your network to see if the VM has in fact started up. If you can get the IP for the machine you can access the web interface. I hope that this helps.

[Delurk]

Thank you @mjmccans

Your way works very well without the console and network aren’t displaying, but I’m now at the same state : without lxd agent I can’t use hardware (conbee gateway) through the host.

[thiscantbeserious]

I’m currently not using my strives but am running HassOS directly … since I didn’t have to much time at hand however see this comment from whiskerz007 about the issue in regards to multicast here:

I've forked this for a generic LXD install on Ubuntu Server (and theoretically any OS that is able to run it)

[ukro]

Yo guyz, what is the situation. So is it possible to run hassos in LXC, i need to pass google coral to the frigate addon and i have some bumps with proxmox. So i think passing the pci device to the CT would be easier.
UPDATE:
I managed to pass google coral to proxmox VM

[xiki (Daniel)]

Hi guys, so far I managed to run the operating system on an LXD virtual machine but I still have some problems. This is what I did so far:

After installing LXD using Snap and initializing it:

1. Downloaded HA OS qcow2 image from here
2. Created a metadata.yaml file with the following contents:

architecture: x86_64
creation_date: 1624888256
properties:
  description: Home Assistant image
  os: Debian
  release: buster 10.10

3. Compress the metadata.yaml file: tar -cvzf metadata.tar.gz metadata.yaml
4. Create the LXC image: lxc image import metadata.tar.gz haos_ova.qcow2 --alias hassio
5. Create LXC VM from the imported image: lxc launch hassio ha --vm -c security.secureboot=false

At this point I can do lxc console ha to get into the HA OS VM’s console. Two problems I have right now.

1. Once I get into the console and login as root (without password), I go straight into the # prompt, skipping the hassio prompt.
2. The IP address is not obtained for the instance from outside the VM. I need this of course to access Home Assistant service on the browser.

Any help with this is greatly appreciated! I also posted in here for assistance:

VM from disk Image - LXD - Linux Containers Forum
LXD create VM for Hassio · Issue #118 · whiskerz007/proxmox_hassos_install (github.com)

[xiki (Daniel)]

Well it worked when using a bridged network connection. My only issue is that I can only use lxc console NAME to run commands in the VM and this does not work if you are writing a bash script to automate things. Someone has a workaround?

[xiki (Daniel)]

I bumped into an issue trying to run the script you provide on a debian 10 raspberry pi 4, with module loading. I can pass the overlay module to the lxc, but not aufs. I get:
Error: Failed to load kernel module 'aufs': Failed to run: modprobe aufs: modprobe: FATAL: Module aufs not found in directory /lib/modules/5.10.0-0.bpo.7-arm64

I’m quite stuck as google doesn’t give me any good results. Would appreciate any help!

UPDATE

My actual issue is that not even overlay storage driver was working for me since I was using ZFS as my LXC storage pool. I had switch to BTRFS ( and also skipped from passing aufs module as I didn’t have this on my arm64 debian kernel 5.10 – also doesn’t seem to matter as I will be using overlay storage driver for docker )

I bumped into another brick wall, and this time it’s about the USB. I cannot seem to passthrough the USB I have on host to the Home Assistant setup.

I have added a USB like so:
lxc config device add homeassistant usb1 usb vendorid=<vendorid>

Inside the container I can see that there is a new folder created something like: /dev/bus/usb/001/005
I find this quite weird. There is no serial folder inside dev. I’m not sure why Home Assistant cannot detect the USB. Would appreciate some help!

[kingat (kinghat)]

how did this all end up working for you in the end? also, why not just run the supervised installer inside of a lxc debian image?

[lars1 (lars)]

Try adding your usb device like this:

lxc config device add homeassistant ttyusb unix-char path=/dev/ttyACM0

[thiscantbeserious]

Figured its much simplier to just run Hass OS … and the Portainer Addon for my purposes.

Whenever I need anything I can simply pull the Image from Dockerhub and create an Container easily besides Homeassistant on the same Host.

Messing with LXD was a nice try but far from stable / complete … Homeassistant is moving much to fast for that so it’s best to stick as close to their supported installations I guess.

I’d use their VM-Images next time instead If I’d want to run it on something more powerfull.

[seanblanchfield]

I got Home Assistant OS running in LXD without too much difficulty, and wrote up my notes here:

How to Run Home Assistant OS under LXD

Hopefully that’s of help to anyone who finds this thread looking for a guide.

[Wolvboot]

Very nice! I actually asked about this a month back and @tteck (famous for his proxmox setup repo) told me it was technically impossible, so I gave up and put HASS in a VM. I don’t see why LXD would be materially different than Proxmox’s implementation of base lxc.

You didn’t set that stuff up, but I see no reason why USB passthrough and nested docker (HASS OS “addons” are actually docker containers) wouldn’t work too.

Edit: Nevermind, you’re running a VM, not a container. Womp womp.

[seanblanchfield]

I’ve updated the title of the blog post to hopefully avoid future container vs VM confusion.

[Wolvboot]

Totally my fault, you did say VM. I keep forgetting LXD does both lxc and VMs now. Used to run it myself before I moved to proxmox.