I’m hoping to have some time to update this in the next couple of weeks. I’m going to do a lessons-learned setup and use quadlet for the systemd files – and put all of that in a git repo to share with everyone.
2 Likes
maybe newer versions? i’m stiil using podman 3 on my rpi4
Didn’t have much time to play with this this summer. Going to update it soon though, either using quadlet or podman play kube.
Hi @mattdm , I’m currently in the process of switching from an arm server to an Intel NUC running Fedora Server 37 with container running using Podman. Before I make the jump I’m researching everything as best I can to understand a problem should it arise.
I have a few questions about your setup , currently I’m on the fence about whether to use systemd or podman play kube I was thinking of running the containers from my docker-compose file with podman compose and then generating the systemd and kubernetes yaml files with podman and then deciding from there.
I take it you are running all your containers, rootless and unprivileged? do you run esphome at all? I notice esphome’s docker compose runs it as privileged I susoect this is for access to USB on the host. If needs be will your selinux module allow usb access to esphome and home assistant? (I use home assistant to manage my zigbee device rather than a another container).
Do you run any containers in pods or are they all stand alone containers?
Is there a benefit to placing your config in /srv/ for exanple from a selinux perspective? I intended to store everything in /home/ as I don’t believe I will create other users that will need access to the files.
What issues do think there will be with podman 4.0 and networking ?
Finally, I use Caddy for my reverse proxy setup and I was going to just change the privileged port down to 80 so I can run the container rootless , but do you recommend your option above of forwarding 80,443 to 8080,8443 and setup caddy to listen on those ports?
thanks in advance
Bookmarking this thread too have a look at it.
I want to move from Docker-Compose to Podman but the proper way and not just copying my Docker setup through e.g., Podman-Compose.
Did you consider working with pods?
I.e. HA and all its add-ons in one pod, so that they can communicate via localhost and you only expose what’s needed?
Identifying which ports I’d need to expose is where I’m getting stuck at 
Or is it best to stick to individual containers?
Does the use of VLANs make sense in that case? I.e. containers that don’t need internet (like Z2M, hassconf, MariaDB) could go into a NOT VLAN?
Are you running this rootless?
Most stuff uses high ports anyway and Nginx can be exposed on a high port and port-forwarded on 80/443 anyway.
Things would only get tricky if you want to set up e.g. PiHole.
Why would you gor for IoT and not Core?
As you undoubtably are more experienced in Fedora than me, I’m curious to hear your opinion.
EDIT1:
Found a typo in ~/.config/systemd/user/container-nginx.service:
--detatch should be --detach or just -d
Additional question: How do you get the IP of the Home Assistant container (10.0.2.2) and how are you certain it’s static?
It’s the virtual gateway for the podman network
1 Like
I have no experience with SELinux. First time Fedora user as well.
Does the following mean SELinux may not be installed/enabled on my Fedora CoreOS system?
$ checkmodule -M -m -o container-usbtty.mod container-usbtty.te
$ semodule_package -o container-usbtty.pp -m container-usbtty.mod
$ sudo semodule -i container-usbtty.mod
libsemanage.semanage_pipe_data: Child process /usr/libexec/selinux/hll/mod failed with code: 1. (No such file or directory).
container-usbtty: libsemanage.semanage_pipe_data: Unable to execute /usr/libexec/selinux/hll/mod : No such file or directory
container-usbtty: (No such file or directory).
libsemanage.semanage_direct_commit: Failed to compile hll files into cil files.
(No such file or directory).
semodule: Failed!
I already needed to install checkmodule, as it wasn’t installed, using
$ rpm-ostree install checkpolicy
$ sudo rpm-ostree apply-live
Hey all — just wanted to note that I’ve seen your recent comments. Been busy lately and haven’t had time to update this.
I really like the quadlet concept, but I think maybe play kube is going to be the longer-term and more portable winner. Not sure, though. I might end up doing… both?
I’m running all of the containers under a hass user, so they’re rootless and I’m not using privileged for any of them currently. (This does mean that DHCP discovery doesn’t work. Haven’t tried to mess with the specific privs.) I’m not using esphome, but you shouldn’t need privs or root access for USB access as log as the user is in the dialout group (and see the selinux stuff in the first post).
Once upon a time when I first learned OpenShift stuff a few years ago, I was taught “1 container per pod is a the right choice, unless they are extremely coupled and share the same lifecycle”. Since these services are fairly loosely coupled and can (mostly) be updated and started/stopped/restarted independently, I chose to separate them. I’m open to revisiting that, especially if that particular bit of advice hasn’t really survived into modern Kubernetes practices. I try to keep up, but all of my hands-on stuff is small potatoes.
Not really sure — it’s just different, so I want to be sure I’m not missing anything.
Plusses and minuses — using high ports means it’s easier for a different non-root process (maybe even running as another user) to grab those ports. But using the low ports directly means you need root privs at some point.
1 Like
Hi @mattdm
I’ve read through this whole thread but I still get stuck at permissions for the Zigbee serial adapter.
I even tried readlink -f as suggested above.
/usr/bin/podman run \
--cidfile=%t/%n.ctr-id \
--cgroups=no-conmon \
--rm \
--sdnotify=conmon \
--replace \
--detach \
--label "io.containers.autoupdate=registry" \
--name=zigbee \
--group-add keep-groups \
--network=slirp4netns:allow_host_loopback=true \
--device=$(readlink -f /dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_ee6789cf0260ec11ba5a345f25bfaa52-if00-port0):/dev/zigbee:rw \
--volume=/srv/hass/zigbee:/app/data:Z \
-p 8081:8081 \
docker.io/koenkk/zigbee2mqtt:latest
$ cat /srv/hass/zigbee/log/2022-11-30.10-59-24/log.txt
info 2022-11-30 10:59:24: Logging to console and directory: '/app/data/log/2022-11-30.10-59-24' filename: log.txt
info 2022-11-30 10:59:24: Starting Zigbee2MQTT version 1.28.2 (commit #360a777)
info 2022-11-30 10:59:24: Starting zigbee-herdsman (0.14.68)
error 2022-11-30 10:59:24: Error while starting zigbee-herdsman
error 2022-11-30 10:59:24: Failed to start zigbee
error 2022-11-30 10:59:24: Check https://www.zigbee2mqtt.io/guide/installation/20_zigbee2mqtt-fails-to-start.html for possible solutions
error 2022-11-30 10:59:24: Exiting...
error 2022-11-30 10:59:24: Error: Error while opening serialport 'Error: Error: Permission denied, cannot open /dev/zigbee'
at SerialPort.<anonymous> (/app/node_modules/zigbee-herdsman/src/adapter/z-stack/znp/znp.ts:146:28)
at SerialPort._error (/app/node_modules/@serialport/stream/lib/index.js:198:14)
at /app/node_modules/@serialport/stream/lib/index.js:242:12
$ groups hass
hass : hass dialout
$ ls -al /dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_ee6789cf0260ec11ba5a345f25bfaa52-if00-port0
lrwxrwxrwx. 1 root root 13 Nov 30 07:35 /dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_ee6789cf0260ec11ba5a345f25bfaa52-if00-port0 -> ../../ttyUSB0
$ ls -al $(readlink -f /dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_ee6789cf0260ec11ba5a345f25bfaa52-if00-port0)
crw-rw----. 1 root dialout 188, 0 Nov 30 07:35 /dev/ttyUSB0
As mentioned before, I’m running Fedora CoreOS, should that make a difference. I’m a first-time Fedora user as well.
$ cat /etc/os-release
NAME="Fedora Linux"
VERSION="36.20221030.3.0 (CoreOS)"
ID=fedora
VERSION_ID=36
VERSION_CODENAME=""
PLATFORM_ID="platform:f36"
PRETTY_NAME="Fedora CoreOS 36.20221030.3.0"
$ podman version
Client: Podman Engine
Version: 4.3.0
API Version: 4.3.0
Go Version: go1.18.7
Built: Fri Oct 21 08:16:35 2022
OS/Arch: linux/amd64
# as sudo-user
$ cat ~/container-usbtty.te
module container-usbtty 1.0;
require {
type container_t;
type usbtty_device_t;
class chr_file { getattr ioctl lock open read write };
}
#============= container_t ==============
allow container_t usbtty_device_t:chr_file { getattr ioctl lock open read write };
$ checkmodule -M -m -o container-usbtty.mod container-usbtty.te
$ semodule_package -o container-usbtty.pp -m container-usbtty.mod
$ sudo semodule -i container-usbtty.mod
libsemanage.semanage_pipe_data: Child process /usr/libexec/selinux/hll/mod failed with code: 1. (No such file or directory).
container-usbtty: libsemanage.semanage_pipe_data: Unable to execute /usr/libexec/selinux/hll/mod : No such file or directory
container-usbtty: (No such file or directory).
libsemanage.semanage_direct_commit: Failed to compile hll files into cil files.
(No such file or directory).
semodule: Failed!
It’s probably SELinux, given those errors. Does it work with sudo setenforce 0?
If that works, try turning it back on again (sudo setenforce 1, or reboot), and then use sudo semanage boolean -m --on container_use_devices. This will allow containers to use all devices (more dangerous, obviously, but they at least do have to be mapped in first), and will persist across reboots.
If that works, we’ll figure out what’s going on with semodule.
It works when I manually do podman run.
When I do systemctl --user start container-zigbee.service it fails but because of a different issue: the only error I see in journalctl is Error: reading CIDFile: open /run/user/1001/container-zigbee.service.ctr-id: no such file or directory.
I know you set cidfile in all of the service configs, this is the first to complain.
Any reason you use this instead of the name for stop and rm?
Looks like the -f parameter of readfile is interpreted as a parameter for podman run when you use it in the .service file. However, with sudo setenforce 0 I can use the symbolic link. Even using /dev/ttyUSB0 when setenforce is set to 1 again, doesn’t fix it.
$ sudo semanage boolean -m --on container_use_devices
sudo: semanage: command not found
Looks like Fedora CoreOS may be adding some additional challenges?
@mattdm
Another thing I’d like to do is mount /run/dbus in the HA container so it gets access to my built-in Bluetooth module.
I’m getting permissions errors on that as well.
@mattdm I have written down my experiences so far in a blog post.
Most of it is based on this forum thread. I then also added ESPHome and HASSConfigurator, and my backup setup.
Sadly, I still have to run with setenforce 0 but at least Home Assistant is alive again 
1 Like
When I get some cycles I’ll see if I can help you figure that out, or find someone who can. 
1 Like
so I managed to switch from my current ARM server to a NUC running Fedora server edition, I got everything working thanks to @mattdm and @DezeStijn.
I had the same issue as @DezeStijn with the zigbee device, the Selinux module didn’t work, same exact error
libsemanage.semanage_pipe_data: Child process /usr/libexec/selinux/hll/mod failed with code: 1. (No such file or directory).
container-usbtty: libsemanage.semanage_pipe_data: Unable to execute /usr/libexec/selinux/hll/mod : No such file or directory
container-usbtty: (No such file or directory).
libsemanage.semanage_direct_commit: Failed to compile hll files into cil files.
(No such file or directory).
semodule: Failed!
I had to use the allow container to access all devices boolean although I did it in cockpit which gave me the command
setsebool -P container_use_devices 1
I assume it does the same thing as your semanage boolean
I was still having issues until I realised I needed to add
--group-add keep-groups \
option from your zigbee systemd unit file into my home assistant systemd unit file
I could then add my zigbee device in ZHA integration and it worked perfectly.
oddly enough in the home assistant container if I run the command
ls -la /dev/zigbee
crw-rw---- 1 nobody nobody 188, 0 Dec 5 12:35 /dev/zigbee
the device is in group nobody and owner nobody
and if I type
groups
root nobody nobody
not sure what to make of this, but it works
also weirdly enough auto discovery did work for some of my integrations but not all, I did open a few more ports on the firewall which is probably what did it.
however I still get the error message
Cannot watch for dhcp packets: [Errno 1] Operation not permitted
so obviously it shows discovery is not working fully
1 Like
Ah, yeah, that totally makes sense — in my zigbee2mqtt setup, Home Assistant itself doesn’t need the access.
This is fine. Inside the container, Home Assistant is running as what appears to be root. It doesn’t know that that’s mapped to a non-root user outside the container. So the actual group doesn’t matter.
Yeah, snooping on DHCP packages requires the container to have actual root-level privileges. It would be possible to run the container that way (or possibly with just that particular capability), but for me DHCP discovery isn’t all that useful so not really worth it.
2 Likes
Hi all!
I have HA running in a rootless container with podman and I have managed to connect to another rootles container running ZWaveJS UI. So far so good. I am though having problems with getting tmy USB bluetooth adapter to work inside the rootles container. This is the way I am starting the container:
$podman run -d --privileged --name homeassistant --restart=unless-stopped --device=/dev/bus/usb/001/004 -v /run/dbus:/run/dbus:ro -v /etc/localtime:/etc/localtime:ro -v /etc/homeassistant.podman:/config --network=host ghcr.io/home-assistant/home-assistant:latest
As you see i’m binding both /run/dusb and the actual USB bluetooth dongle (/dev/bus/usb/001/004) but HA keeps on complaining that the bluetooth adapter is not found. I have opened a terminal in the container and bluetoothctrl bails out with the following message:
“bash-5.1# bluetoothctl
Waiting to connect to bluetoothd…dbus[418]: arguments to dbus_connection_get_object_path_data() were incorrect, assertion “connection != NULL” failed in file dbus-connection.c line 5921.
This is normally a bug in some application using the D-Bus library.
Aborted (core dumped)”
I have checked that /run/dbus exists in the container:
bash-5.1# ls -la /run/dbus
total 4
drwxr-xr-x 2 nobody nobody 60 Jan 20 07:46 .
drwxr-xr-x 1 root root 4096 Jan 20 10:20 …
srw-rw-rw- 1 nobody nobody 0 Jan 20 07:46 system_bus_socket
and as you can see it is there even if owned by nobody.nobody… I wonder whether it is a problem of uid mappings when running the container, but I am our of ideas. Anyone that has any tips?
I forgot to add that I have SELinux completely diabled
Thanks!
/Nacho
1 Like
Did you add the host user to the dialout group ? And then add
--group-add keep-groups
to the podman run command ?
Also I believe the only benefit to running as privileged is to enable auto discovery , I’m not sure if it helps at all with USB being passed through to the container
Hi!
Yes, the host user is in the dialout group (I’m using the same host use to run the ZWaveJS container in podman with access to the USB zway dongle… ). I have tried now as well with --group-add keep-groups but still the same. The --priviledged is just an attempt to test whichever flag I can feel could be relevant to see if it works… no success so far… 
Thanks for the suggestion!
New_info: It is definitely something with the DBUS access. I see this in the Podman logs:
“dbus_fast.errors.AuthError: authendbus_fast.errors.AuthError: authentication failed: REJECTED: [‘EXTERNAL’]”
Does not the access to DBUS get regulated by SELinux? But I don’t have SELinux enabled… Any DBUS guru around?
I think I read further down that you got past this, but I think this is a typo in the original post. The third line should be
$ sudo semodule -i container-usbtty.pp
I’m in the process of building out my HA environment and following this guide, so this may not solve everything, but it got rid of the error you’re seeing about no such file or directory.
Matt, you mentioned wanting to put all this into a git repo - just wondering if there was any movement down that path? Thanks for writing this up, I’m currently running through this on RHEL 9.
1 Like
I just haven’t had time to work on it. I’m optimistically hoping this weekend, but I keep thinking that every week…