Static USB ports via UDEV rules mapped into a docker container via compose.yaml do not work after reboot?

tl;dr; The symlinks to /dev/ttyUSBx devices mapped via compose.yaml to static USB ports in the docker container do not work after a reboot of the RasPi if the symlinks changed during that reboot and if the container image has not been newly composed after the symlinks changed.

Hi all,
I have a bit of a tricky issue and hope for the wisdom of the crowd to help me! I use three USB devices in my HA installation on a RasPi4 running on a docker container. One for Skyconnect (Zigbee), one to connect to my LG Chem batteries and my Studer Xtenders inverters via an XCom485i and then a RS485 interface to read values from some meters connected to a RS485 modus.

In the HA configuration, the three USB devices always need to appear on the same USB ports in the docker container since they are written into the configs.

Therefore I created “symlinks” via UDEV rules so the USB ports on the host device can be accessed via these static symlinks. here are my /etc/udev/rules.d/10-usb-serial.rules:

SUBSYSTEM=="tty", ATTRS{idProduct}=="ea60", ATTRS{idVendor}=="10c4", ATTRS{serial}=="ae94a629d960ed119dcdcde05720eef3", SYMLINK+="ttyUSB_SkyConnect"
SUBSYSTEM=="tty", ATTRS{idProduct}=="6001", ATTRS{idVendor}=="0403", ATTRS{serial}=="AQ00K6WL", SYMLINK+="ttyUSB_RS485"
SUBSYSTEM=="tty", ATTRS{idProduct}=="6001", ATTRS{idVendor}=="0403", ATTRS{serial}=="AB0KTQ80", SYMLINK+="ttyUSB_XCom485i"

The symlinks created after reboot are always correct (but do change from time to time):

0 crw-rw-rw- 1 root dialout 188, 0 Aug 25 20:39 /dev/ttyUSB0
0 crw-rw-rw- 1 root dialout 188, 1 Aug 25 20:39 /dev/ttyUSB1
0 crw-rw-rw- 1 root dialout 188, 2 Aug 25 13:53 /dev/ttyUSB2
0 lrwxrwxrwx 1 root root         7 Aug 25 13:53 /dev/ttyUSB_RS485 -> ttyUSB1
0 lrwxrwxrwx 1 root root         7 Aug 25 13:53 /dev/ttyUSB_SkyConnect -> ttyUSB2
0 lrwxrwxrwx 1 root root         7 Aug 25 13:53 /dev/ttyUSB_XCom485i -> ttyUSB0

Also the serial by-ID links are always created correctly after reboot (and again change from time to time):

0 lrwxrwxrwx 1 root root 13 Aug 25 13:53 /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_AB0KTQ80-if00-port0 -> ../../ttyUSB0
0 lrwxrwxrwx 1 root root 13 Aug 25 13:53 /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_AQ00K6WL-if00-port0 -> ../../ttyUSB1
0 lrwxrwxrwx 1 root root 13 Aug 25 13:53 /dev/serial/by-id/usb-Nabu_Casa_SkyConnect_v1.0_ae94a629d960ed119dcdcde05720eef3-if00-port0 -> ../../ttyUSB2

Here is where the tricky issue starts. In my compose.yaml file the devices section looks like this:

devices:
    - "/dev/ttyUSB_XCom485i:/dev/ttyUSB0"
    - "/dev/ttyUSB_RS485:/dev/ttyUSB1"
    - "/dev/ttyUSB_SkyConnect:/dev/ttyUSB2"

so I basically use the symlinks to map them to the static ports in the container so that I always access the devices via the right ports. So far so good, and everything actually works fine in the HA docker container if and only if the mapping of the USB ports is exactly like it was when I last run: docker compose up -d, i.e.:

/dev/ttyUSB_RS485 -> ttyUSB1
/dev/ttyUSB_SkyConnect -> ttyUSB2
/dev/ttyUSB_XCom485i -> ttyUSB0

However, if I reboot and the configuration ends up being e.g.:

/dev/ttyUSB_RS485 -> ttyUSB0
/dev/ttyUSB_SkyConnect -> ttyUSB1
/dev/ttyUSB_XCom485i -> ttyUSB2

then on the host system everything still works fine, i.e. if I access /dev/ttyUSB_XCom485i it is the correct XCom485i device that I expect, likewise if I access /dev/ttyUSB_RS485 it is the RS485 device that I expect.

However, within the HA docker container unfortunately, the ports seemed to be stuck with the configuration that was active during the last docker compose up -d, i.e. if I want to access the XCom485i in the container, it is still mapped to /dev/ttyUSB0 on the host system (which was the symlink for XCom485i during the last docker compose up -d) but that obviously does not work anymore since the symlink now points to /dev/ttyUSB2.

If I access the XCom485i on /dev/ttyUSB2 within the container, it does work, but that doesn’t help me since the configurations for the three devices are hardcoded in Home Assistant, i.e. in a modbus.yaml file, or via the Skyconnect configuration or in a pyscript for the XCom485i. But then the mapping in the compose.yaml file via the symlinks does not seem to do the trick that I really need? I.e. it does not actually newly map the ports from the symlinks to USB0, USB1 and USB2 ports as specified in the compose.yaml every time I restart the raspi only if I recreate the container.

Now I saw that @AseKarlsson and @mwav3 had a similar/related discussion almost 2 years ago here, but my particular issue described above wasn’t actually covered. So do you have any ideas or suggestions? Could it be that I would need to newly compose the HA docker container at every reboot to get the correct USB port mappings specified in the compose.yaml file after the current symlinks for the USB ports are created? If so how would I best do that?

Or would this be something that could be handled by the restart: policy in compose.yaml? I currently have restart: unless-stopped.

Any help would be greatly appreciated. It took my quite a while to get to this point to understand and verufy the root cause but I am stuck a bit, unless it is really that the containers will need to be recreated at every reboot? Is there any disadvantage to doing this? Obviously I would need to be careful with the image versions specified in compose.yaml and avoid ‘image: ghcr.io/home-assistant/home-assistant:latest’ so that it not automatically updates the HA version before I had a chance to test it.

Any help, idea, link, tip would be greatly appreciated. Thanks much!

Are you using privileged mode? Maybe you should post your entire service definition.

thanks, yes, the docker container is running in privileged mode. And I just did some more tests and set restart: no so the container does not automatically restart after a reboot. Then after the reboot I manually run the following commands to make sure the container is indeed newly created:

sudo docker stop homeassistant                                                                                                                                                                
sudo docker rm homeassistant                                                                                                                                                                  
sudo docker system prune -f                                                                                                                                                                   
sudo docker compose create homeassistant                                                                                                                                                      
sudo docker compose up -d homeassistant

And to my surprise, HA really only runs when the USB ports in the host system are assigned exactly as they are configured and needed in HA, which is when the following assignments happen to appear after reboot:

/dev/ttyUSB_RS485 -> ttyUSB1
/dev/ttyUSB_SkyConnect -> ttyUSB2
/dev/ttyUSB_XCom485i -> ttyUSB0

In HA, RS485 is configured to use /dev/ttyUSB1, Skyconnect is configured to use /dev/ttyUSB1, and XCom485i is configured to use /dev/ttyUSB0. So it seems like the mapping in the compose file actually doesn’t matter and the USB ports in the container are always 1:1 mappings to the host system?

If you mean the compose.yaml file with service definition, it looks like this:

version: '3'                                                                                                                                                                                  
services:                                                                                                                                                                                                                                                                                                                                                                      
  homeassistant:                                                                                                                                                                              
    container_name: homeassistant                                                                                                                                                             
    image: "ghcr.io/home-assistant/home-assistant:2023.8.3"                                                                                                                                   
    volumes:                                                                                                                                                                                  
      - /home/pi/docker/homeassistant:/config                                                                                                                                                 
      - /home/pi/docker/homeassistant/media:/media                                                                                                                                            
      # Directory for Home Assistant logfiles on the SSD drive (specified in "command:" below with the --log-file argument)                                                                   
      - /home/pi/ssd/log:/ssd-log                                                                                                                                                             
      # Directory for ZigBee databases on the SSD drive (specified in configuration.yaml, see www.home-assistant.io/integrations/zha/#configuration-variables)                                
      - /home/pi/ssd/zha:/ssd-zha                                                                                                                                                             
      - /etc/localtime:/etc/localtime:ro                                                                                                                                                      
    environment:                                                                                                                                                                              
      - PUID=1000                                                                                                                                                                             
      - PGUI=1000                                                                                                                                                                             
      - UMASK=007                                                                                                                                                                             
    devices:                                                                                                                                                                                  
        - "/dev/ttyUSB_XCom485i:/dev/ttyUSB0"                                                                                                                                                 
        - "/dev/ttyUSB_RS485:/dev/ttyUSB1"                                                                                                                                                    
        - "/dev/ttyUSB_SkyConnect:/dev/ttyUSB2"                                                                                                                                                             
    restart: unless-stopped                                                                                                                                                                               
    privileged: true                                                                                                                                                                          
    network_mode: host                                                                                                                                                                        
    ports:                                                                                                                                                                                    
      - "8123:8123"                                                                                                                                                                           
    depends_on:                                                                                                                                                                               
      - mariadb                                                                                                                                                                               
      - influxdb                                                                                                                                                                              
    command: python -m homeassistant --config /config --log-file /ssd-log/home-assistant.log                                                                                                  
    # Note: this is the standard CMD from the homeassistant docker file (https://hub.docker.com/r/homeassistant/home-assistant/dockerfile),                                                   
    # but adds the --log-file parameter to point to a non-standard log file location. I do that because the standard home-assistant.log file                                                  
    # is in the /config directory but I like it to be on the ssd medium where frequent write operations are less of an issue than on the                                                      
    # Micro SD card that HA runs from. You can just remove the entire "command:" line then the log file is created in /config.

This mapping is a bit odd I think, because when you use privileged mode they would automatically be exposed in the container.

I suspect these paths included in privileged mode are taking priority over your device mappings (which is why I asked), at least it did that in my testing. In other words, the host /dev/ttyUSBX paths are being exposed in the container, instead of what you have mapped them as.

First, I would drop your custom udev rules. Not worth it to maintain custom ones IMO. Second, I would not map the paths using actual device names, but just alias them as you did with udev:

    devices:                                                                                                                                                                                  
        - "/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_AB0KTQ80-if00-port0:/dev/ttyUSB_XCom485i"
        - "/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_AQ00K6WL-if00-port0:/dev/ttyUSB_RS485"
        - "/dev/serial/by-id/usb-Nabu_Casa_SkyConnect_v1.0_ae94a629d960ed119dcdcde05720eef3-if00-port0:/dev/ttyUSB_SkyConnect"

Then use /dev/ttyUSB_XCom485i, /dev/ttyUSB_RS485 and /dev/ttyUSB_SkyConnect in your configuration.

You can keep your udev rules if you really want, just don’t map them using existing device names, e.g.

    devices:                                                                                                                                                                                  
        - "/dev/ttyUSB_XCom485i:/dev/ttyUSB_XCom485i"                                                                                                                                                 
        - "/dev/ttyUSB_RS485:/dev/ttyUSB_RS485"                                                                                                                                                    
        - "/dev/ttyUSB_SkyConnect:/dev/ttyUSB_SkyConnect"

symlinks aren’t passed into the container with privileged mode, so that’s OK.

1 Like

Thanks so much @freshcoast! I used your suggestion with the by-id devices configuration in compose.yml and recreated the container with that. It somewhat worked. In the container I now see the newly mapped ports next to the system ports which is already a success:

And RS485 and XCom485i are working well with these new ports and were easy to reconfigure in the config files. But unfortunately I still struggle with the Skyconnect adapter, which seems to be rater unstable… I was able to get it configured to /dev/ttyUSB_SkyConnect with “Migrate Radio”, see steps here:




(I kept the radio network settings)


So far so good. But then I dared to click on:


and it failed pretty badly with the following error messages and HA was frozen afterwards:

2023-08-26 03:12:19.045 ERROR (MainThread) [zigpy.application] Couldn't start application
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/serial/serialposix.py", line 322, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 5] I/O error: '/dev/ttyUSB_SkyConnect'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 193, in startup
    await self.connect()
  File "/usr/local/lib/python3.11/site-packages/bellows/zigbee/application.py", line 133, in connect
    self._ezsp = await bellows.ezsp.EZSP.initialize(self.config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bellows/ezsp/__init__.py", line 164, in initialize
    await ezsp.connect(use_thread=zigpy_config[conf.CONF_USE_THREAD])
  File "/usr/local/lib/python3.11/site-packages/bellows/ezsp/__init__.py", line 181, in connect
    self._gw = await bellows.uart.connect(self._config, self, use_thread=use_thread)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bellows/uart.py", line 406, in connect
    protocol, connection_done = await thread.run_coroutine_threadsafe(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bellows/uart.py", line 385, in _connect
    transport, protocol = await zigpy.serial.create_serial_connection(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy/serial.py", line 37, in create_serial_connection
    transport, protocol = await pyserial_asyncio.create_serial_connection(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/serial_asyncio/__init__.py", line 448, in create_serial_connection
    serial_instance = serial.serial_for_url(*args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/serial/__init__.py", line 90, in serial_for_url
    instance.open()
  File "/usr/local/lib/python3.11/site-packages/serial/serialposix.py", line 325, in open
    raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 5] could not open port /dev/ttyUSB_SkyConnect: [Errno 5] I/O error: '/dev/ttyUSB_SkyConnect'
2023-08-26 03:12:19.292 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [547563592848] Error handling message: Unknown error (unknown_error) Mac User from 192.168.2.50 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/decorators.py", line 26, in _handle_async_response
    await func(hass, connection, msg)
  File "/usr/src/homeassistant/homeassistant/components/zha/websocket_api.py", line 1097, in websocket_update_zha_configuration
    zha_gateway: ZHAGateway = hass.data[DATA_ZHA][DATA_ZHA_GATEWAY]
                              ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^

Then after the next reboot of the RasPi, unfortunately SkyConnect fell back to /dev/ttyUSB2 again.

Since I was lucky and USB2 happend to be the right mapped port, HA started again, but obviously that isn’t a good solution. It is rather later (or early) over here in Europe, so I’ll let it run like that for now. Not sure if you have a suggestion how to get the SkyConnect configuration stable migrated to the symlink. I don’t think there is a config file with the USB configuration for SkyConnect so “Migrate Radio” seems to be the only way but it somehow didn’t like the migration? Thanks again, it feels like we are getting closer!

Just to close on this thread, thanks so much to @freshcoast for the help and the solution. The mapping of my three USB ports into the docker container is now stable! It was indeed the issue that mapping of the symlinks created by udev rules in the host OS to /dev/USB0, /dev/USB1 and /dev/USB2 in the docker container via compose.yaml failed, since that conflicts with the USB ports /dev/USB0, /dev/USB1 and /dev/USB2 from the host OS that are also exposed in the docker container under the same names since I run the container in privileged mode. With different names in the compose.yaml mapping, this now works. Thanks again so much!

Unfortunately SkyConnect is still not working reliably. After a reboot (and if the host OS USB2 port happened to be the port assign to the SkyConnect device, otherwise HA won’t start in my case since the SkyConnect device then interferes on the physical USB ports of one of the other two devices and then HA won’t start), it is possible to use “Migrate Radio” to assign the mapped /dev/ttyUSB_SkyConnect port in the docker container (which was mapped via compose.yaml from the /dev/serial/by-id/usb-Nabu_Casa_SkyConnect_v1.0_ae94a629d960ed119dcdcde05720eef3-if00-port0) and ZigBee works fine afterwords (and also /dev/ttyUSB_SkyConnect shows up in the SkyConnect configuration setting in HA).

However, unfortunately that assignment to /dev/USB_SkyConnect does not persist after a reboot. After a reboot, the SkyConnect configuration still seems to assume /dev/ttyUSB2 (which was the USB port with which it was originally set up and configured). But if that port happens to be assigned to one of the other two USB devices, SkyConnect interferes with those devices and HA won’t start and I have to reboot until USB2 in the host OS happens to be mapped to the SkyConnect device. Then HA starts, but instead of /dev/ttyUSB_SkyConnect, /dev/ttyUSB2 shows up in the SkyConnect configuration settings. I can then again migrate the Radio to /dev/ttyUSB_SkyConnect and it again works fine, but again it does not persist after a reboot…

A search with fgrep -r "ttyUSB_SkyConnect" * in the HA config directory does not reveal this string (at least not in clear text) in any of the config files so I don’t know where that configuration information is actually stored?

But since this is a different issue, I’ll create a new issue for that and close this one as solved, thanks again and kudos to @freshcoast for the solution.

I use Home Assistant container myself and don’t run it in privileged mode. Instead, I have UDEV rules and map the symlinks I created with the UDEV rules with the -device flag. If you are specifying the paths with the -device flag privileged mode is not necessary, and using privileged mode you will just inherit all the host machines device mappings into the docker container which I believe is creating your conflicts and shifting device names.

I wrote up reasons why I don’t use privileged mode in the linked post below. Also, see the log file in that post which has warnings due to conflicts created by using both the -privileged and -device flags in the same docker run command. You may want to check your logs for similar warnings.

The old documentation for Home Assistant container did not require privileged mode and they changed it to try and make things simpler for users. My general advice is to stick with install documentation in most cases, but if you are using UDEV rules you have a more complex install that can cause you to need to make modifications to the docker run command from the default documentation.

Instead of dropping your UDEV rules, I would use them and your -device mappings in the docker run command and get rid of privileged mode instead (just delete the entire privileged: true line from your compose), and see if that fixes everything. Then, your skyconnect will map from the host to /dev/ttyUSB2 in your container all the time, which should resolve your current issue you are having with it. In my opinion, privileged mode should be avoided whenever possible.

2 Likes

Thanks @mwav3, I removed privileged mode and used /dev/serial/by-id/* mappings in the -device section instead of UDEV rules and it works great. And it s of course so much better without privileged mode (thanks for sharing the blog, very insightful), so I am glad this works! Thanks again!

1 Like

To be honest I want to give @freshcoast also credits for finding my solution, From @mwav3 I took instant action on that privileged part. As I understand now for docker containers we can skip UDEV and use instead the “by-ID”. I want to thank everybody for their input and share my revised compose.yaml for Zigbee2MQTT:

# https://www.zigbee2mqtt.io/guide/getting-started/#installation
# https://community.home-assistant.io/t/static-usb-ports-via-udev-rules-mapped-into-a-docker-container-via-compose-yaml-do-not-work-after-reboot/607020/6

version: '3.8'
services:
  zigbee2mqtt:
    container_name: zigbee2mqtt
    restart: unless-stopped
    image: koenkk/zigbee2mqtt
    volumes:
      - ./data:/app/data
      #- /run/udev:/run/udev:ro (no need for this anymore)
    ports:
      - 8888:8080  
    environment:
      - TZ=Europe/Amsterdam
    devices:
      # Use for USB device its ID and not the assigned USB port because when there are more than 1 USB devices attched on a
      # systemreboot assigned USB ports can (and will) change. UDEV can be used also but requires extra work outside the container.
      # Command to show USB devices by ID: sudo ls -la /dev/serial/by-id
      # for your docker-container use a logical USB name. Allign this name in configuration.yaml.
      - /dev/serial/by-id/usb-Silicon_Labs_Sonoff_Zigbee_3.0_USB_Dongle_Plus_0001-if00-port0:/dev/ttyUSB_ZIGBEE

Thanks again