Symlink to /dev/serial/by-id/ - Not working

I know that several of you have this configured this way and wonder if you could provide some guidance.
My Setup: Running Ubuntu Server 18.04 with Docker and Docker-Compose to spin up my containers. Currently trying to setup my RFXtrx but will want to do the same with my other USB toys that I want to plug in.

Tried the UDEV rules, but that didn’t work either, it never created the symlinc, so I decided to use the one created by the operating system in /dev/serial/by-id

In configuration.yaml, this is my entry for rfxtrx:
device: /dev/serial/by-id/usb-RFXCOM_RFXtrx433_A12D8M05-if00-port0
(for some reason, formatting is not working, but I have 2 spaces before “device”)

I can see this symlinc created in the same path:
lrwxrwxrwx 1 root root 13 Dec 5 13:16 usb-RFXCOM_RFXtrx433_A12D8M05-if00-port0 -> …/…/ttyUSB0

when I launch Home-Assistant, it gives me a configuration error and when I look at the logs the error state “No folder or file present”
if I switch my configurations to
device: /dev/ttyUSB0
Then it works fine. I could keep it this way, but would rather make my system as bullet-proof as possible so if in the future, I add or remove devices, my components should still work.
Would any of you have any clues? I searched the web for an answer, but can’t find anything similar. Do I have permission issues on the Symlink?
I also tried adding the port in my docker-compose.yaml file but that didn’t help, also thought that since I run Home-Assistant in network_mode: host and privileged: true, that it shouldn’t matter as it should pick these things up from the system itself.

So a little puzzled here.