I use Home Assistant container myself and don’t run it in privileged mode. Instead, I have UDEV rules and map the symlinks I created with the UDEV rules with the -device flag. If you are specifying the paths with the -device flag privileged mode is not necessary, and using privileged mode you will just inherit all the host machines device mappings into the docker container which I believe is creating your conflicts and shifting device names.
I wrote up reasons why I don’t use privileged mode in the linked post below. Also, see the log file in that post which has warnings due to conflicts created by using both the -privileged and -device flags in the same docker run command. You may want to check your logs for similar warnings.
The old documentation for Home Assistant container did not require privileged mode and they changed it to try and make things simpler for users. My general advice is to stick with install documentation in most cases, but if you are using UDEV rules you have a more complex install that can cause you to need to make modifications to the docker run command from the default documentation.
Instead of dropping your UDEV rules, I would use them and your -device mappings in the docker run command and get rid of privileged mode instead (just delete the entire privileged: true line from your compose), and see if that fixes everything. Then, your skyconnect will map from the host to /dev/ttyUSB2 in your container all the time, which should resolve your current issue you are having with it. In my opinion, privileged mode should be avoided whenever possible.