Restart HA (not when rebooting) usb serial converter not starting: serial.serialutil.SerialException: [Errno 5] could not open port

Systeminfo:
VMWare ESXi
Operating system HassOS 4.13 production
HA Version: 0.116.3
Supervisor Version 247
QinHeng USB2.0-Serial

Restarting HA gives an error when restarting: serial.serialutil.SerialException: [Errno 5] could not open port
The serial connection is made via a USB to RS232 for the DSMR.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/dsmr/sensor.py", line 158, in connect_and_reconnect
    transport, protocol = await hass.loop.create_task(reader_factory())
  File "/usr/local/lib/python3.8/asyncio/coroutines.py", line 124, in coro
    res = func(*args, **kw)
  File "/usr/local/lib/python3.8/site-packages/serial_asyncio/__init__.py", line 410, in create_serial_connection
    ser = serial.serial_for_url(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/serial/__init__.py", line 88, in serial_for_url
    instance.open()
  File "/usr/local/lib/python3.8/site-packages/serial/serialposix.py", line 268, in open
    raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 5] could not open port /dev/serial/by-id/usb-1a86_USB2.0-Ser_-if00-port0: [Errno 5] I/O error: '/dev/serial/by-id/usb-1a86_USB2.0-Ser_-if00-port0'
2020-10-15 19:28:16 ERROR (MainThread) [homeassistant.components.dsmr.sensor] Error connecting to DSMR
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/serial/serialposix.py", line 265, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
OSError: [Errno 5] I/O error: '/dev/serial/by-id/usb-1a86_USB2.0-Ser_-if00-port0'

It happens only when I restart HA. Rebooting solves the problem.
Restarting cost way more time than rebooting the whole system. It is waiting for the serial adapter to finish which never does.

This HA is running in a VM environment and the console screen shows:

image

It happened after a few updates ago and was I not aware what was going on. Newer updates did not fix it.

Can this issue be a driver related thing?

Is it broken since 0.115?

I think you should report an issue for this on Github.

I think it has to do with the OS or Docker and not HA itself.

To me it looks like a driver issue for the USB to serial converter.
Another instance which uses serial is a true RS232 port via VMware where a RS485 converter is plugged in. Maybe I will move over to another free RS232 port on the VMware environment which has the option to leave the USB converter.

Updated to HassOS 4.15 and still the same errorsw.

The ch341 UART chip looks to be the suspect. I have read multiple issues with this chipset is known.

On HA versions earlier it worked well. What can be the issue?