Is anyone running the UPB code in Core? I’m tempted but nervous about the changes I’ll have to make to my automations.
@gwww
I have tried the release version of the UPB integration and have a failure to report. Shall I post it in this thread or start a new one?
Start with posting here.
I’m testing the UPB integration on a fresh installation of Home Assistant 0.110.0 on an RPI3 (i.e. clean test-bed; default configuration; no other integrations). I’m using a ‘known good’ serial-to-USB cable and a Simply Automated Serial PIM (in Message mode). The cable and PIM were used for testing the beta version of the UPB integration.
Upon plugging the cable into the RPI3, it is detected by HassOS and reported in Supervisor > System > Hardware as two devices (by name and by port number):
serial:
/dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0
/dev/ttyUSB0
I copied the UPB configuration file to the RPI3 in /config/upb.upe.
Here’s a screenshot of the initial config flow screen:
After submitting the data, several seconds pass and then it returns with the following error:
Failed to connect to UPB PIM, please try again.
Screenshot:
The connection failure is reported in the log:
2020-05-22 12:42:27 WARNING (MainThread) [upb_lib.upb] Could not connect to UPB PIM ([Errno 2] could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: '/dev/ttyUSB0'). Retrying in 1 seconds
2020-05-22 12:42:28 WARNING (MainThread) [upb_lib.upb] Could not connect to UPB PIM ([Errno 2] could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: '/dev/ttyUSB0'). Retrying in 2 seconds
2020-05-22 12:42:30 WARNING (MainThread) [upb_lib.upb] Could not connect to UPB PIM ([Errno 2] could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: '/dev/ttyUSB0'). Retrying in 4 seconds
2020-05-22 12:42:34 WARNING (MainThread) [upb_lib.upb] Could not connect to UPB PIM ([Errno 2] could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: '/dev/ttyUSB0'). Retrying in 8 seconds
2020-05-22 12:42:42 ERROR (MainThread) [homeassistant.components.upb.config_flow] Timed out after 15 seconds trying to connect with UPB PIM at serial:///dev/ttyUSB0
I repeated the test using the serial port’s alternate name:
/dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0
but the result was the same.
2020-05-22 13:08:40 WARNING (MainThread) [upb_lib.upb] Could not connect to UPB PIM ([Errno 2] could not open port /dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0: [Errno 2] No such file or directory: '/dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0'). Retrying in 1 seconds
2020-05-22 13:08:41 WARNING (MainThread) [upb_lib.upb] Could not connect to UPB PIM ([Errno 2] could not open port /dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0: [Errno 2] No such file or directory: '/dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0'). Retrying in 2 seconds
2020-05-22 13:08:43 WARNING (MainThread) [upb_lib.upb] Could not connect to UPB PIM ([Errno 2] could not open port /dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0: [Errno 2] No such file or directory: '/dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0'). Retrying in 4 seconds
2020-05-22 13:08:47 WARNING (MainThread) [upb_lib.upb] Could not connect to UPB PIM ([Errno 2] could not open port /dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0: [Errno 2] No such file or directory: '/dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0'). Retrying in 8 seconds
2020-05-22 13:08:55 ERROR (MainThread) [homeassistant.components.upb.config_flow] Timed out after 15 seconds trying to connect with UPB PIM at serial:///dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0
I disconnected the cable and PIM from the RPI3, connected it to the laptop I had used (0.108.5, docker) with the beta UPB integration (configured in YAML) and it found the PIM and worked correctly. This re-confirms the cable and PIM are functional.
From my understanding by default the RPI images for home assistant run home assistant in a container which doesn’t have access to USB devices unless configured a certain way.
I’m unfamiliar with that constraint. It’s my understanding that whatever is reported in the Supervisor > System > Hardware list is readily accessible.
Here’s a post that says the same:
Otherwise this alleged USB-device constraint would affect all RPI3 users who plug in coordinators for Zigbee and zwave.
Hmmm. I don’t know how to fix this. I’ve never used a USB connected device in hassio. When I did ElkM1 it was network connected. The UPB is USB connected to a windows box that runs a serial to TCP proxy, so UPB is network connected.
What I have tested and what I know works is a USB connected UPB PIM using a serial to USB adapter. That was tested on a MacBook. Similar to your hardware. The difference is Docker.
I’ll do some searches to see what I can uncover. I’d recommend you do the same. Let’s see what we can come up with.
What is clear from the error message is that the device name is not understood. Perhaps poke on the hassio forums or Discord chat.
A few more details regarding the system used for the beta version (i.e. the one that works). It’s running Home Assistant Core 0.108.5 in Docker. I’m using docker-compose and here’s the relevant portion that maps the serial-to-USB cable from its “real-world” device name to the port name visible within the Docker container.
devices:
- "/dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0:/dev/ttyUSB0"
In configuration.yaml, I specify the port name visible within the Docker container, namely /dev/ttyUSB0.
upb:
url: serial:///dev/ttyUSB0:4800
file_path: ./upb.upe
All of this plumbing works correctly on this system running the beta version.
Home Assistant 0.110.0 on an RPI3 (a.k.a. the old “hassio”) includes Home Assistant Core as a docker container. However, there’s no user intervention required to map “real world” ports to how they are seen within the container. Theoretically, this is done for the user and revealed in the Hardware list.
I switched to the UPB integration in 110 Core. Thanks to the core devs I had to spend a good 10 minutes of intense clicking to do the migration. I had to tweak the scene automatons which wasn’t terrible. Can’t say I like their new GUI much, but that’s a whole different topic.
So far it seems solid!
Thank you @gwww for all the hard work!
Have you learned anything new today? I think your best bet is to ask on the hass Discord. Clearly the TTY device is not understood for whatever reason. The Discord will have people hanging out that should be able to help you debug further. Keep us posted. I want to understand too!
As much as I would like to help you fix the integration, joining Discord is not in the cards for me.
I started a separate thread asking how USB ports are mapped in Home Assistant (RPI3 disk-image). So far, it remains unclear why /dev/ttyUSB0 is unreachable by the integration.
Worst case, you may have to indicate that the integration cannot be used with Home Assistant installed on an RPI3 (and possibly others as well; anything where Core is running on HassOS).
EDIT
Just saw this:
Sounds similar; unable to connect to a USB-connected device. The main difference is that his ssh console is showing the USB device in /dev whereas it is not present on my system.
When working with containers you generally need to be careful to check if you are working in the base system or in the container environment as which /dev devices are exposed to the container may be limited compared to the base system. There’s also many types of containers and isolation methods, systemd for example has its own native container management system called systemd-nspawn which uses similar techniques to docker such as cgroups/namespaces to isolate a container.
When you ssh in are you in the same namespace/container as home assistant or are you in the base system? Also what does dmesg show?
I am using the Portainer Add-On to give me access to the homeassistant docker container (i.e. the one containing Home Assistant Core). Portainer allows me to open a console in the container.
lsusb shows the Raspberry Pi 3B’s hardware:
bash-5.0# lsusb
Bus 001 Device 006: ID 0557:2008 ATEN International Co., Ltd UC-232A Serial Port [pl2303]
Bus 001 Device 003: ID 0424:ec00 Microchip Technology, Inc. (formerly SMSC) SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 002: ID 0424:9514 Microchip Technology, Inc. (formerly SMSC) SMC9514 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
bash-5.0#
dmesg reports the number of times I connected/disconnected the converter cable. In all instances, it detected the converter and assigned it to ttyUSB0
bash-5.0# dmesg | grep -i usb
[227725.011438] usb 1-1.4: new full-speed USB device number 4 using dwc_otg
[227725.143792] usb 1-1.4: New USB device found, idVendor=0557, idProduct=2008, bcdDevice= 3.00
[227725.149981] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[227725.156251] usb 1-1.4: Product: USB-Serial Controller D
[227725.159440] usb 1-1.4: Manufacturer: Prolific Technology Inc.
[227725.226218] usbcore: registered new interface driver pl2303
[227725.228585] usbserial: USB Serial support registered for pl2303
[227725.240810] usb 1-1.4: pl2303 converter now attached to ttyUSB0
[228939.439616] usb 1-1.4: USB disconnect, device number 4
[228939.443497] pl2303 ttyUSB0: pl2303 converter now disconnected from ttyUSB0
[229085.649906] usb 1-1.4: new full-speed USB device number 5 using dwc_otg
[229085.782166] usb 1-1.4: New USB device found, idVendor=0557, idProduct=2008, bcdDevice= 3.00
[229085.788235] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[229085.794384] usb 1-1.4: Product: USB-Serial Controller D
[229085.797471] usb 1-1.4: Manufacturer: Prolific Technology Inc.
[229085.811877] usb 1-1.4: pl2303 converter now attached to ttyUSB0
[230076.079572] usb 1-1.4: USB disconnect, device number 5
[230076.083527] pl2303 ttyUSB0: pl2303 converter now disconnected from ttyUSB0
[230679.769963] usb 1-1.4: new full-speed USB device number 6 using dwc_otg
[230679.902304] usb 1-1.4: New USB device found, idVendor=0557, idProduct=2008, bcdDevice= 3.00
[230679.908398] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[230679.914576] usb 1-1.4: Product: USB-Serial Controller D
[230679.917703] usb 1-1.4: Manufacturer: Prolific Technology Inc.
[230679.931096] usb 1-1.4: pl2303 converter now attached to ttyUSB0
bash-5.0#
Nevertheless, for reasons I don’t understand, ttyUSB0 is not present in the /dev directory:
bash-5.0# ls /dev/ttyUSB*
ls: /dev/ttyUSB*: No such file or directory
bash-5.0#
You could try mknod /dev/ttyUSB0 4 64 to create the USB device. Need to do that inside the container. The tty name is not important /dev/ttyFOO would work equally well. The 4 says create a tty. The 64 is more magic and that’s what all the articles I just read use. If that doesn’t work and you need to dig deeper the 4 64 are known as major and minor numbers.
I’ll try it as an experiment. However, it’s my understanding that this kind of patching inside of a container cannot survive a reboot.
Update:
It does appear to survive a reboot. I guess it will disappear only if the container is recreated from its image (like during an upgrade).
Update:
That worked but not without some resistance. The command as shown results in BusyBox responding with a help menu suggesting to use b, c, or p for the second option. I tried it with c and it produced the help menu again. At that point I looked in /dev and it now listed /dev/ttyUSB0 so I can’t be sure which of the two commands actually created it (despite both variations resulting in a help menu).
The UPB integration was able to find the port, open it, and poll all the devices listed in upb.upe. I successfully turned one light on/off to confirm the integration is working.
So it all boils down to my first observation: the assigned port (/dev/ttyUSB0) is not visible within the homeassistant container.
I assume this is something that should be managed by Supervisor (or some other service).
What’s next? Shall I post this as an Issue in the Supervisor repo?
I’d try to get the attention of @frenck 
At least that might be enough to figure out another what’s next.
Thank you for your reply.
I found your predicament interesting because, in your case, the ttyUSB* device is present in /dev. In my case,Supervisor reports the converter cable is on /dev/ttyUSB0 yet no such tty device exists in the homeassistant container.
After performing the test on Home Assistant 0.110.2 on an RPI3 (using the official disk-image) I repeated the test using a machine running Home Assistant Supervised 0.110.2 on Ubuntu. The result is exactly the same:
- Supervisor reports the converter cable in the Hardware view
- There is no
/dev/ttyUSB0device in thehomeassistantcontainer. - The UPB integration fails to find and open
/dev/ttyUSB0.
To exclude the possibility that this behavior is due to the model of converter cable being used, I tried one from another manufacturer. The results remain the same:
Supervisor’s Hardware view reports it as follows:
/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A906U593-if00-port0
/dev/ttyUSB0
Nevertheless, there is no /dev/ttyUSB0 in the homeassistant container and the UPB integration fails to connect.
UPDATE
My third and final test using Home Assistant Core 0.110.2 installed as a docker container on Ubuntu.
The docker-compose.yaml file contains the required device mapping:
devices:
- "/dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0:/dev/ttyUSB0"
The homeassistant container contains /dev/ttyUSB0 and the UPB integration finds and opens it successfully.
Conclusion:
For whatever reason, these two installation methods fail to expose the USB port as a tty device:
- Home Assistant 0.110.x (Raspberry PI3 disk-image)
- Home Assistant Supervised 0.110.x on generic linux
This prevents the UPB integration from accessing the PIM.
That means only these two installation methods are currently compatible with the UPB integration:
- Home Assistant Core 0.110.x in a python virtual environment
- Home Assistant Core 0.110.x in a docker container