HAOS 12.4 borks USB devices on Yellow / RPi - USB kernel bug?

Installation Home Assistant OS
, ,
1

Hi,

After what seemed like a routine HAOS update to v12.4 on my Yellow, /dev/serial/* devices like an Aeotec Series 5 Z-Wave coordinator and generic Bluetooth stick disappeared. Both are USB devices and have been connected for months.

Checking Z-WaveJS (v12.11.0), the USB serial device used by my Aeotec S5 stick had disappeared - namely /dev/serial/by-id/usb-0658_0200-if00. Only /dev/ttyAMA* were present.

Reverting the HAOS upgrade, and checking the Z-WaveJS Configurations Device (back to the full device tree /dev/serial/by-id/usb-0658_0200-if00) got my Yellow working again.

ha os update --version 12.3

It looked like HAOS 12.4 had failed to load drivers for USB devices as lsusb only showed one device, whereas under HAOS12.3…

[core-ssh ~]$ lsusb
Bus 001 Device 001: ID 1d6b:0002
Bus 001 Device 003: ID 0bda:8771
Bus 001 Device 002: ID 1a40:0101
Bus 001 Device 008: ID 0658:0200

Reading the release notes more carefully, shows a declared issue which could be shouted out more loudly:

Some Raspberry Pi users have reported system lockups with some Z-Wave and Zigbee USB adapters.

Yes - that “some” seems to be Yellow users using any USB devices!

Take care updating,

James

2

FYI - GitHub discussion suggests there are intermittent (upstream kernel?) issues with interrupts on the USB interface for RPi both on HAOS 12.3 and 12.4. Worst case, 12.3 may not boot fully, although my Yellow worked apart from lacking USB devices.

If you need a workaround on a HAOS device, get a HASS shell (web or ssh)…:

# Revert pre-12.3 and pre-12.4
ha os update --version 12.2

…however if you are able, I’m sure the developers would appreciate some more debug logs and helpful reports in GitHub to support their efforts.

# check HASS USB devices passed through from the HAOS host (expect >1)
lsusb

dmesg >dmesg.txt

# check the HAOS host last boot
ha host logs --boot -1 --lines 10000 >lastboot.txt
grep nobody lastboot.txt

Tracing intermittent issues is never easy, but hardware/ kernel timing issues can be particularly challenging.

1 Like
3

I saw the issue too and have held off upgrading because of it. I am currently on 12.3 with no issues so maybe this won’t impact me? Regardless, I am going to wait. It does appear to a larger kernel issue unrelated to HA.

The other thing they mention is adding adding dtoverlay=dwc2 to config.txt.

4

Unfortunately, we seem to encounter several USB related issues but (as it seems) not with the exact same root cause/underlying issue. The one mentioned in the release notes is Raspberry Pi 3 related (at least, that is what the original poster of Raspberry Pi bootloop after 12.3 update (usb related) · Issue #3362 · home-assistant/operating-system · GitHub uses, and what is reproducible on our end).

The Yellow issue sounds more like the issue mentioned here: USB devices not available after update to 12.3 on Yellow. Also preventing boot. · Issue #3347 · home-assistant/operating-system · GitHub.

Reverting to 12.2 will downgrade the Linux kernel to 6.1 as well, and the other poster in 3347 reported it too that 12.3 and newer is affected. So this really seems Linux 6.6 related. It also seems intermittend, so that is probably why you’ve noticed only with the 12.4 upgrade :thinking:

Currently we are not able to reproduce this problem, so indeed more information in for this issue is indeed very welcome!

1 Like