UPB lighting

gwww · July 12, 2020, 1:40am

You’ve got me curious. The command that you sent is nothing I have seen before!

When I activate a link here is the message that I see:

PU 8904 C20909 20 FFFF81

C20909 is my network, source link, and dest link. 20 is activate link command. The rest does not matter for this discussion.

Yours…

PU 8900 9E070C FF FFFFC9

9E is your network, 07 your source link, and 0C the destination. The FF is supposed to be the a command, presumable activate in your case, which is 20. FF is not a recognized UPB command, hence the message is ignored by the library.

Any more that you can share?

samgreco · July 12, 2020, 3:47am

Well, that’s curious. I’ll need to get UPStart connected and check comms and programming.

This is a different link on the same switch:
DEBUG (MainThread) [upb_lib.proto] data_received: 'PU89009E0A0CFFFFFFC6

samgreco · July 12, 2020, 3:38pm

So this led me down the path of assuming the switch was not programmed correctly. I wiped it (5-10-2) and rebuilt it from scratch. And Viola! all works exactly as expected. All 4 buttons are received and my HA automation and scene trigger as expected.

Thanks again Glenn.

123 · July 16, 2020, 3:44pm

The UPB integration’s documentation states that its “IoT Class” is “Local Polling”.

Does the integration do any periodic polling of all devices?

My impression is that it doesn’t but I’d like to confirm that.

bbrendon · July 16, 2020, 5:12pm

Yea. That’s wrong. It only polls when HASS starts or when there is a service call to do it manually.

EDIT: I submitted a PR.

gwww · July 16, 2020, 5:21pm

Mostly. Slightly different wording. It polls whenever HASS connects to the PIM. This is different than when starting HASS since there are cases where the PIM will disconnect. In those cases periodic attempts to reconnect are made. Two specific cases are when the PIM is unplugged from the serial port and the second is when using a network PIM and a connection cannot be made to the PIM.

123 · July 16, 2020, 6:15pm

I noticed that even the ZigBee (ZHA) integration is listed as Local Polling. I guess if there’s even a limited possibility of the integration performing polling, the convention is to classify it as “Local Polling”.

It’s slightly misleading because, for example, the Philips Hue integration is “Local Polling” and it really polls periodically to ensure device states are synced with Home Assistant. Their API provides no means for a device to “publish” its state-changes (like a UPB device can do). So the three integrations, Hue, ZHA, and UPB, are all Local Polling but only Hue fits that description perfectly.

Anyway, the reason I asked is because I’m trying to get the bottom of an issue with disabling/re-enabling an entity. Go to Configuration > Entities, click the filter icon (upper right hand corner) and select “Show disabled entities”. Now click a UPB-based light, turn off Enable_entity, then click Update.

The light’s name will now be Unavailable and it will disappear from the States view and the auto-generated Lovelace UI. So far, everything works as one would expect of a disabled entity.

While still in the Entities page, click the disabled entity and re-enable it (don’t overlook to click Update). The entity’s name remains Unavailable and it’s still missing from the States page. That’s unexpected. However, wait about 20 seconds and its name will be restored and it will re-appear everywhere else as well. Note: if you refresh the web-page during this time period, it will not fix the problem.

This behavior can be duplicated in other integrations that use config-flow. However, the entity’s “restoration time” varies from mere seconds to about a minute (based on the handful of integrations I have tried). My theory was that this time period was due to some sort of polling activity performed by the integration itself.

In contrast, the MQTT integration’s IoT Class is Local Push. If you disable an auto-disovered MQTT entity, re-enabling it never restores it to full functionality. You have to restart Home Assistant. The same is true for any Helper (input_number, input_boolean, etc) created via the UI.

This is what led me to theorize that only “Local Polling” integrations eventually restore the re-enable entity due to periodic refreshing. Well, it was my theory until it was discredited by the fact the UPB integration performs no periodic polling. If the 20-second delay to restore the entity is not the integration’s doing then I don’t know what’s responsible for it.

tl;dr
Re-enabling a disabled entity produces inconsistent behavior that may even require restarting Home Assistant in order to truly re-enable the entity.

bbrendon · July 16, 2020, 6:53pm

Interesting. I would think the integration would be listed as what it does most. I guess we’ll see what happens to the PR I submitted

bbrendon · July 17, 2020, 4:20pm

PR accepted!! We are now better than ZHA :-p
Heh.

jameshilliard · July 19, 2020, 10:16pm

My patches adding support for the Simply Automated USB PIM to the Linux kernel have landed in time for the v5.9 release(I think they will get backported to stable kernels as well fairly soon as well).

https://github.com/torvalds/linux/commit/5c45d04c5081c1830d674f4d22d4400ea2083afe

Are you able to get notifications of switches changing state in message mode without polling for the state change?

123 · July 20, 2020, 12:43pm

This is a surprise but I confess to being ignorant of the full meaning and potential of this development. I would appreciate it if you could explain the implications of this announcement. (i.e. does it imply native support for UPB?)

jameshilliard · July 20, 2020, 8:37pm

It effectively means that on Linux systems the Simply Automated USB PIM(the PCS USB PIM variant should be recognized already as it uses the exact same VID/PID as the standard Cypress HID->COM RS232 adapter) should be recognized in the same way as a USB to serial adapter connected to a serial PIM.

The Simply Automated USB PIM’s appear to use an internal Cypress HID->COM RS232 adapter converter chip, and is by default recognized as a HID device due to the hardware advertising that capability over USB. The upstart application is capable of speaking HID protocol directly to the device by essentially encapsulating the serial commands inside of HID protocol commands as opposed to the usual COM port commands typically used by the serial PIM’s(usually exposed via a normal USB to serial adapter). Most UPB software other than upstart is not capable of communicating with this HID wrapped serial protocol used by the the Simply Automated USB PIM as there is significant complexity involved in translating normal UPB serial commands into the HID wrapped serial commands required when using the default HID driver.

Now on Linux someone had written a kernel driver that can be used to essentially translate this Cypress HID wrapped serial protocol used by the USB PIM’s to a standard /dev/ttyUSB0 style serial interface so that software like home-assistant doesn’t need to understand the HID wrapped protocol but can instead use the same interface used with normal serial based PIM’s. This works essentially by having the kernel driver transparently translate normal serial interface commands to HID wrapped serial commands that the device understands.

Since by default Linux would recognize the Simply Automated USB PIM as a HID device(due to USB protocol feature advertisements) and not a USB to serial adapter it wouldn’t be usable via the standard /dev/ttyUSB0 interface that we need. What I’ve done is submit 2 patches to the kernel to fix this, one patch adds the unique USB VID(Vendor ID) and PID(Product ID) pair for the Simply Automated USB PIM to the Linux kernel’s HID driver blacklist(which basically tells the normal HID kernel driver to ignore the HID protocol feature flag sent by the UPB PIM) so that the HID driver will not attempt to load/bind to the USB PIM device.

The other patch adds the same VID/PID pair to the cypress_m8 kernel serial driver device table so that the cypress_m8 driver will be auto-loaded when the device is plugged in, this driver will then create the /dev/ttyUSB0 serial style interface that we want for use by home-assistant whenever the USB PIM is plugged in and transparently translate commands sent to the standard serial protocol /dev/ttyUSB0 file descriptor to HID wrapped serial protocol expected by the device(the cypress_m8 driver itself implements the HID protocol command subset required to handle the wrapped serial protocol translation independently from the normal HID kernel driver that we blacklisted this device in).

My assumption is that the reason for USB PIM’s to use the HID wrapped serial protocol is so that a device driver(say for example a FTDI USB to RS232 driver) doesn’t need to be installed on windows for Upstart to be able to recognize the USB PIM. Since windows like Linux will recognize HID devices automatically via the USB feature flag this somewhat simplifies driver installation, it however has the downside that the application(like upstart) itself now has to understand how to speak HID wrapped serial unless you do something like in Linux where we override the loading of the default HID kernel driver and load a different serial kernel driver that will transparently do the translation.

123 · July 20, 2020, 8:52pm

Thank you for the detailed explanation!

It also answers the question I posted back in March which asked how to get a USB PIM to be recognized by the UPB integration. It was appearing as an HID device and I (wrongly) assumed that /dev/usb/hiddev0 would be handled like /dev/ttyUSB0 … it definitely wasn’t.

jameshilliard · July 20, 2020, 9:02pm

Yeah, using /dev/usb/hiddev0 would def not work with home assistant because that is the HID protocol interface exposed by the normal HID kernel driver(the one I’ve blacklisted in my patches), any application using that interface would need to itself handle the translation similar to how upstart does its internal HID wrapped serial translation.

My patches also just got backported to the mainline stable kernel trees today as well, once those end up in a release and get picked up by the downstream distributions the Simply Automated USB PIM should show up as /dev/ttyUSB0 instead of /dev/usb/hiddev0 automatically when you plug it in.

bbrendon · July 22, 2020, 6:07pm

I wonder how UPB sales will change based on all the work you guys are doing.

PCS and other manufacturers should be sending you gifts!

123 · July 22, 2020, 6:51pm

If PCS is aware of Home Assistant’s UPB integration, they aren’t reporting it in any of their emailed announcements. I receive them fairly regularly and don’t recall it being mentioned. Either they don’t know it exists or they have chosen not to report it.

gwww · July 22, 2020, 7:04pm

I reached out to PCS, I guessing back in February. I did not receive a response. My message was that maybe you want to follow this work, you might find it useful for people connecting with your products.

samgreco · July 22, 2020, 9:55pm

Considering we have @webmtn helping so much on this very thread, we should probably all support him when it comes time to buy UPB stuff.

bbrendon · July 22, 2020, 10:15pm

webmtn told me verbally he would give HASS users discounts and he was planning an official announcement. Maybe he just never got around to it? If your order, I would ask!

webmtn · July 23, 2020, 4:47am

samgreco and bbrendon - thanks for the kind words. And, yes, I’m planning on providing good discounts to HASS users. However, here’s the issue at the current time. Both SA and I are out of almost everything, due to delays getting product out of China, presumably due to the virus. So, even if you tried to order anything from me, I probably couldn’t deliver it. Crossing my fingers that this is going to get resolved soon, but at the moment, we’re not getting any updates from China, so we just don’t know.

Something I do have - lots of faceplates in colors other than white - all brand new, but some have been on the shelf for multiple years. I’d like to do a clearance sale on faceplates / rockers / multibuttons in non-white colors. I’ll work on that, but if anyone needs some of these now, get in touch with me directly.

Something else I’ll try to get done shortly. I have quite a few older UPB devices that I ended up with, often because they were replaced with newer versions, I’d like to have a clearance sale and offer them to HASS users at fire sale prices. Most are NOS - New Old Stock - brand new units but they’ve been on the shelf for many years. Some are units that have been returned to me for various reasons. But, they are all still good units and would be a way to get some different UPB devices at next to nothing. I’ll let you know once I get the list put together.

And, again, I want to thank all of you, especially Glenn, for being persistent and creating a UPB solution for HASS. And, I’m looking forward to seeing the results of Jim Hilliard’s work. .I have been asked about this for several years, so all of you are to be commended.