UPB lighting

@gwww how do I detect a link that was activated on a physical switch in HASS?

Also, it looks like a message is considered a repeat even after 40 seconds. (I obfuscated some of my UPB info below)

2020-01-30 15:58:00 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:00 DEBUG (MainThread) [upb_lib.links] Activate BedMaster-Button4 x_x
2020-01-30 15:58:00 DEBUG (MainThread) [upb_lib.links]   Updating 'LivingRm Lamp1' to dim level 0
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.message] Repeated message!!!



2020-01-30 15:58:44 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:44 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.message] Repeated message!!!

Question 1 was how to detect when a link was activated on a physical switch. The PIM will receive a message and use the contents to update all of the lights affected by that link. Nothing in the actual scene object is changed. What would you like to happen and why?

With respect to question 2 and the repeated messages. Here is what the code does. When the first message is received it is striped of sequence number and then saved. When the next message it is striped of sequence number. If it is the same as the saved message then you get the ā€œRepeated message!!!ā€ log. If it is different, it is saved and then processed.

The theory is that repeated messages, no matter the time frame, have no value. Nothing changes in hass.

Is there a practical application for not throwing away duplicate messages. Note, the code SHOULD throw away dups when the sequence number is increasing from the previous message. They are truly duplicate. I think what we are discussing is when the seqence number in a message is less than the sequence number in the previous message.

Hope all the rambling makes sense!

With respect to the log. Its not very useful without the actual messages. The only thing in the messages that is possibly sensitive is the network ID. But even that is not that sensitive. So, I can try guessing at what you are seeing and Iā€™ll give it yeomanā€™s effort but I may not be answer questions that way. You can always email me a log or portion of if you trust me enough. I basically delete such stuff as soon as the problem is fixed. Or you can edit the network ID out of each message. Orā€¦

Yes, LIFX has services on lights. Last night I uploaded a new integration where I added services on Scenes (thatā€™s how UPB links are represented now). Like to know what you think.

The hass devs recommended I give Scenes a shot. I did.

Hey @webmtn! Thank you for your support! I feel like I getting to be an expert. I feel like I figuring out most of the protocolā€™s ins and outs. Stuff that I cannot test because I donā€™t have the equipment is any kind of multi-channel device. I added support for multi-channel this afternoon (sorry folks ā€“ that will cause the unique_ids to change for lights). The other thing that I cannot test is the TCP PIM/CIM. The code is there and ā€œshouldā€ work because it was taken from the ElkM1 library that I wrote. Of should and does never quite line up :wink: .

One protocol question that I guessed the answer to is around goto and channels. Goto for channels take 3 arguments. Level, rate, and channel. The spec doesnā€™t talk about what to use for default rate when sending a channel goto.

The other bit I have no idea about is the device status report for multi-channel unit. Got it figured out for single channel. Iā€™m just guessing but the second byte of the status report would be the status for channel 2.

Another thing that could be useful is if you have connections with PCS or Simply Automated. Are they interested in this work? Maybe they can help test?

I believe the code I have for UPB is structurally pretty good shape. Now the work is supporting additional command such as Blink, etc. And there is questions like @bbrendonā€™s and link status. Most of these are better understanding peopleā€™s requirements then the implementation is relatively easy.

@gwww thanks. Iā€™ll provide the details next time. I just wasnā€™t sure. Iā€™d rather not give out my network id but I know itā€™s easily grabbed off the powerline anyway.

Regarding the linksā€¦ Right now I have buttons on my UPB switches I use to trigger things in HASS. There is a UPB button near the front door that is technically a link, but when HASS sees it is pressed, it runs an exit script to lock doors, make final warnings, etc. Basically I use UPB buttons to do things (in HASS) other than UPB lights.

Ditto. One of the four buttons on a US2-40 is programmed to activate a link when pressed (deactivate when double-pressed).

This link has no UPB devices linked to it. Itā€™s sole purpose is to serve as a command signal that is heard by Premise. Upon receiving the link, it runs an automation to control an X10 dimmer module (for a table lamp).

Similarly, another button is programmed to turn all interior UPB lights on (of off). Naturally, this link is tied to all interior UPB lights. However, I still have one interior light driven by an X10 dimmer switch. Therefore, when Premise receives the link it is programmed to control the X10 switch.

tl;dr
Iā€™m using UPB links to serve as signals to trigger automations.


Still in the process of transferring my Digi Edgeport/4 (USB to 4-port RS232 converter) from my RPi test-bed (for Node-Red upb2mqtt) to my laptop running a test instance of Home Assistant (Core). Your software is already installed (just refreshed it with the latest version) so itā€™s just a matter of getting the Edgeport running with Ubuntu (hopefully it will far less of an adventure than Raspbian which doesnā€™t support it by default).

Thatā€™s a good way to put it :slight_smile:

gwww : Regarding the repeat messages. I was re-reading what you wrote. So say you want to press the button to trigger the cat feeder that spits out food quantity of X, so you press the button. You press it again to feed them 2x. But without the repeat message check timing out, you get 1x when you expect 2x. On my current setup I have the repeat message check expire after 3 seconds.

Now that I think about it, the Deconz integration exposes the button actions of a remote control as events. From Deconz Remote Control Devices:

Remote controls (ZHASwitch category) will not be exposed as regular entities, but as events named deconz_event with a payload of id and event and in case of the Aqara Magic Cube also gesture . Id will be the device name from deCONZ and Event will be the momentary state of the switch.

When you press a button on the US2-40, the resulting UPB message contains the link ID and which UPB device produced it. Only thing you donā€™t know is what button action was used to create it. I believe that information would be in the UPE file ā€¦ although I donā€™t see a need for it. The key thing is that a UPB link could be expressed as an event (upb_event).

Iā€™ve added an attribute to the Scene called ā€œlast_changeā€ which is a timestamp of when the link was last changed. Watch for that value to change. Iā€™m testing it then will upload it later tonight.

Iā€™ll be offline most of the weekend. Keep up the testing and questions!!

Iā€™ve also changed the ā€œRepeated message!!!ā€ logic. It will only call repeated when the sequence number is increasing from the previous message that is otherwise identical.

Ubuntu easily discovered the Edgeport but then things turned south. I encountered the following error message in Home Assistant:

Could not connect to UPB PIM ([Errno 2] could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: ā€˜/dev/ttyUSB0ā€™). Retrying in 1 seconds

Configuration:

upb:
 url: serial:///dev/ttyUSB0:4800
 file_path: ./upb.upe

The path /dev/ttyUSB0 exists.

crw-rw---- 1 root dialout 188, 0 Jan 30 21:52 /dev/ttyUSB0
crw-rw---- 1 root dialout 188, 1 Jan 30 21:52 /dev/ttyUSB1
crw-rw---- 1 root dialout 188, 2 Jan 30 21:52 /dev/ttyUSB2
crw-rw---- 1 root dialout 188, 3 Jan 30 21:52 /dev/ttyUSB3

When logged in with my user account, this doesnā€™t work:
cat /dev/ttyUSB0

However, this does:
sudo /dev/ttyUSB0
If I produce some UPB traffic, gibberish appears on the command line thereby confirming the UPB PIM is reporting on /dev/ttyUSB0.

Then I realized Iā€™m running Home Assistant (Core) as a Docker container. Iā€™ve never had a need to allow this arrangement to access a serial port. After some googling, I added the following to the homeassistant section of the docker-compose file:

    devices:
      - /dev/ttyUSB0:/dev/ttyUSB0

I restarted the container and ā€¦ no joy. The error message remains the same. Looks like I need to do some more googling.


Using Portainer, I opened the Home Assistant containerā€™s console. I listed the files in /dev and it doesnā€™t contain /dev/ttyUSB0. :man_shrugging:


I tried this very specific mapping but Home Assistant still complains that /dev/ttyUSB0 doesnā€™t exist.

    devices:
      - /dev/serial/by-id/usb-Inside_Out_Networks_Edgeport_4_04-01-016677-if00-port0:/dev/ttyUSB0

FWIW, in my upb2mqtt flow, I discard repeated messages. The function node is named ā€œde-duplicateā€. :slight_smile:

I also discarded. It was aggressive so I dialed it back a bit.

Previously, if the current message was the same as the previous message, with both of their sequence numbers zeroed out then I would discard.

Now, if the sequence number of the current message is less than or equal to the sequence number of the previous message I keep the message. That means two identical messages (including their sequence number) are not treated as duplicate. This means that for example, a button press two times with no other events in between will be treated as two button presses.

New code will be up in 5 minutes with all my changes talked about.

Iā€™m unable to resolve Home Assistantā€™s inability to access the serial port. Iā€™ve applied all the solutions mentioned in this forum and elsewhere but Home Assistant continues to report the port doesnā€™t exist. Unfortunately, I canā€™t exercise the integration today.

The only thing I can report is that it read the UPE file and created lights and scenes.

Hi, Glenn!. Iā€™m getting more and more impressed watching what you, 123Taras, and bbrendon are doing. Iā€™d like to help you out with the multichannel devices. Can you message me or send me an email (salesatwebmtndotcom)?

I actually have quite a few PCS and SA devices that can be used to test with. Also, I can help with the TCP PIM. Also, if I can find a minute tomorrow, Iā€™ll run a status request on a multichannel device and show you the results. But, and its been a while since I looked at this, Iā€™m pretty sure youā€™re right that the second byte is the level of the second channel.

@gwww You are a machine!!! I just tried the last drop and both issues seem to be resolved. Iā€™m seeing state changes with button presses (links) and the repeat is working.

I might switch over this weekend.

This morning, with fresh eyes, I resolved yesterdayā€™s serial-port issue.

Although I had entered the correct information in docker-compose.yaml, I overlooked to use docker-compose up and started the container with the usual docker-compose start. As a result, it never registered the serial-port information I had added. :man_facepalming:


On to testing ā€¦

Iā€™ve only had time to perform one or two very simple tests (more to come). As per yesterdayā€™s report, the integration registered all the devices it found in the UPE file and today (with the serial-port now operational) it reported their current state. Hereā€™s a sample of one device:

For comparison, here is the same device in Premise. Iā€™ve highlighted two properties (ReportState and DefaultFadeRate) that Iā€™ve found to be useful and would be handy if they were also reported (as attributes) by the UPB integration.

Here is another example. This is a USB dimmer switch that is configured to behave as a non-dimmable switch. The integration correctly identified this arrangement (dimmable: false).

I used light.turn_on to set the patio lightā€™s brightness and it responded correctly. I manually adjusted its brightness and the UPB integration correctly reported the new level.

As expected, using Home Assistant to change a deviceā€™s brightness left Premise unaware of the modification. Similarly, using Premise to change brightness left Home Assistant unaware of it. Thatā€™s because the device doesnā€™t report its new status (so the secondary system isnā€™t informed of the deviceā€™s new state).

In contrast, changing the deviceā€™s brightness manually will inform both Home Assistant and Premise of the change if the device is programmed (via UPStart) to report its manual state-changes (i.e. the ReportState property in Premise). Thereā€™s no such setting to have a device report its state upon receiving a Goto command.

One thing I did notice is that if I used the UPB integration to turn off the device, it reported its new state because Premise was informed the device was now off. I need to investigate how and why the device reported it was commanded to turn off but not when commanded to change brightness. I think it may have a link defined for its rocker action (this is a US2-40).

More test results.

I ran this:
Screenshot from 2020-01-31 10-21-45

and the two devices linked to all_exterior (link 02) responded (turned on to their pre-defined levels).

I ran this:
Screenshot from 2020-01-31 10-22-06
and the devices responded (turned off).


EDIT
Removed the balance of my original post because it reported results that I can no longer reproduce.

__

EDIT 2

More results:

There appears to be a difference between the UPB message sent by a button (to activate/deactivate a link) and what is sent by the UPB integration (as well as what is suggested by the UPB Command Wizard).

To activate link 02, the Command Wizard suggests the following given a source of 0xFF and transmitting 4 repeated messages (Iā€™ve redacted the Network ID and Checksum):
870C--02FF20--

Compare that to what the button sends (the buttonā€™s ID is 0A):
890C--020A20FFFF--

It appends FFFF to the Activate command (20).

I donā€™t know why the buttons do that. The difference only appears to have an effect on Premiseā€™s UPB driver:

  • If it receives the buttonā€™s message with the FFFF then it knows it should set each light entityā€™s powerstate and brightness to 0.
  • If it receives the Wizardā€™s message without the FFFF (or from the UPB integration) then it only sets each lightā€™s powerstate to 0 but not the brightness level (i.e. brightness is left at whatever level it was when the light was on).

One could write this off as just a quirk in Premiseā€™s UPB driver. However, it appears to understand the significance of FFFF and behaves differently when itā€™s not present.

Itā€™s strange because I see nothing in the documentation suggesting thereā€™s optional data for Activate. Yet Iā€™m seeing FFFF in all the activate/deactivate messages transmitted by buttons (Iā€™ve tested it on two separate US2-40 switches).

I intend to look at the source code for Premiseā€™s UPB driver and see if it reveals anything.

1 Like

Examples of a multichannel device providing status:

Device SA US22-40 two channel dimmer programmed for NID 2, DID 1
Receive components Channel 1
Scene 1 100%
Scene 17 50%
Receive components Channel 2
Scene 1 100%
Scene 14 100%

Test:
Use one controller to send commands
Use second controller to log commands
Only display the first of multiple transmissions for clarity

Start off by asking NID 2, DID what itā€™s current status is
Fri Jan 31 10:17:19 MST 2020 : port 1 - PU07040201FF30C3 Report State NID 2 DID 1
Fri Jan 31 10:17:21 MST 2020 : port 1 - PU090002FF018600640B Status Report NID 2 DID 1 (Ch1 level 0% - hex 00) (Ch2 level 100% - hex 64)
Notice that Channel 1 is Off while Channel 2 is full ON

Next, send an Activate Scene 17 (set Channel 1 to 50%), then recheck current status
Fri Jan 31 10:19:30 MST 2020 : port 1 - PU89100211FF20FFFF37 Send Activate Scene 17
Fri Jan 31 10:21:19 MST 2020 : port 1 - PU07040201FF30C3 Report State NID 2 DID 1
Fri Jan 31 10:21:22 MST 2020 : port 1 - PU090002FF01863264D9 Status Report NID 2 DID 1 (Ch1 level 50% - hex 32) (Ch2 level 100%)

Then, send a Deactivate Scene 14 (set Channel 2 to 0%), then recheck current status
Fri Jan 31 10:21:31 MST 2020 : port 1 - PU8910020EFF21FFFF39 Send Deactivate Scene 14
Fri Jan 31 10:21:43 MST 2020 : port 1 - PU07040201FF30C3 Report State NID 2 DID 1
Fri Jan 31 10:21:45 MST 2020 : port 1 - PU090002FF018632003D Status Report NID 2 DID 1 (Ch1 level 50%) (Ch2 level 0%)

Send Activate Scene 1 (set both channels to 100%), the recheck current status
Fri Jan 31 10:50:10 MST 2020 : port 1 - PU87040201FF2053 Send Activate Scene 1
Fri Jan 31 10:51:14 MST 2020 : port 1 - PU07040201FF30C3 Report State NID 2 DID 1
Fri Jan 31 10:51:17 MST 2020 : port 1 - PU090002FF01866464A7 Status Report NID 2 DID 1 (Ch1 level 100%) (Ch2 level 100%)

PU 0704 02 01 FF 30 C3
PIM response Control Word (device command) NID DestID Source ID Command (Report State) Checksum

PU 0900 02 FF 01 86 64 64 A7
PIM response Control Word (device command) NID DestID Source ID Command (Status Report) CH 1 Level Ch 2 Level Checksum

Recognize that this is not my product so I donā€™t know many specifics about it. I tried testing it with GOTO commands and it appears that this device does not respond to a GOTO command,
either as a device command or a scene command. At least this unit doesnā€™t appear to.

Oops. Looks like the two sections towards the bottom where I show the command and break out the pieces lost its formatting when I posted it. My apologies.