UPB lighting

Nuts, while suggesting to model a link as a light (and a scene), I overlooked the glaringly obvious issue that a light must show state but a link is stateless. Duh. Scratch that idea (and it appears to be the same advice from developers on Discord).

A given link can be represented by two scenes, one to activate it and the other to deactivate it.

A scene can’t support a Link Packet’s ability to send Goto, FadeRate, etc to linked devices. Nevertheless, at least it provides an easy way to activate/deactivate a link (heck, even UPStart refers to them as scenes).

FWIW, this is modeled differently in Premise. Premise supports the concept of a button (think of it as a “momentary input_boolean”). I can make a single-press of a button activate a link and a double-press deactivate it. That’s the same way UPStart allows you to configure the physical buttons of a UPB device. Therefore Premise models it exactly as its done in the real world. Easy-peasy.

On the other hand, Premise does not model a Link Packet’s ability to send other commands. The olive branch it offers is that its UPB driver can accept a raw UPB command:
{nid}{link}{mdid}{data}
and then it appends the required checksum prior to sending it. I don’t know if such a thing would is permissible in Home Assistant. I believe integrations are supposed to maintain their distance from the “bare metal”.

Certainly a bug to fix. However, the cause is that the upb.upe cannot be found. If it can be found then my setup is working well. I push a small change that prints the error to the log (right now its not a log call just a simple print). Here’s the message I get when it can’t find my UPE file:

Cannot open UPStart file './upb.upe': [Errno 2] No such file or directory: './upb.upe'

Where did you put the file? What are you running on? Is it a Raspberry Pi? If so you could try changing the path to /config/upb.upe

One more idea. I run hass when my current directory is in the config directory. I’m pretty sure that you are not doing that. So whatever the path is to your config directory put that in the config.

I can add custom services to a Scene for deactivate, goto, etc. So a scene could do some of those things. The question is, does that make the most sense – to have a Scene with custom services?

The hass Scene sounds similar to a Premise button.

I started hass with : /opt/python-venvs/hass/bin/hass -c /opt/hass-upb
So I’m guessing that means the file should be in /opt/hass-upb ?

Once I put the full path to the file it worked. Maybe ./upb.upe needs to be in quotes? I don’t know.

Thoughts…

  1. Awesome.
  2. Confusion. I was wondering what all these additional lights were. Then I realized they are links. I haven’t looked in upstart in a long time :slight_smile:
  3. The additional information (attributes) on the UPB modules is excellent! Not sure what I’d need it for but I’ve been so crippled, who knows. I’ll probably find a use for it.
  4. The dimmable attribute on each device based on real data is great. Mine just listed everything dimmable no matter what.
  5. The links exposed as lights is weird but it seemed to work. Having links show “on” in the GUI is definitely weird mostly because once the devices associated to the link go on to other states, it still shows on (which makes sense).

The only thing I didn’t like is the default naming of entity IDs. Its not light.upb_*. I’m not sure if upstream HASS allows what I want but maybe you could make it a configuration option? I’m not looking forward to renaming everything manually.

I couldn’t find any bugs. I think I’ll try switching over to it once the link stuff is sorted out.

This is super exciting though. I’ve been wanting this for soooo long.

Thanks for all the work you’ve done on this.

That sounds … unusual. I suspect this might cause balloob to squint.

The hass Scene sounds similar to a Premise button.

Mmm, sort of. Buttons (in Premise) also have button-actions (press, hold, release, etc) whereas a scene is just turn_on.

To be fair, it really shouldn’t ‘show on’ because UPB doesn’t track the state of a link. That’s why modeling it as a light (or even a switch) is problematic. Like you said, the linked devices may go to other states (like all are off) yet the ‘light link’ continues to report on. It even feels weirder when it comes times to activate that scene again. The light link is still reporting its state is on and now you want to turn it on again. Weird.

It needs to be a stateless entity. Something that can be triggered but never reports it has changed state. It’s like “do it” and “undo it”. One scene to activate a link and another to deactivate it.

scene.link_03_activate
scene.link_03_deactivate
  action:
  - service: scene.turn_on
    entity_id: scene.link_03_activate

Added this to the development channel… UPB (Universal Powerline Bus) integration

I’ve intentionally left out as many details as I thought I could to keep the focus on the modelling issue rather than on how UPB works. I tried to keep the UPB details as slim as I could.

Wow, I was gone a few days and this forum thread went crazy. Looks like a lot of work going on.

I wanted to comment that being a UPB provider, I would consider you guys - Glenn, 123Taras, JamesHilliard, and bbrendon - UPB experts. It’s been years since I got down into these low level details and it’s been a long time since I’ve talked with anyone who has as well. In fact, in many respects, you’re probably more up to date than I am, as it’s been so long since I’ve dived this deep. So, don’t discount your level of expertise - it’s much greater than almost anyone else out there.

A couple of things, then I’m going to shut up and stay out of your way (I’ll be happy to help out with documentation and can try to answer any questions I can, but you guys are already pretty advanced - and if anyone needs help with products to test with, let me know). There’s another System document you guys might want to look at, if you haven’t already. This is the PIM System Description document. This is where you can find the detailed description of Pulse and Message modes, plus lots more information. For example, there is a way to get signal level and noise information even when in Message mode. You can download from the PCS website or I can get you a copy - just let me know.

I know there is a desire by some to use Pulse mode, in order to simplify device discovery - a very laudable goal. However, using Pulse mode adds a lot of overhead to the code and project - I would suggest at least starting in Message mode and get something workable, which appears to be the way Glenn is approaching it with his implementation, and then later move to Pulse mode to simplify operation, if so desired.

There was some discussion about reading device registers on individual devices in order to get more detailed information, for example which devices belong to which links (scenes - we used the terms links for so long, it’s hard to break). A big issue with this would be determining what registers to read for this information. The main registers for NID, MID, PID, Firmware Version, Network name, network password, etc - will have good consistency between products and manufacturers, but once you get past those, the registers are going to be different depending on the type of unit, the manufacturer of the unit, whether it is single or multichannel, etc. So, you would need to read the MID and PID, then have a table showing what registers to read for those devices. It’s going to get complicated and trying to get that information from most of the manufacturers is going to be problematic for multiple reasons. It could be reverse engineered but that’s going to take quite a bit of time and effort, plus you would need sample units of all desired units.

And, all of this information is already included in the Upstart UPE file. So, again, I would encourage an initial faster development that pulls that information from the UPE file then enhance it later if so desired as part of a discovery process.

I know practically nothing about HASS. I did just bring up a RPi4 with HASS and will integrate Glenn’s implementation shortly so I can play with it. bbrendon has been a huge help - thanks so much. I won’t be able to contribute, but at least I can start playing with HASS and UPB.

Now, I’ll get out of your way and try to shut up. If I can help any, let me know. Also, as this comes together, I’ll work on offering a special product discount code for HASS users to purchase product at a nice discount. bbrendon asked the moderators about this and they seem to be fine with it.

1 Like

New release up on github. Links are now represented as hass Scenes. I started using the github Releases so you can see them there with changes and do compares. You will have to cleanup your old entities. FYI… as I’m testing I use a custom configuration that can be easily deleted or recreated.

While perusing the LIFX integration, I couldn’t help but notice it has custom services:

lifx.set_state
lifx.effect_pulse
lifx.effect_color_loop
lifx.effect_stop

If the LIFX integration is permitted to have custom services (allowing it to talk directly to the underlying LIFX library versus through an existing Home Assistant light) then it opens the door to allowing the UPB integration to have its own custom service for a UPB link.

@gwww how do I detect a link that was activated on a physical switch in HASS?

Also, it looks like a message is considered a repeat even after 40 seconds. (I obfuscated some of my UPB info below)

2020-01-30 15:58:00 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:00 DEBUG (MainThread) [upb_lib.links] Activate BedMaster-Button4 x_x
2020-01-30 15:58:00 DEBUG (MainThread) [upb_lib.links]   Updating 'LivingRm Lamp1' to dim level 0
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:01 DEBUG (MainThread) [upb_lib.message] Repeated message!!!



2020-01-30 15:58:44 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:44 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.message] Repeated message!!!
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.proto] message received:     update 'X'
2020-01-30 15:58:45 DEBUG (MainThread) [upb_lib.message] Repeated message!!!

Question 1 was how to detect when a link was activated on a physical switch. The PIM will receive a message and use the contents to update all of the lights affected by that link. Nothing in the actual scene object is changed. What would you like to happen and why?

With respect to question 2 and the repeated messages. Here is what the code does. When the first message is received it is striped of sequence number and then saved. When the next message it is striped of sequence number. If it is the same as the saved message then you get the “Repeated message!!!” log. If it is different, it is saved and then processed.

The theory is that repeated messages, no matter the time frame, have no value. Nothing changes in hass.

Is there a practical application for not throwing away duplicate messages. Note, the code SHOULD throw away dups when the sequence number is increasing from the previous message. They are truly duplicate. I think what we are discussing is when the seqence number in a message is less than the sequence number in the previous message.

Hope all the rambling makes sense!

With respect to the log. Its not very useful without the actual messages. The only thing in the messages that is possibly sensitive is the network ID. But even that is not that sensitive. So, I can try guessing at what you are seeing and I’ll give it yeoman’s effort but I may not be answer questions that way. You can always email me a log or portion of if you trust me enough. I basically delete such stuff as soon as the problem is fixed. Or you can edit the network ID out of each message. Or…

Yes, LIFX has services on lights. Last night I uploaded a new integration where I added services on Scenes (that’s how UPB links are represented now). Like to know what you think.

The hass devs recommended I give Scenes a shot. I did.

Hey @webmtn! Thank you for your support! I feel like I getting to be an expert. I feel like I figuring out most of the protocol’s ins and outs. Stuff that I cannot test because I don’t have the equipment is any kind of multi-channel device. I added support for multi-channel this afternoon (sorry folks – that will cause the unique_ids to change for lights). The other thing that I cannot test is the TCP PIM/CIM. The code is there and “should” work because it was taken from the ElkM1 library that I wrote. Of should and does never quite line up :wink: .

One protocol question that I guessed the answer to is around goto and channels. Goto for channels take 3 arguments. Level, rate, and channel. The spec doesn’t talk about what to use for default rate when sending a channel goto.

The other bit I have no idea about is the device status report for multi-channel unit. Got it figured out for single channel. I’m just guessing but the second byte of the status report would be the status for channel 2.

Another thing that could be useful is if you have connections with PCS or Simply Automated. Are they interested in this work? Maybe they can help test?

I believe the code I have for UPB is structurally pretty good shape. Now the work is supporting additional command such as Blink, etc. And there is questions like @bbrendon’s and link status. Most of these are better understanding people’s requirements then the implementation is relatively easy.

@gwww thanks. I’ll provide the details next time. I just wasn’t sure. I’d rather not give out my network id but I know it’s easily grabbed off the powerline anyway.

Regarding the links… Right now I have buttons on my UPB switches I use to trigger things in HASS. There is a UPB button near the front door that is technically a link, but when HASS sees it is pressed, it runs an exit script to lock doors, make final warnings, etc. Basically I use UPB buttons to do things (in HASS) other than UPB lights.

Ditto. One of the four buttons on a US2-40 is programmed to activate a link when pressed (deactivate when double-pressed).

This link has no UPB devices linked to it. It’s sole purpose is to serve as a command signal that is heard by Premise. Upon receiving the link, it runs an automation to control an X10 dimmer module (for a table lamp).

Similarly, another button is programmed to turn all interior UPB lights on (of off). Naturally, this link is tied to all interior UPB lights. However, I still have one interior light driven by an X10 dimmer switch. Therefore, when Premise receives the link it is programmed to control the X10 switch.

tl;dr
I’m using UPB links to serve as signals to trigger automations.


Still in the process of transferring my Digi Edgeport/4 (USB to 4-port RS232 converter) from my RPi test-bed (for Node-Red upb2mqtt) to my laptop running a test instance of Home Assistant (Core). Your software is already installed (just refreshed it with the latest version) so it’s just a matter of getting the Edgeport running with Ubuntu (hopefully it will far less of an adventure than Raspbian which doesn’t support it by default).

That’s a good way to put it :slight_smile:

gwww : Regarding the repeat messages. I was re-reading what you wrote. So say you want to press the button to trigger the cat feeder that spits out food quantity of X, so you press the button. You press it again to feed them 2x. But without the repeat message check timing out, you get 1x when you expect 2x. On my current setup I have the repeat message check expire after 3 seconds.

Now that I think about it, the Deconz integration exposes the button actions of a remote control as events. From Deconz Remote Control Devices:

Remote controls (ZHASwitch category) will not be exposed as regular entities, but as events named deconz_event with a payload of id and event and in case of the Aqara Magic Cube also gesture . Id will be the device name from deCONZ and Event will be the momentary state of the switch.

When you press a button on the US2-40, the resulting UPB message contains the link ID and which UPB device produced it. Only thing you don’t know is what button action was used to create it. I believe that information would be in the UPE file … although I don’t see a need for it. The key thing is that a UPB link could be expressed as an event (upb_event).

I’ve added an attribute to the Scene called “last_change” which is a timestamp of when the link was last changed. Watch for that value to change. I’m testing it then will upload it later tonight.

I’ll be offline most of the weekend. Keep up the testing and questions!!

I’ve also changed the “Repeated message!!!” logic. It will only call repeated when the sequence number is increasing from the previous message that is otherwise identical.

Ubuntu easily discovered the Edgeport but then things turned south. I encountered the following error message in Home Assistant:

Could not connect to UPB PIM ([Errno 2] could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: ‘/dev/ttyUSB0’). Retrying in 1 seconds

Configuration:

upb:
 url: serial:///dev/ttyUSB0:4800
 file_path: ./upb.upe

The path /dev/ttyUSB0 exists.

crw-rw---- 1 root dialout 188, 0 Jan 30 21:52 /dev/ttyUSB0
crw-rw---- 1 root dialout 188, 1 Jan 30 21:52 /dev/ttyUSB1
crw-rw---- 1 root dialout 188, 2 Jan 30 21:52 /dev/ttyUSB2
crw-rw---- 1 root dialout 188, 3 Jan 30 21:52 /dev/ttyUSB3

When logged in with my user account, this doesn’t work:
cat /dev/ttyUSB0

However, this does:
sudo /dev/ttyUSB0
If I produce some UPB traffic, gibberish appears on the command line thereby confirming the UPB PIM is reporting on /dev/ttyUSB0.

Then I realized I’m running Home Assistant (Core) as a Docker container. I’ve never had a need to allow this arrangement to access a serial port. After some googling, I added the following to the homeassistant section of the docker-compose file:

    devices:
      - /dev/ttyUSB0:/dev/ttyUSB0

I restarted the container and … no joy. The error message remains the same. Looks like I need to do some more googling.


Using Portainer, I opened the Home Assistant container’s console. I listed the files in /dev and it doesn’t contain /dev/ttyUSB0. :man_shrugging:


I tried this very specific mapping but Home Assistant still complains that /dev/ttyUSB0 doesn’t exist.

    devices:
      - /dev/serial/by-id/usb-Inside_Out_Networks_Edgeport_4_04-01-016677-if00-port0:/dev/ttyUSB0