Hopefully some of the recent test results and observations posted above will help jump-start your project. In a nutshell:
Running the PIM in Pulse Mode provides access to more information (it’s the mode Upstart uses). However, it requires far more programming effort to decode/encode the packets in this mode. Running it in Message Mode allows for simpler packet-decoding but you lose the ability to access advanced features like determining pulse strength, the network’s configuration, etc.
Performing a ‘press and hold’ on the switch’s rocker will generate two UPB messages where the first one is completely misleading. In practice, the first message causes the UI to momentarily show a dimmer switch is fully on (or off) and then the second message shows its true brightness level.
To properly support links (scenes), the integration will need access to the network’s configuration (i.e. to determine which devices respond to a given link). If using Message Mode, the integration will need to import Upstart’s export file (there is an option to export the configuration in JSON format). In Pulse Mode you’re able to perform a discovery and maintain your own record of the network’s configuration.
I just stumbled across this thread. I started implementing a library to talk to the PIM. The goal is for HA integration.
So far I have it communicating with the PIM, importing a UPE file, handling activate and deactivate commands.
The python library I’m writing is based on the ElkM1 python library I wrote. I did that because it has async IO – so necessary for working with HA. My github id is gwww.
Nothing uploaded to git yet but I could put something up. It is in a VERY early stage but the structure is there for IO, receiving and decoding message, and the ability to send messages (but MUCH more to do there).
I have mostly SA devices. Two SA serial PIMs, one of them connected to my MacBook Pro with a serial to USB device.
I saw this thread over a year ago, didn’t realize people has reignited the discussion. I’ll upload stuff to git today and post.
I came across this thread as I was googling for more info on the protocol. In particular the status reports just show in the docs that they return lots of values. But no description of what the values are, trying to figure out how to use them. I have not explored much yet but suspect the value returned is the dim level. Here is one example message:
I’m aware of the app note (I referenced it in this post) and “right thing to do” depends on the design requirements.
The app note suggests Message Mode because it’s the path of least effort. Parsing packets in Message Mode is simpler than in Pulse Mode. The trade-off is Message Mode provides a subset of Pulse Mode’s available information and control (yes but see note below). If the design requirements don’t need the additional info (or control) then Message Mode offers an easier implementation path.
I believe the most significant difference is that Pulse Mode allows for the implementation of auto-discovery (see note below). That’s a feature commonly available in other lighting protocols like Zigbee, z-wave, etc.
I had mentioned in a previous post that Premise’s UPB driver uses Pulse Mode; I don’t need to import a UPE file whenever I use Upstart to add a new device, or change its configuration, or add/modify links. Premise’s driver discovers the changes by itself. It is also aware of all links (scenes) and which devices respond to a given link. It provides the end-user with a more seamless, transparent integration (compared to manually exporting/importing a configuration file). However, offering this level of end-user convenience comes at the cost of greater programming effort in Pulse Mode.
NOTE
After doing more reading, I need to walk back this statement (mea culpa). Pulse Mode does provide more information about the received UPB pulses, thereby allowing you to determine pulse energy and/or noise levels. However, it does not provide more commands than those available in Message Mode. Therefore it should be possible to implement auto-discovery using either Pulse or Message Modes.
Yeah, so I think pulse mode probably makes more sense to use, especially since the Premise UPB driver is very well commented while most other implementations do not seem to be nearly as solid. I also think having to load an Upstart generated config will end up making debugging significantly more complex than directly reading device registers like Premise does. I’m going to try and create a new library based on @gwww’s library but using pulse mode.
Yes, DeviceStateReport (MDID=0x86) provides information about the device’s current state. The documentation indicates it can return up to 17 values but, in the tests I’ve performed so far, I’ve only seen one value (i.e. the device’s Level). In the example you provided, the value following 86 is 00 and indicates the device’s Level is zero (off).
I don’t know what values Arg2-Arg17 might represent.
You can use ReportState (MDID=0x30) to request a DeviceStateReport from a given device. A device can also be programmed to generate a DeviceStateReport whenever someone manually changes its state (i.e. manually operating its rocker to dim/brighten it).
One thing I’ve observed is that a device will not automatically report its state upon receiving a command that affects its level (either via GoTo or a link). For example, if you create a scene (let’s say it’s link 03) containing six devices, when link 03 is transmitted those six devices won’t report their state. If you send a GoTo to change a device’s Level to 0x64 (decimal 100) it won’t automatically report its new Level.
What’s the implication of this behavior? Not much if your UPB network has only one controller (i.e. one PIM). However, if you have two (or more) systems, each with their own PIM, they can easily get out of sync. For example, if PIM1 commands Device5 to turn on, PIM2 is unaware of Device5’s new state because Device5 does not report it (i.e. no DeviceStateReport). PIM2 would have to listen to PIM1’s commands (GoTo, StartFade, StopFade, etc) and use them as a substitute for DeviceStateReport. That’s kind of unusual behavior for a controller (to listen to another controller’s commands) since the usual assumption is it’s the sole controller on the network (i.e. the only source of truth).
For the purposes of developing a UPB integration for Home Assistant, there’s probably no need to be overly concerned with keeping multiple controllers in sync. However, keep in mind that it exists. I quickly discovered it while experimenting with my UPB to MQTT flow. The moment I used it to turn on a device, Premise’s PIM was blind to the change (because it received no DeviceStateReport) and continued to indicate the device was off.
NOTE
You can request to have a device acknowledge receipt of a command. That’s useful to confirm the device ‘got your memo’. However, the transmitted acknowledgement doesn’t contain information about the device’s new Level (or at least none that I’ve seen in my tests).
I see the command to read the setup registers and get the basic config of a device.
What I cannot find is a way to read all the links programmed into a device. The docs don’t say anything about that. Any idea how to do that? In order to do discovery, the links need to be discovered too.
Interesting. Any idea how to read that? Pseudo-code available? All I really want is a short doc that show the command to write and what the response looks like. The code appears to process a response. I can’t see the command written.
Thanks @jameshilliard. So as I suspected the links are read using the 0x10 get registers command. I can try reading a bunch a registers and reverse engineer as the docs only describe the first 64 bytes. The links are beyond what is described.
I have some experience with writing a native Premise driver (C++). I created one to support the USB-UIRT. The function you linked to is for populating the UPB device driver’s record of each UPB device’s ReceiveComponents (i.e. the links the UPB device supports and what it’s supposed to do when a given link is activated/deactivated). This function is not reading those values from the physical UPB devices, just updating the driver’s record of them.
For example, in this screenshot, the Explorer pane displays the UPB device driver’s hierarchical structure. The Properties pane is showing the properties of:
Devices > upb > UPB_PIM > USQ2 > SwitchReceiveComponents > SwitchReceiveComponent
This is the first ReceiveComponent of the sixteen each UPB device can support.
It shows that activating LinkID=3 will cause USQ2 to set its LightLevel to 25% (at the specified FadeRate).
So the PopulateSwitchReceiveComponents function assigns values to the SwitchReceiveComponent properties.
The second link you posted points to code that reads a physical UPB device’s registers. Specifically this line. It transmits this command: MDID_CORE_COMMAND_GETREGISTERVALUES which corresponds to command 0x10 which is this:
I am not a UPB expert. Whatever I state about UPB is based on what I’ve learned by reading the documentation and experimenting with my own UPB system. I may make statements that are completely correct or completely wrong. I have no problem with discovering I’ve been wrong about something; it’s all part of the learning process.
With that said, I now have my doubts that Pulse Mode is needed to perform discovery; I may need to walk back my statements about what can, or cannot, be achieved with Message Mode. You can’t read a UPB pulse’s energy level in Message Mode but it now seems to me that you can do much more in this mode than I originally thought.
Yeah, I know it doesn’t send the actual query, but it does parse the cached RegisterData right?
So my understanding is that it’s populating the LinkID mappings from a cached bytearray of the registers retrieved from the switches using ReadModuleRegisters right?
Like @123, I’m going to state non-expert status too
Pulse vs. message mode for discovery… As far as I can tell discovery in message mode would require that all 250 possible devices in the network get polled – at about 2-3 seconds per device. In pulse mode, there’s a discovery mechanism that takes about 2 seconds to figure out which devices are out there and then you would only have to download info from those devices. Anyway, that’s my current understanding.
Irregardless of pulse or message mode I have not found a way to learn which links are in a device. I have tried reading the registers. I can get all the stuff in the document. When I go beyond byte 64 (which is the highest byte documented) all I get are 16 bytes of 0xFF. Happy to keep poking at that if anyone has any ideas.
I have a bunch of stuff working now. I’ve uploaded the latest to github (link in previous email by me). The API has the ability to:
Read a UPStart UPE export file to setup all the links and devices
Activate/Deactivate links
Goto at specified rate for both lights and links.
Some other bits are partly in there such as reading setup registers
Read each device’s dim state at startup
Also there is support for PIM-U (the TCP connected one). That has not been tested as I don’t have one. But the code is there. I’m only able to test with a serial PIM, connected with a serial to USB converter to my MacBook. The TCP and Serial code was done when I wrote the ElkM1 integration library a year or so ago. Interestingly, I only tested the TCP code for that one. Now I’m testing the opposite.
It seems reliable. Note that the library is an API and there’s really nothing to “use” until its integrated with something that calls the library. You can try putting you UPE export file in the bin directory and then from the library root directory execute the command bin/simple. That’s the simplest use of the API possible. How I’ve been testing is to “hack” lines of code in to turn things on/off.
If you want to try something out change the pass statement in links.pysync() method to:
link = self.pim.links.element[11]
link.activate()
The 11 is the number of a link. Change appropriately. You can also call link.deactivate() in a similar way.
Personally, I’m continuing down the message mode path. For the number of times discovery will actually happen, assuming that part gets written (the UPE file does everything I need right now… but I’m a geek and willing to write more )
My goal is to get some reliable HASS integration. I’m hoping to put some of that together on the weekend.
UPB Setup Registers are a block of at least 64 (up to 256) non-volatile 8-bit registers that are used to hold all of the configuration information about that UPB device. The first 64 Setup Registers are known collectively as the UPBID and are defined to have the same meaning in every UPB device. The remaining Setup Registers are known collectively as Configuration Registers and are open for use by the individual application designer.
Page 25 explains one of the uses of the Configuration Registers, namely a place to store the device’s Receive Components and Transmit Components. In other words, where the device’s links are stored.
One possible use for the Configuration Registers is for an optional, but very powerful, UPB concept known as Device Components. Device Components are logical objects (records) that the UPB device has, usually associated with physical entities on the device (such as pushbuttons, switches, indicators, input channels, output channels, etc.), that are intended to either receive or transmit UPB Link Packets for control purposes. Device Components (Figure 11) are classified into one of two types: either as Receive Components, which receive UPB Link Packets, or as Transmit Components, which transmit UPB Link Packets.
Do the device’s you are testing have Receive and/or Transmit Components defined? If they don’t, seeing 0xFF would be expected.
EDIT
I figure some of your device’s must have defined Receive Components because you stated you’ve tested Activate/Deactivate links. So if they report nothing but 0xFF for their Device Components then that’s puzzling.
