Awesome, that will definitely make things easier.
Iām aware of the app note (I referenced it in this post) and āright thing to doā depends on the design requirements.
The app note suggests Message Mode because itās the path of least effort. Parsing packets in Message Mode is simpler than in Pulse Mode. The trade-off is Message Mode provides a subset of Pulse Modeās available information and control (yes but see note below). If the design requirements donāt need the additional info (or control) then Message Mode offers an easier implementation path.
I believe the most significant difference is that Pulse Mode allows for the implementation of auto-discovery (see note below). Thatās a feature commonly available in other lighting protocols like Zigbee, z-wave, etc.
I had mentioned in a previous post that Premiseās UPB driver uses Pulse Mode; I donāt need to import a UPE file whenever I use Upstart to add a new device, or change its configuration, or add/modify links. Premiseās driver discovers the changes by itself. It is also aware of all links (scenes) and which devices respond to a given link. It provides the end-user with a more seamless, transparent integration (compared to manually exporting/importing a configuration file). However, offering this level of end-user convenience comes at the cost of greater programming effort in Pulse Mode.
NOTE
After doing more reading, I need to walk back this statement (mea culpa). Pulse Mode does provide more information about the received UPB pulses, thereby allowing you to determine pulse energy and/or noise levels. However, it does not provide more commands than those available in Message Mode. Therefore it should be possible to implement auto-discovery using either Pulse or Message Modes.
Yeah, so I think pulse mode probably makes more sense to use, especially since the Premise UPB driver is very well commented while most other implementations do not seem to be nearly as solid. I also think having to load an Upstart generated config will end up making debugging significantly more complex than directly reading device registers like Premise does. Iām going to try and create a new library based on @gwwwās library but using pulse mode.
Yes, DeviceStateReport (MDID=0x86) provides information about the deviceās current state. The documentation indicates it can return up to 17 values but, in the tests Iāve performed so far, Iāve only seen one value (i.e. the deviceās Level). In the example you provided, the value following 86 is 00 and indicates the deviceās Level is zero (off).
I donāt know what values Arg2-Arg17 might represent. 
You can use ReportState (MDID=0x30) to request a DeviceStateReport from a given device. A device can also be programmed to generate a DeviceStateReport whenever someone manually changes its state (i.e. manually operating its rocker to dim/brighten it).
One thing Iāve observed is that a device will not automatically report its state upon receiving a command that affects its level (either via GoTo or a link). For example, if you create a scene (letās say itās link 03) containing six devices, when link 03 is transmitted those six devices wonāt report their state. If you send a GoTo to change a deviceās Level to 0x64 (decimal 100) it wonāt automatically report its new Level.
Whatās the implication of this behavior? Not much if your UPB network has only one controller (i.e. one PIM). However, if you have two (or more) systems, each with their own PIM, they can easily get out of sync. For example, if PIM1 commands Device5 to turn on, PIM2 is unaware of Device5ās new state because Device5 does not report it (i.e. no DeviceStateReport). PIM2 would have to listen to PIM1ās commands (GoTo, StartFade, StopFade, etc) and use them as a substitute for DeviceStateReport. Thatās kind of unusual behavior for a controller (to listen to another controllerās commands) since the usual assumption is itās the sole controller on the network (i.e. the only source of truth).
For the purposes of developing a UPB integration for Home Assistant, thereās probably no need to be overly concerned with keeping multiple controllers in sync. However, keep in mind that it exists. I quickly discovered it while experimenting with my UPB to MQTT flow. The moment I used it to turn on a device, Premiseās PIM was blind to the change (because it received no DeviceStateReport) and continued to indicate the device was off.
NOTE
You can request to have a device acknowledge receipt of a command. Thatās useful to confirm the device āgot your memoā. However, the transmitted acknowledgement doesnāt contain information about the deviceās new Level (or at least none that Iāve seen in my tests).
Lots of great info!
I see the command to read the setup registers and get the basic config of a device.
What I cannot find is a way to read all the links programmed into a device. The docs donāt say anything about that. Any idea how to do that? In order to do discovery, the links need to be discovered too.
Interesting. Any idea how to read that? Pseudo-code available? All I really want is a short doc that show the command to write and what the response looks like. The code appears to process a response. I canāt see the command written.
Thanks @jameshilliard. So as I suspected the links are read using the 0x10 get registers command. I can try reading a bunch a registers and reverse engineer as the docs only describe the first 64 bytes. The links are beyond what is described.
I have some experience with writing a native Premise driver (C++). I created one to support the USB-UIRT. The function you linked to is for populating the UPB device driverās record of each UPB deviceās ReceiveComponents (i.e. the links the UPB device supports and what itās supposed to do when a given link is activated/deactivated). This function is not reading those values from the physical UPB devices, just updating the driverās record of them.
For example, in this screenshot, the Explorer pane displays the UPB device driverās hierarchical structure. The Properties pane is showing the properties of:
Devices > upb > UPB_PIM > USQ2 > SwitchReceiveComponents > SwitchReceiveComponent
This is the first ReceiveComponent of the sixteen each UPB device can support.
It shows that activating LinkID=3 will cause USQ2 to set its LightLevel to 25% (at the specified FadeRate).
So the PopulateSwitchReceiveComponents function assigns values to the SwitchReceiveComponent properties.
The second link you posted points to code that reads a physical UPB deviceās registers. Specifically this line. It transmits this command: MDID_CORE_COMMAND_GETREGISTERVALUES which corresponds to command 0x10 which is this:
Head to page 53 of the UPB Technology Description document to see a little more about it:
1 Like
I wish to post a disclaimer:
I am not a UPB expert. Whatever I state about UPB is based on what Iāve learned by reading the documentation and experimenting with my own UPB system. I may make statements that are completely correct or completely wrong. I have no problem with discovering Iāve been wrong about something; itās all part of the learning process.
With that said, I now have my doubts that Pulse Mode is needed to perform discovery; I may need to walk back my statements about what can, or cannot, be achieved with Message Mode. You canāt read a UPB pulseās energy level in Message Mode but it now seems to me that you can do much more in this mode than I originally thought.
Yeah, I know it doesnāt send the actual query, but it does parse the cached RegisterData right?
So my understanding is that itās populating the LinkID mappings from a cached bytearray of the registers retrieved from the switches using ReadModuleRegisters right?
Yes to both questions. ReadModuleRegisters is sending GetRegisterValues (0x10) and requesting a range of registers.
Like @123, Iām going to state non-expert status too 
Pulse vs. message mode for discoveryā¦ As far as I can tell discovery in message mode would require that all 250 possible devices in the network get polled ā at about 2-3 seconds per device. In pulse mode, thereās a discovery mechanism that takes about 2 seconds to figure out which devices are out there and then you would only have to download info from those devices. Anyway, thatās my current understanding.
Irregardless of pulse or message mode I have not found a way to learn which links are in a device. I have tried reading the registers. I can get all the stuff in the document. When I go beyond byte 64 (which is the highest byte documented) all I get are 16 bytes of 0xFF. Happy to keep poking at that if anyone has any ideas.
I have a bunch of stuff working now. Iāve uploaded the latest to github (link in previous email by me). The API has the ability to:
- Read a UPStart UPE export file to setup all the links and devices
- Activate/Deactivate links
- Goto at specified rate for both lights and links.
- Some other bits are partly in there such as reading setup registers
- Read each deviceās dim state at startup
Also there is support for PIM-U (the TCP connected one). That has not been tested as I donāt have one. But the code is there. Iām only able to test with a serial PIM, connected with a serial to USB converter to my MacBook. The TCP and Serial code was done when I wrote the ElkM1 integration library a year or so ago. Interestingly, I only tested the TCP code for that one. Now Iām testing the opposite.
It seems reliable. Note that the library is an API and thereās really nothing to āuseā until its integrated with something that calls the library. You can try putting you UPE export file in the bin directory and then from the library root directory execute the command bin/simple. Thatās the simplest use of the API possible. How Iāve been testing is to āhackā lines of code in to turn things on/off.
If you want to try something out change the pass statement in links.py sync() method to:
link = self.pim.links.element[11]
link.activate()
The 11 is the number of a link. Change appropriately. You can also call link.deactivate() in a similar way.
Personally, Iām continuing down the message mode path. For the number of times discovery will actually happen, assuming that part gets written (the UPE file does everything I need right nowā¦ but Iām a geek and willing to write more )
My goal is to get some reliable HASS integration. Iām hoping to put some of that together on the weekend.
From page 21 of the documentation:
UPB Setup Registers are a block of at least 64 (up to 256) non-volatile 8-bit registers that are used to hold all of the configuration information about that UPB device. The first 64 Setup Registers are known collectively as the UPBID and are defined to have the same meaning in every UPB device. The remaining Setup Registers are known collectively as Configuration Registers and are open for use by the individual application designer.
Page 25 explains one of the uses of the Configuration Registers, namely a place to store the deviceās Receive Components and Transmit Components. In other words, where the deviceās links are stored.
One possible use for the Configuration Registers is for an optional, but very powerful, UPB concept known as Device Components. Device Components are logical objects (records) that the UPB device has, usually associated with physical entities on the device (such as pushbuttons, switches, indicators, input channels, output channels, etc.), that are intended to either receive or transmit UPB Link Packets for control purposes. Device Components (Figure 11) are classified into one of two types: either as Receive Components, which receive UPB Link Packets, or as Transmit Components, which transmit UPB Link Packets.
Do the deviceās you are testing have Receive and/or Transmit Components defined? If they donāt, seeing 0xFF would be expected.
EDIT
I figure some of your deviceās must have defined Receive Components because you stated youāve tested Activate/Deactivate links. So if they report nothing but 0xFF for their Device Components then thatās puzzling.
Thx @123. My bad, I picked a device that had no links programed. Just tried one that does and got the link information! Appreciate the pointer.
If it reports 0xFF (255) thatās the way it indicates thereās no assigned link. You can see that in this screenshot where Premiseās UPB driver reports that USQ4ās eighth ReceiveComponent is not assigned a link (itās 255).
A question for you: Why did you choose to parse the export file using its native UPE format? It can also be exported in JSON format which, I would imagine, is easier to import. Was there something you found in the UPE format that works better?
I didnāt know about the JSON format. Iām using Simply Automated version of UPStart and I donāt see a JSON export on it.
Anyway, the UPE format is trivial to parse. 75 lines of python code to parse it and add the objects. If interested take a look at parse_upstart.py in my repo.
Iāll take a look at the JSON later. Right now my next thing I want to write is the HASS code. The library is working well enough to start that integration. By no means does that mean that the library is complete, just good enough to see some stuff working in HASS.
Iām working right now on published the lib on PyPi. Should be up in the next hour.
Iām also using the Simply Automated version of UPStart. I canāt recall where I learned about it (possibly from a tech note) but when you export the configuration, append .json to the file name (this will override the default .upe extension). The contents of the resulting export file (filename.json) will be in JSON format, not the native UPE format. I realize this capability amounts to āeaster eggā status and they probably should make it more self-evident.
FWIW, for my upb2mqtt Node-Red flow, a combination of the File In and JSON nodes is sufficient to expose the UPB export file as a JSON object, easily navigable. Anyway, if you found UPE trivial to parse then itās all good.
Whenever you have a test article available, I can exercise it using my 20-odd UPB devices. Itās all from Simply Automated, consisting mostly of US11-40-W single rocker dimmers and a few US2-40 universal dimmers (top rocker plus 4 lower buttons). The buttons are all programmed to transmit links; all devices are programmed to receive links (Iāve defined several scenes).
I have one PCS PIM connected to Premise (my reference system) and a second Simply Automated CIM connected to an RPi (running my experimental upb2mqtt flow). I can re-purpose the CIM for testing your UPB integration. This way I can compare its behavior to the reference system.
I have the custom component working. Everything is all āalphaā quality, meaning that lots could change. It seems to work well for activating and deactivating links, goto on links and lights. To add to your configurations something such as thisā¦
upb:
url: serial:///dev/cu.KeySerial1:4800
file_path: ./upb.upe
About the url, it is like a ārealā url, so serial:// means connect using serial port. Anything that follows is the name of the device where the PIM is connected. So it could be something such as COM1 on Windows or something such as /dev/tty02 on Linux/Unix, or on my Mac the one you see above.
About the file_path this is the standard UPStart file export in UPE format. The file should be placed in your hass config directory. Note I have only tested running on my Mac and starting hass from my config directory. Your mileage may vary. On the Raspberry PI hass.io I would expect that what I have in my path would work.
Install hass support as you would a standard custom_component. No HACS support yet ā Iād rather spend my time solidifying the code first. https://github.com/gwww/upb
Note that the support UPB library moved to https://github.com/gwww/upb-lib. The UPB library is uploaded to PyPi and you DO NOT have to install it. Running HASS should install it for you.
Iād love to hear how it goes. What features would you like to see next?
1 Like