UPB lighting

Hi All, new to HASS, moving from CQC; I see some old friends here!!!

I just purchased a PulseWorx gateway with the hopes of connecting from HASS to the gateway via IP. When I configure the integration it never connects and slows HASS down to a crawl (repeat disconnect errors in the log.) so I have removed.

I’m running HASS on a Pi 4. I may have been mistaken in understanding that the gateway would function the same way as the IP PIM from PulseWorx.

I am able to connect to the gateway via Upstart or from the phone app and control my lights so I know it is configured and working. The setup documentation for the gateway has you set up a user ID and password for the gateway, that may be part of the problem but I don’t see any documentation with regard to entering said info for HASS to process with.

Any suggestions on how to make this work would be most appreciated. I’ll be reaching out to HomeControls support tomorrow to get some feedback there as well. I may have to return the gateway and purchase the PulseWorx PIM IP instead. Confirmation that the PIM IP works with this integration would be appreciated.

Thanks!

It’s probably just the initialization process being a little different between the different types of gateways, I’m actually working on that right now as part of my UPB pulse mode library since it needs to be able to put the PIM into pulse mode. It shouldn’t be that difficult to make it work with my pulse mode library or @gwww’s message mode library. If you want to port forward it and PM me the IP/port I can try and take a look at what’s going on.

Edit: I pushed an update to my library adding some basic PIM register read support.

If you run it with:

python3 -m upb.tools.dumpreg --host=192.168.1.32 --network=1 --device=5

There might be something in the logs that shows what the problem is.

Welcome (back) @batwater!

Post debug logs. If you browse through this thread you’ll see how to configure hass to collect the right level.

I use TCP to access my PIM but it is through a serial to TCP program that I wrote. I’ve not had my hands on a real TCP PIM or gateway. Logs will help.

Thanks Glenn. After turning debugging on I get the following in the log (repeating)

2020-07-30 15:00:56 INFO (MainThread) [upb_lib.upb] Connecting to UPB PIM at tcp://192.168.1.218
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] connected callback
2020-07-30 15:00:56 INFO (MainThread) [upb_lib.upb] Connected to UPB PIM
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write ''
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] write_data ''
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.upb] Synchronizing status of UPB network...
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '0700101FFF309B'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001036FF3084'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001005FF30B5'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001033FF3087'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001064FF3056'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001006FF30B4'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001017FF30A3'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001015FF30A5'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001004FF30B6'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001008FF30B2'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001019FF30A1'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001016FF30A4'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001018FF30A2'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001038FF3082'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '0700101AFF30A0'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '0700101BFF309F'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001009FF30B1'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '0700103CFF307E'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '0700103DFF307D'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001051FF3069'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001052FF3068'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001007FF30B3'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001028FF3092'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001050FF306A'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001014FF30A6'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001032FF3088'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001034FF3086'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001035FF3085'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '0700101EFF309C'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '0700103EFF307C'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001037FF3083'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '07001039FF3081'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] queued write '0700100AFF30B0'
2020-07-30 15:00:56 DEBUG (MainThread) [upb_lib.proto] disconnected callback
2020-07-30 15:00:56 WARNING (MainThread) [upb_lib.upb] PIM at tcp://192.168.1.218 disconnected

It would appear that the gateway doesn’t like what I send so it just disconnects. When I get some time I’ll look for some gateway development docs and see what it says. If you happen to find some first that would be great :wink: There is surely some code to be written to work with the gateway. If its a tweak I’ll get on it. If its more then we have to see.

I found this authentication sequence in the upstart sources which seems to be what gets used for that type of gateway:

CUPBPIM::Status PulseWorxGatewayConnect 
   (int protocol, CString userName, CString password, CString& version, int *pctClients)
{
   GetAppProperties;
   GetUPBPIMObject;

   CUPBPIM::Status rc;
   CUPBPIM::GatewayRequest request;
   CUPBPIM::GatewayConnectRequest connectRequest;

   request.type = CUPBPIM::GatewayConnectMessage;
   request.pConnectRequest = &connectRequest;

   int buildNumber = ((TheApp *)AfxGetApp())->m_BuildNumber;
   int majorVersion = ((TheApp *)AfxGetApp())->m_MajorVersion;
   int minorVersion = ((TheApp *)AfxGetApp())->m_MinorVersion;

   connectRequest.send.Format("PulseWorx App/%d.%d.%d/%d", majorVersion, minorVersion, buildNumber, protocol);

   rc = SendGatewayRequest(&request, pProperties->gatewayReplyTimeout);
   if (rc != CUPBPIM::OK)
      return (rc);

   //
   // I can get back:
   //
   // "MAX CONNECTIONS REACHED"
   //
   // "PULSE MODE ACTIVE" - if a pulse mode app is currently connected
   //
   // "PIM NOT INITIALIZED" - the gateway is still setting uup the PIM
   //
   // "PCS PIM-IP2/<version>/<accepted protocol>/AUTH NOT NEEDED/<client count string>"
   //
   // "PCS PIM-IP2/<version>/<accepted protocol>/AUTH REQUIRED/<128 characters>"
   //
   CString response(connectRequest.receive);

   if (response == "MAX CONNECTIONS REACHED")
      return (CUPBPIM::errGatewayConnectionLimit);

   if (response == "PULSE MODE ACTIVE")
      return (CUPBPIM::errGatewayPulsemodeAppInUse);

   if (response == "PIM NOT INITIALIZED")
      return (CUPBPIM::errGatewayNotInitialized);

   if (response == "FIRMWARE CORRUPT - FLASH WITH UPSTART")
      return (CUPBPIM::errGatewayFirmwareCorrupt);

   if (response.GetLength() < 12)
      return (CUPBPIM::errGatewayConnectError);

   response += '/';

   CString prefixText, versionText, protocolText, authText, suffixText;
   int protocolAccepted;

   CString clientText;
   int ctClients;

   prefixText = response.SpanExcluding("/");
   response = response.Right(response.GetLength() - (prefixText.GetLength() + 1));

   versionText = response.SpanExcluding("/");
   response = response.Right(response.GetLength() - (versionText.GetLength() + 1));

   protocolText = response.SpanExcluding("/");
   response = response.Right(response.GetLength() - (protocolText.GetLength() + 1));

   authText = response.SpanExcluding("/");
   response = response.Right(response.GetLength() - (authText.GetLength() + 1));

   suffixText = response.SpanExcluding("/");
   response = response.Right(response.GetLength() - (suffixText.GetLength() + 1));

   if (!ConvertTextToInt(protocolText, &protocolAccepted))
      return (CUPBPIM::errGatewayConnectError);

   if (protocolAccepted != protocol)
      return (CUPBPIM::errGatewayNoSupportedProtocol);

   if (authText == "AUTH NOT NEEDED")
   {
      version = versionText;
      clientText = suffixText;

      CString clientCount = suffixText.SpanExcluding(" ");
      ConvertTextToInt (clientCount, &ctClients);
      *pctClients = ctClients;

      return (CUPBPIM::OK);
   }

   else if (authText != "AUTH REQUIRED")
   {
      return (CUPBPIM::errGatewayConnectError);
   }

   if (suffixText.GetLength() != 128)
      return (CUPBPIM::errGatewayConnectError);

   BYTE authData[64];
   int cbAuth;

   if (!HexToBytes(suffixText, authData, 64, &cbAuth))
      return (CUPBPIM::errGatewayConnectError);

   BYTE hashResult[16];
   MD5_PIMIP2Challange (authData, pProperties->connection.gatewayPassword, hashResult);

   connectRequest.send.Format("%s/%s", pProperties->connection.gatewayUserName, HexImage(hashResult, 16, FALSE));
   connectRequest.receive = "";

   rc = SendGatewayRequest(&request, pProperties->gatewayReplyTimeout);
   if (rc != CUPBPIM::OK)
      return (rc);

   response = connectRequest.receive;
   response += '/';

   CString resultText;

   resultText = response.SpanExcluding("/");
   response = response.Right(response.GetLength() - (resultText.GetLength() + 1));

   clientText = response.SpanExcluding("/");
   response = response.Right(response.GetLength() - (clientText.GetLength() + 1));

   if (resultText == "AUTHENTICATION FAILED")
      return (CUPBPIM::errGatewayLoginFailed);

   else if (resultText != "AUTH SUCCEEDED")
      return (CUPBPIM::errGatewayConnectError);

   version = versionText;

   CString clientCount = clientText.SpanExcluding(" ");
   ConvertTextToInt (clientCount, &ctClients);
   *pctClients = ctClients;

   return (CUPBPIM::OK);
}

I just found the docs here: http://www.pcslighting.com/pulseworx/resources

There’s a login sequence required and messages to the PIM are prefaced with a command byte. The response from the gateway is also different. It’s not a trivial piece of work to get this working and I don’t have a gateway to test with.

I’ll ponder this for a bit. Happy to take well-constructed patches to the UPB lib :wink:

Edit: It appears that the PIM-IP uses the same protocol as the Gateway.

I actually just pushed up some decoding for it to my upbshark proxy, it’s not complete but it decodes the auth sequence challenge response properly.

It’s pretty much just a wrapped protocol, a little annoying to deal with but not super difficult.

Edit:
Pushed some initial gateway command unwrapping for upbshark.

No, I’m pretty sure it doesn’t, there appears to be a few common codepaths but the auth sequence and protocol wrapping looks to be mostly different.

I’m happy to remotely share my gateway to work through this. My only other choice is to attempt to return the unit and purchase the PIM-IP. I would rather hold on to what I have if at all possible plus this expands the compatibility of the integration and allows for more secured remote communication should that use case be desired by someone.

Ok, thx, when I get some time I’ll reach out. Or, if someone else picks it up and integrates, that’ll work too.

I’m not 100% sure, but what I understand is that the PIM-IP has the same protocol requirements as the gateway. The TCP support in the library now is only “ser2tcp” support which means that connecting using TCP is straight up like its talking to a serial PIM without login etc.

Yeah, I don’t think it will be too difficult to support, I’ve gotten enough decoding working in my upbshark tracing tool that I think I should be able to handle it. It might actually be harder handling it in the upbshark tracing tool than in the normal UPB client library due to the double sided state tracking being more complex than when using a normal client library. I’ll try and see if I can get a client/protocol unwrapper working with it today.

Well the PIM-IP from my reading of the upstart code uses PacketCheckTelnet for its packet processing while the pulseworx gateway uses PacketCheckGateway. I’m pretty sure the PacketCheckTelnet codepath is what’s used for the plain ser2tcp/ser2net serial PIM bridge protocol. I’ve mostly been testing my upb pulse library using a ser2net bridge and serial/USB PIM myself so I’m quite sure that the outer gateway protocol is different from the normal simple TCP bridged serial protocol. The good thing is that the inner protocol encapsulated within the gateway protocol appears to be identical to the serial/ser2net bridge protocol so a protocol wrapper/unwrapper along with auth handling should be all that’s needed as opposed to a completely separate implementation of the base serial/TCP bridge protocol handler.

enum Hardware {
   HardwareUnknown,
   UPB_PIM,              // RS232 PIM
   UPB_CRR,              // Obsolete 3-phase interface
   USB_UPB_PIM,          // USB PIM
   VSP_UPB_PIM,          // USB PIM via virtual serial port
   NETWORK_UPB_PIM,      // PIM-IP version 1
   WMT_RUC,              // Web mountain RUC
   NETWORK_UPB_PIM2,     // PIM-IP version 2 (Pulseworx Gateway)
   VSP_T24_TIMER,        // Timer via virtual serial port
   NETWORK_T24_TIMER     // Timer via network gateway
};
case UPB_PIM:
case USB_UPB_PIM:
case VSP_UPB_PIM:
case NETWORK_UPB_PIM:
case WMT_RUC:
   if (m_IsTelnetConnection)
      cbCmdRemove = PacketCheckTelnet(pDecode, cbDecode);
   else if (pPIM->m_InPulseMode)
      cbCmdRemove = PacketCheckPIM_PulseMode(pDecode, cbDecode);
   else
      cbCmdRemove = PacketCheckPIM_CommandMode(pDecode, cbDecode);
   break;

case VSP_T24_TIMER:
   cbCmdRemove = PacketCheckTimer(pDecode, cbDecode);
   break;

case NETWORK_T24_TIMER:
   cbCmdRemove = PacketCheckGateway(pDecode, cbDecode);
   break;

case NETWORK_UPB_PIM2:
   cbCmdRemove = PacketCheckGateway(pDecode, cbDecode);
   break;

What is your ability to patch your system if I did provide some code to test? It is possible that something reasonably simple could be done (still looking). It would required for you to change out one of the files in the UPB library. That’s easy to do when running on a PC/Mac but more challenging when running on a Pi. I believe that @123 has done something like that and might be able to give some direction (I do all dev work on a Mac and don’t have to patch files).

I’m running HASS on a Pi and accessing remotely via a Mac. With some guidance I’m sure I can swap a file out. If @123 can provide some guidance I can figure it out.

Python virtual environment or docker?

To be honest, I don’t know. I installed HASS from a downloaded image for a Pi 4 and didn’t think anything more about.

Got a bit busy yesterday with cleaning up config flow for one of the home assistant integrations I maintain so I didn’t yet have a chance to get a pulseworx gateway client unwrapper working yet, I have remote access set up to @batwater’s gateway along with a dev environment that I can use to test stuff if you have something already. Going to try and work on a protocol unwrapper myself shortly otherwise.

1 Like

Based on your description, I believe you installed Home Assistant OS.

It’s distributed as a disk image (for specific hardware platforms, such as Raspberry Pi) containing Home Assistant Core, Supervisor, and an operating system (and a few other things).

I just pushed some code to a gateway branch. Untested. The code is to wrap/unwrap PIM commands with gateway wrappers.

Missing is login logic and something to turn on the gateway mode in the library. It’s a start.

This will require upstream HASS code changes to support turning on “Gateway mode” and to supple userid/password is auth is being used.

Here is a link to the branch. Totally untested, just in case you missed that above :wink: Actually I did a small test to see if regular PIM stuff works. It appears that it does. https://github.com/gwww/upb-lib/tree/gateway

Ah, cool, that’s the part I hadn’t looked at extensively, I’ve worked out the auth logic already in my upbshark proxy so shouldn’t be much trouble creating the client auth capability from that. I’ve been refactoring my upb pulse library to split out the different protocol backends so that I can add gateway support without interfering with the normal TCP socket protocol.

I’ll take another look at your auth code to see how to refactor it into upb-lib.

If you could take a look at my change for the gateway that would be helpful. I’m working off a pretty poor document without examples. Curious, are all packets null terminated? And within, when there’s PIM packet, is it a full PIM packet the PU header, content, and PIM checksum?