New Insteon PLM modem integration option via MQTT

After crashing Hassio and rebuildnig HA as a Hassbian install. I tried again with the same failure on the insteon_mqtt side. I was able to enable $sys on the mosquitto broker and the logs showed nothing with less than 60sec, granted I would still consider myself a noob and really trying to understand why all my other MQTT devices work, but struggling to get your amazing code to work.

2018-07-10 16:13:50,300  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/messages/received
2018-07-10 16:13:50,300  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/messages/sent
2018-07-10 16:13:50,300  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/publish/messages/received
2018-07-10 16:13:50,300  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/publish/messages/sent
2018-07-10 16:13:50,300  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/bytes/received
2018-07-10 16:13:50,300  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/bytes/sent
2018-07-10 16:14:01,314  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/uptime
2018-07-10 16:14:01,324  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/messages/received/1min
2018-07-10 16:14:01,328  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/messages/sent/1min
2018-07-10 16:14:01,328  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/publish/received/1min
2018-07-10 16:14:01,328  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/publish/sent/1min
2018-07-10 16:14:01,329  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/bytes/received/1min
2018-07-10 16:14:01,329  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/bytes/sent/1min
2018-07-10 16:14:01,329  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/sockets/1min
2018-07-10 16:14:01,329  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/connections/1min
2018-07-10 16:14:01,329  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/messages/received/5min
2018-07-10 16:14:01,330  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/messages/sent/5min
2018-07-10 16:14:01,330  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/publish/received/5min
2018-07-10 16:14:01,330  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/publish/sent/5min
2018-07-10 16:14:01,330  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/bytes/received/5min
2018-07-10 16:14:01,330  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/bytes/sent/5min
2018-07-10 16:14:01,330  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/sockets/5min
2018-07-10 16:14:01,331  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/connections/5min
2018-07-10 16:14:01,331  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/messages/received/15min
2018-07-10 16:14:01,331  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/messages/sent/15min
2018-07-10 16:14:01,331  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/publish/received/15min
2018-07-10 16:14:01,331  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/publish/sent/15min
2018-07-10 16:14:01,331  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/bytes/received/15min
2018-07-10 16:14:01,331  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/bytes/sent/15min
2018-07-10 16:14:01,332  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/sockets/15min
2018-07-10 16:14:01,332  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/load/connections/15min
2018-07-10 16:14:01,337  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/messages/received
2018-07-10 16:14:01,337  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/messages/sent
2018-07-10 16:14:01,337  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/bytes/received
2018-07-10 16:14:01,337  INFO --- MqttFX ClientModel             : messageArrived() with topic: $SYS/broker/bytes/sent

Clone the dev branch down instead of main - it has updates to stop this from happening. Or you can wait until I merge it over to main and another release. FYI it doesn’t really hurt anything to disconnect - the MQTT client will auto-reconnect.

Was trying to get start with this project again but having trouble with the install. When I issue the pip3 install . when try to collect pyyaml I get 404 errors when trying to collect. I can confirm when browsing to the site I see the directory but the files do not exist.

Collecting pyyaml>=3 (from insteon-mqtt==0.6.2)
  HTTP error 404 while getting https://www.piwheels.org/simple/pyyaml/PyYAML-4.1-cp35-cp35m-linux_armv7l.whl#sha256=a5b125a24244830d574ccfd8fab861198d2ffe491b73179ffd75abf78caee6ff (from https://www.piwheels.org/simple/pyyaml/)
  Could not install requirement pyyaml>=3 from https://www.piwheels.org/simple/pyyaml/PyYAML-4.1-cp35-cp35m-linux_armv7l.whl#sha256=a5b125a24244830d574ccfd8fab861198d2ffe491b73179ffd75abf78caee6ff (from insteon-mqtt==0.6.2) because of error 404 Client Error: Not Found for url: https://www.piwheels.org/simple/pyyaml/PyYAML-4.1-cp35-cp35m-linux_armv7l.whl
Could not install requirement pyyaml>=3 from https://www.piwheels.org/simple/pyyaml/PyYAML-4.1-cp35-cp35m-linux_armv7l.whl#sha256=a5b125a24244830d574ccfd8fab861198d2ffe491b73179ffd75abf78caee6ff (from insteon-mqtt==0.6.2) because of HTTP error 404 Client Error: Not Found for url: https://www.piwheels.org/simple/pyyaml/PyYAML-4.1-cp35-cp35m-linux_armv7l.whl for URL https://www.piwheels.org/simple/pyyaml/PyYAML-4.1-cp35-cp35m-linux_armv7l.whl#sha256=a5b125a24244830d574ccfd8fab861198d2ffe491b73179ffd75abf78caee6ff (from https://www.piwheels.org/simple/pyyaml/)

@masterdka No idea. That looks like someone messed up the pyyaml installs. It looks like the file is there (it’s in the dir listing) but it’s not actually there. You could try asking for beta’s which are there (I’m guessing there is a pip flag for that somewhere).

@masterdka See: https://github.com/yaml/pyyaml/issues/201. Looks like forcing it to install just the 3.13 version is the solution.

Yes thank you followed the thread to force install the 3.13 version with pip install PyYAML==3.13 and was able to install successfully thank you!

I’m currently playing with the Hassio add-on in this PR: https://github.com/TD22057/insteon-mqtt/pull/76 and it’s been working great so far. I’ve just been using it with Node-RED but so far so good. Plan to replace my Insteon Hub install with this. I also like the idea of this running as a docker container.

I’ve pushed the development branch to the master for a 0.6.3 release. This includes the update to support Hassio. Also note that because of the support for on/off keypadlinc’s, the config file input for keypadlinc’s in general was changed. See the new sample config.yaml for details.

Breaking Changes

  • KeypadLinc now supports both dimmer and on/off device types. This required changing the KeypadLinc inputs in the MQTT portion of the config.yaml file. See the file in the repository for the new input fields. (Issue #33).

Additions

  • HassIO and docker support and is now available. See the main doc page for a link to the docs (thanks @lnr0626) (Issue #63, PR #76).
  • Added on/off outlet support (Issue #48).
  • Added support for older I1 devices (thanks @krkeegan). This should allow refresh command and database manipulation for 5+ year old devices. (Issue #17).
  • Added support for automatic maximum hop computations in messages. This should reduce delays and load on the insteon network (thanks @krkeegan). (Issue #43).
  • Added support for waiting to write messages until the maximum number of hops have elapsed. This should reduce subtle bugs where the Insteon network could get overloaded by sending messages to quickly. (thanks @krkeegan). (Issue #45]I45).
1 Like

I’ve upgrade to 0.6.3 and ran a few tests, everything looks good so far. Thanks for the continued development!

I’ve not seen the little RF only USB dongle (2448a7) mentioned, but wanted to confirm it does appear to work fine. I’ve had no issues getting it to work. I am having to use the linking direction here to link stuff with it as it doesn’t have a link button, but that’s not a big deal.

I’ve actually not messed with this in a long while, was waiting on the auth fixes to go in and then just sorta went off working on other things. The recent updates to the main Insteon components though have brought me back since I’ve got to rebuild the configs for all my lights anyways (for what feels like the 3rd time this year). So I figured I’d mess with this again and I’m pretty pleased, it seems to be working good. I’ve not added anything to HA yet, but just using the commands works well.

One thing I’ve noticed though is changes to states aren’t reflected if I use the hub. I still use the Insteon app and Alexa, both of which use the hub (I have a newer model hub v2). If I use either Alexa or the app to turn a light on/off, that change doesn’t seem to ever get reflected in MQTT. Only state changes initiated by physically flipping the switch or using the insteon-mqtt command line tools are reflected in MQTT. Is there anything that might get it to do an update on state change via the hub? Or, is maybe there a way to add a command to just update the state of devices? The refresh command appears to work, but it grabs a lot more than just the state, so it takes a bit of time. If that were doable, I could just cron the state refresh until I eventually drop the Insteon app and integrate Alexa with HA bypassing the hub.

Sorry. The hub has a plm model inside it but insteon-mqtt can’t talk to it so there is no way to get updates. The whole system is designed to work by talking to the plm model to get the updates and using the hub makes that impossible.

Thanks, was afraid that would be the case. Was really hoping the hub and PLM would be able to work side by side with no weirdness. I had hoped the lights themselves would broadcast a state change out to everything in their table including the PLM. I actually even tried linking the PLM and hub, but that doesn’t seem to have worked.

Oh well, I’ll finish setting up and get it working good with HA and then I’ll do the Alexa integration. Obviously this is far more reliable than using the Insteon component for HA so the buttons in HA will actually be useful and I can stop using the Insteon app. Those are the only two things the hub serves a purpose for that I can think of for me, Alexa and app control.

Somewhat related, what’s the proper way to remove a device :blush:
Just delete it out of the config file?

FYI I’ve pushed 0.6.4 to the master branch on github. This includes a fix for a bug introduced in 0.6.3 that would cause output messages to Insteon devices to be lost if multiple messages were sent in rapid succession (like in a refresh-all). FYI I haven’t been able to get my local docker to work properly so I can’t build/update the hassio files.

Additions

  • Added on_level flag support for dimmers and KeyPadLinc’s to set the light
    level when the on button is pressed ([Issue #70][I70]).

Fixes

  • Multiple output messages queued to the Insteon devices causes some messages
    to be lost ([Issue #86][I86]).

why would i use MQTT over the home assistant module for Insteon PLM? do i no longer need an ISY device?

My first post here. A novice. I have HA 0.81.2 working with the current Insteon component using a 2414U PLM. I also have a 2245 Hub2 working with Alexa and the Insteon App. The Hub2 is from my start with Insteon about 2 years ago. The PLM and HA (on a P12B with Hassbian) is recent and I intend it to be the base for some future automations.

Like lizaoreo has posted above I would like to use both the PLM and the HUB2 concurrently. My reasons are:
(1) I have them both. (2) the Hub2/Alexa/Insteon setup has been stable and reliable, (3) Using both avoids a single point of failure at little extra hardware cost. The behaviour of the two is also just as [lizaoreo](https://community.home-assistant.io/u/lizaoreo) described. The PLM side does not see device changes done by the HUB2 side. This is confirmed by the HA Insteon component docs as follows:

    You cannot use both Home Assistant and the INSTEON app. If you do, the changes made in the app will not appear in Home Assistant. Changes made in Home Assistant will appear in the app after a period of time, however.

I have checked and the Insteon app does see changes made via HA/PLM, but not the reverse Seems to me the difference in behaviour here has to be a coding/design choice, not a fundamental defect. Both see all Insteon message traffic yet what they do with it is slightly different.

The HomeSeer product has a forum thread on this issue HERE

It seems the 3rd party Insteon plug-in has a configuration option that may provide the desired symmetrical behaviour.

I would like some feedback. Could a config option be added so HA Insteon ?

Wes

@Wes: This isn’t a thread for the HA Insteon component - please check the original post if you’re interested in what this thread is about. You should post your questsion over here: Insteon_PLM on Hass.io, Anyone have it working?

@david1: It’s about modularity and features - some of this is explained in the first few posts of the thread. When I started this project, the HA Insteon component didn’t really work and wasn’t being maintained. Rather than build a new one or try to take that one over, I decided to go this route. Having a separate service allows for a lot more control of the Insteon devices, makes it easier to test, and allows it to work with any home automation system. A separate service allows for commands that HA doesn’t know anything about - this tool allows you to create and modify Insteon scenes on the devices, query the device databases, and modify internal Insteon flags that control behavior. In the future, it will able to back up and restore all of the Insteon scenes and allow for restoring devices that fail and are replaced.

Thanks for the reply. It was easier to get HA going for me using the built in Insteon component, but your MQTT may be better for me longer term. Thus the question about the possibility of “symmetrical” two controller operation of an Insteon network using your option.

If it is possible, I think the best argument for doing it is to eliminate a single point of failure. The cost of two controllers is small compared to the cost of 20 or so various Insteon devices.

Somewhat related is the “feature” of gradually polling to correct any “out of sync” conditions caused by lost network messages (noise) or another controller.

Finally, I saw a You-Tube video about weak Insteon protocol security. Listening for ACKs from a linked device but to an unknown controller (rather than a configured other controller) would be a good security check.

Wes

@Wes I have created a new thread regarding the Hub and PLM coexisting. Please feel free to add your comments. I would like to start this discussion to see how much interest there is in this feature. I believe you are the second person to request it.
Insteon PLM and Hub coexist

FYI - HA 0.84 made a breaking change to the MQTT JSON API. When you update to >= 0.84, you’ll need to change your HA config file for any dimmers that you defined.

Old way:

light:
   platform: mqtt_json
   ...

New way:

light:
   platform: mqtt
   schema: json
   ...

I’ll update the docs and example file in the repo later this weekend.

FYI - Version 0.6.5 has been released. This includes initial support for Insteon thermostats, fast on/off messages, and manual mode messages.

Additions

  • Initial support for Insteon thermostats has been added (thanks @krkeegan).
  • Support for fast on and off commands and reporting has been added. The on/off mode (normal, fast, instant) is now a command input as well as an output flag in the state templates. This allows double-clicking of switches to be used in automation triggers (thanks @NickWaterton). (Issue #66).
  • Added support for manual mode state reporting (holding buttons down). Supported by dimmer, keypadlinc, remote, and switch (thanks @NickWaterton). (Issue #104).
  • New command ‘get_model’ added to the command line tool to retrieve and save the Insteon device cat, sub_cat, and firmware revision (thanks @krkeegan). (Issue #55).
  • New command ‘join’ added to the command line tool to perform a two-way pairing and refresh to link a device to the modem. This combines the previous linknig and pair command into a single command (thanks @krkeegan). (Issue #97).
  • New improved NAK error response codes makes it easier to understand errors when the devices can’t communicate (thanks @krkeegan). (Issue #95).
  • Added a new command line input (get-devices) to get a list of the curren Insteon devices in JSON format. (Issue #84).
  • Added a new command line input to factory reset the modem.

Fixes

  • Added better messages when the pair command fails because the modem db is out of date (Issue #69).
  • Updated docs and example config file because of breaking HomeAssistant MQTT light change for dimmers (Issue #88).
1 Like