New Insteon PLM modem integration option via MQTT

I wanted to share a new project I’ve been working on. I’ve written an Insteon PLM <-> MQTT bridge server which converts Insteon messages to and from MQTT. This allows you to command changes to Insteon devices and receive notifications about state changes from those devices using MQTT. Integration with Home Assistant is handled using standard MQTT components like MQTT JSON light, MQTT switch, and MQTT binary sensor.

You can find the repository here: https://github.com/TD22057/insteon-mqtt

The library currently supports Insteon switches, dimmers, FanLinc, IOLinc, KeypadLinc, leak sensors, motion sensors, mini-remotes, outlets, smoke bridge, thermostats. Feel free to request new devices - they should be easy to add and I can iterate with you to test them out.

My motivation for writing this was that none of the existing Insteon solutions had the features that I wanted. One of the things this package does is add in the management of the Insteon all link database (scenes) on the devices the way Insteon-Terminal does. That allows complicated devices (like Smoke Bridge) to be easily configured and managed. It also means that the server knows about scenes that you create on the devices. Once the databases for each device have been read, the server will publish state changes when a scene is triggered for all the devices in the scene. So if a motion sensor causes two lights to turn on, the system knows that and sends an MQTT state update message for the motion sensor and both lights when ever the sensor trips. Future versions of the server will allow you to write the scenes in a configuration file and push that out to all the devices (like MisterHouse does).

I think this device database management and persistent storage system would be hard to implement inside Home Assistant which is part of the reason I made this a separate server process. I also like the idea of moving everything that I use through MQTT as it allows me to use any kind of home automation system and makes it easy to interface with lots of other systems.

Keep in mind this is an initial release. There is documentation for installing and using the system and the supplied example configuration file has documentation in it for how to set up Home Assistant for each component. The examples in that configuration have the MQTT topic and payload templates already written for talking with Home Assistant (though you can change them to support any style of MQTT commands you want).

Feel free to try it out, suggest new features (either here or on github), or report bugs. I’m still finishing code commenting and unit testing so I wouldn’t be surprised if there are some bugs still present.

12 Likes

Thank you for this I will give it a try and would definitely be interested in seeing the fanlinc device supported and can provide any log debugs or testing if needed.

@masterdka Sounds good. I added an issue to add fanlinc support - why don’t you reply/subscribe to that and we can coordinate testing of it. I should have time to add it this weekend.

@TD22057 Thanks I will check in with you on the github issue page :grinning:

I’m having a hard time getting setup. The logs look good but when I try and send a command I get an error “Reply timed out”.

root@9808f08b2f0a:/# insteon-mqtt start /config/insteon-mqtt.yaml
2017-12-10 17:28:16 INFO Modem: Loading configuration data
2017-12-10 17:28:16 INFO Modem: Modem address set to 39.57.0a
2017-12-10 17:28:16 INFO Modem: Modem 39.57.0a database loaded 0 entries
2017-12-10 17:28:16 INFO Modem: Created dimmer at 2e.4d.f9 ‘tree’
2017-12-10 17:28:17 INFO Serial: Serial device opened /dev/insteon
2017-12-10 17:28:17 INFO Mqtt: MQTT device opened 10.0.0.11 1883

root@9808f08b2f0a:/#  insteon-mqtt refresh-all /config/insteon-mqtt.yaml
Reply timed out
root@9808f08b2f0a:/# insteon-mqtt on -l 10 /config/insteon-mqtt.yaml 2e.4d.f9
Reply timed out

I’m also very confused about how to setup a switch in home-assistant to work with MQTT.

@larizzo That’s expected now. I haven’t finished the reply system for the command line tool so it doesn’t get any status messages back yet. Look in the config.yaml file mqtt section, it includes examples on how to to configure the HA inputs to match up w/ the MQTT settings for each device in that file.

Thanks I got it working! Was having trouble and just assumed that error message was relevant. Can’t believe how fast it responds to state changes.

Also I made a docker container, let me know if you want I can do a PR and you could include it in your code folder.

Great! Adding docker sounds like a good plan. How hard is it to keep it up to date?

FYI - the reason it’s so fast is that it’s what I would call “optimistic”. Because it downloads the all link database for every device, it knows when it sees a group broadcast what state every device will end up in when they receive the same broadcast. So while the devices in the scene are updating, the bridge is already sending out the state update messages. Insteon device don’t report state changes when a scene is triggered so the only other way to handle this is to poll them all when the scene is triggered. But that’s slow and can cause weird behavior if you start triggering a lot of broadcasts in quick succession because the modem seems to get overloaded. This way is much better, but it assumes that your Insteon network is reliable and that devices actually respond the way they are supposed to.

In the long run, I might implement a polling algorithm so that when a device is triggered a part of scene, it it gets added to a “poll some time in the future” queue so that if a device somehow doesn’t get the message, the system won’t be in the incorrect state for very long. But figuring out when it’s ok to do that is kind of a pain so I haven’t tried yet. Another thought is to do a nightly poll of all devices to make sure states are current and aren’t out of sync.

FYI - FanLinc support has been added in the dev branch (thanks to @masterdka) for testing it. I’ve also added code so that when you trigger a battery powered device, it will download the database automatically if it doesn’t have it yet. This way, you can add the battery device to the config, restart the server, and trip the device (assuming that it’s been linked to the modem already) either by pushing a button on the remote, or tripping the sensor. Since the device is awake, it should respond to the db request and it will get downloaded (seems to work 80-90% of the time for my devices). These are both on the dev branch (not the main) yet. I’m working on a reply system for the command line tool now - when that’s done, I’ll update to v0.6 and push that to the main branch.

1 Like

First many thanks to @TD22057 this is the solution that I have been looking for. I have had a number of different insteon Home automation products over the years and this one I really like (coupled with home assistant). I have been watching in the background for a little while and I like what you have been able to accomplish in what seems like such a short period of time.
Also, the docker sounds good over here too.

Here are a few questions for the community:

  1. How do I send the buttons from a (6 button) keypad to home assistant? (obviously one set is already handled on/off)

  2. Is there any reason that my Access points (#2443) would need to be linked to the modem? they were with the hub.

hope this makes sense or at least a little.

ThanX,
Michael

@mikew

  1. There is an issue here for key pad support. Feel free to subscribe there to help w/ testing and get info about when it’s available. I think it looks similar to the mini remotes (uses groups 1-8) but where group 1 is a dimmer. I can take a crack at putting it in over the next couple of days
  1. Nope - those are just repeaters as far as I can see and don’t generate or respond to any commands.

FYI the dev branch in the github repo now has support for leak sensors, door and window sensors (pretty much any battery powered sensor), and KeypadLinc’s now. Feel free to give it a try and let me know if you have any issues (I don’t own those devices so I’d love to hear from anyone who can help test them).

1 Like

Nice documentation. I am not sure if this is possible but can I install this onto an RPI 3 that is currently running HASS.io? The reason I ask is when I tried to run a sudo apt command after logging into RPI via SSH it did not let me do anything. I currently have a decent size insteon setup in my home being ran thru my ISY but would like to move it all over to hass. This sounds like it would be the most flexible option.

Any help would be appreciated.

I think it has to be added as a custom add-on. There is a tutorial here and more advanced docs here. I don’t run hass.io so I’m afraid I can’t help much with that (though I’d be happy to accept a pull request for the necessary config files, setup scripts, and/or docs.

This is great to see. I unfortunately don’t have the PLM connection and use the hub (2245-222). What would need to be added to connect to hub via IP.

@djryan012 Check the config file here . Give your hub a fixed IP address and put that in the config file and I think it will work. I don’t use a hub but I’m pretty sure the serial system will connect to it. If you find that doesn’t work, open a github issue and I’ll try and take a look.

I think that only works with the old hub. The new one doesn’t have the serial plm. I could be wrong though, I’ll try tonight I have the new hub.

FYI - the dev branch on github has a number of new features. Once I finish getting IOLinc support in, I’ll push this out as a 0.6 release on the main branch. Those features include:

  • Scene simulation topic and payloads. Can now send a message (or hook a HASS switch) to simulate pressing an Insteon button to trigger a scene.
  • Full support for KeypadLinc. Each button LED can be controller or clicked to trigger it’s scene.
  • Software based linking - new devices can be paired with the modem purely through software and links for every group (for multi-group items like FanLincs, KeypadLincs, and Smoke bridge) can be automatically created.
  • Multi-group scene creation (link button 3 on device to button 4 on another) is now supported.
2 Likes

Hi all, firstly I’d like to say that @TD22057 what you have done here sounds awesome.

I’ve been putting off committing to either a Z-Wave or Insteon solution for a while as neither has really fitted my needs, in particular I have some light switch panels with 5 circuits fitted on a single AU wallplate. The Insteon 6/8 keypads were always attractive because I could use them with the in-wall relays to solve my problems. However the limited integration with HASS had prevented me pulling the trigger on the purchase so I started looking closer at the Z-Wave range.

TD22057’s MQTT solution might just be the answer to my woes. (I’m a bit excited)

I’m happy to commit to an initial purchase of some Insteon hardware to enable me to play with this but I’m looking for some guidance for you Insteon users. I run HASS on an Ubuntu virtual server running on a Windows Server 2008 R2 host so I am keen to try and connect to the Insteon network via Ethernet rather than having difficulties getting the USB/Serial device presented directly to the VM.

Can anyone recommend the hub/modem/controller combo that would be suitable for my circumstances and allow me to fully utilize TD22057’s solution?

Thanks in advance for any recommendations you can provide.

1 Like

FYI version 0.6.0 is now available on the main branch of the github repo. Here’s a summary of the changes:

  • Battery powered devices will now attempt to download the all link database if it doesn’t exist locally when a broadcast message from that device is seen (since the device is awake). This way you can trip a battery device (motino sensor or push a remote button) and it will initiate a database download.
  • Added FanLinc support (Issue #4). Thanks to @masterdka for testing.
  • Added battery powered sensors (door and window sensors) support (Issue #5).
  • Added leak sensor support (Issue #6).
  • Added KeypadLinc support (Issue #8).
  • Added IOLinc support (Issue #12).
  • Added reply system to the command line tool. Messages are now sent from the server back to the command line tool to indicate the result of running the command.
  • Added !include tag support to the yaml loader which reads the configuration file to allow the file to be split into multiple files (Issue #11).
  • Added linking command to put a modem/device into linking mode without touching it. This lets a new device be added purely through software commands (Issue #7).
  • Updated the modem db delete commands so that specific records can be removed. Removed the modem only db_delete command and replaced it with db_del_ctrl_of and db_del_resp_of just like the devices.
  • Low level MQTT commands now accept “nice” names from the config file for all inputs including the topic. “modem” is the nice name for the PLM modem.
  • Added support for simulating button presses on devices to trigger device scenes (Issue #9).
  • Added support for triggering virtual scenes defined on the modem. (Issue #24).
  • Added support for creating multi-group controller/responder links to allow linking between different buttons on multi-button devices. (Issue #20).

Did you get your 2014 newer hub to work??? I setup my config.yaml file but not successful connecting for me. Got a connection time out error

1 Like