Separately the last 2 or 3 weeks I’ve been playing with creating a component / set of platforms for Elk integration as well, but directly using either RS-232 or Ethernet using Elk M1XEP (since the M1XEP just gives you serial ethernet it’s basically no difference using PySerial, you just pass a socket://IP:port rather than /dev/device).
My time is sporadic and I’m new to HASS development and this is my first “real” Python project as well, so it could be weeks or months before something truly functional exists.
Once I have a sane / functional real time update callback from ‘PyElk’ into the HASS component/platform side of things, it should be all downhill from there (though some features may be long or never in coming,at least from myself, because I can’t test them such as the various lighting system integrations, as we lack the necessary hardware)
Hi there. @stipex and I managed to get his Elk M1 working with HA. We ended up using the MQTT Alarm Control Panel in HA. Behind the scenes, we wrote a small Node.js app that sends/receives MQTT messages, and wraps the following Node.js module (which handles the ‘real’ communication with the Elk M1):
I ended up forking the project to add more coverage of the Elk M1 protocol.
Below is what @stipex has in his HA front end. The garage roller door, and front door, are actually control outputs on the Elk M1. The alarm and sensors/zones speak for themselves. The lights (with dimmers) are also handled by the Elk M1. As far as HA is concerned, the alarm control panel is just a generic MQTT device, so HA has no knowledge about anything Elk specific as such.
What name are you using for your component/platforms? I want to make sure I don’t use the same name (in case they both someday get rolled up into being officially part of HASS)
Existing platforms/components from HA are being used. Examples shown below. As you can see, nothing really specific to Elk M1. It;'s really the Node.js app that does the mapping between these MQTT messages and the physical panel itself.
Well, I’ve got outputs working (both following changes from the Elk as well as being able to turn them on and off from within HASS) and zones (following changes from the Elk), so things should be generally down hill from here (time permitting).
One caveat is that it takes 30~60s at startup to get all the definitions/states/descriptions/etc of everything (and I’m only getting the zone/output/area/keypad data so far, not loading counters, custom settings, and such yet). Right now I’m just blocking the start up of HASS (which is fine while developing) but I imagine this would be frowned upon generally. If I continue loading all of this on a separate thread while HASS finishes starting and I’m still populating devices, this should be okay since HASS supports adding and removing devices dynamically, right? The only concern would be if someone’s automation failed due to a device not being loaded yet?
Also, in addition to spinning off another thread to finish loading things, I could be using async coroutines, though this will require some rewrites of the PyElk side of things - this is probably where I will end up though I am a bit concerned about the state of PySerial.asyncio being “experimental”. I might be able to cheat and use the non-asyncio routines, since all my calls will either send a short burst of data or only receive data if there’s data to receive, essentially operating in a non-blocking-esque fashion, even if using the regular non-asyncio PySerial.
But whether another thread or coroutines, I shouldn’t be blocking the startup of HASS, and any issues (automation fails) due to things not existing yet after startup are on the end user to deal with?
For anyone interested: https://github.com/BioSehnsucht/ha-elkm1 (the HASS components, I don’t think it will work in custom components, you will have to actually place in (or symlink in) to the homeassistant/components/… directories) https://github.com/BioSehnsucht/pyelk (the Elk ASCII library interface, which the HASS components make use of)
Wow, this looks like really great work, and your Python looks much prettier than mine does
I started at the other end with some code that parses most of the useful Elk messages, with the intention of wiring it up to MQTT. I have some things in mind where integrating with MQTT might be useful, removing a dependency on HASS. But this looks so pretty, in particular, being able to perform discovery an populate HASS with the discovered zones and such… I’m not sure how to implement discovery with an MQTT interface; probably still need some sort of custom component to do that.
One of the things that I want to be able to do is invoke Elk M1 “Automation Tasks”. Not because of a desire to do anything clever, but because I can’t find another way (using the serial protocol) to “chirp” the alarm speakers. I have a scenario where this would be useful.
So we’ve gone from zero Elk M1 integrations to 2 (and another half for my unfinished version). Very nice!
If you just want to use MQTT @lmamakos, then I think @pkozul has already covered that base (using a modified version of Node.JS Elkington, and then using HASS’s existing MQTT alarm panel / etc support).
On the other hand, there’s probably some way of publishing things from HASS into a MQTT if you still need MQTT for something but wanted to go the more directly integrated route for the Elk with HASS (rather than using the MQTT solution he already engineered).
If you want something that works today, then pkozul can probably hook you up. It’s probably going to be a week or two at least before my work is “polished” enough to use for even basic zone/output/arm/disarm, never mind the fancier features of the Elk.
@BioSehnsucht, I’ve installed the latest version of your stuff from github, and it seems to be working for me. Just some thoughts…
First, one curious result from having some other software generating queries to the Elk for temperature measurements from a zone temperature sensor, two keypads and a thermostat results in periodic errors like:
Since there’s nothing looking for these message types, they never get removed from the queue, and eventually they timeout for being too old. This might be deliberate depending on what you had in mind…
This sort of begs a question for how to deal with zones that return temperature; I’ll have to see if there’s a non-violent way to introduce that.
As suggested, I symlink’ed the various files alongside the existing components. I didn’t try to treat it as custom components. I’m going to play with this a bit further before pursuing the code I started. I did a bottoms-up approach of just parsing the various elk messages, while you’ve modeled various aspects of the stuff connected to the elk, which seems like a fairly clean way to approach this…
Thanks for code, and the great work! I’m not sure what your plans are, but I for my needs I would like to be able to control the thermostat that I have connected to the Elk. Being relatively new to HASS, I’m not sure of what the conventions are here; in some cases, I’ve seen thermostats exposed as “sensors” for the current temperature ; and also as “climate” devices with a bunch of attributes, including the current temperature.
@lmamakos Yeah, I queue (technically using a deque) all incoming messages and anything implemented eventually gets removed as it is processed. Eventually all messages will be handled (even if only to throw them away like I do for XK ethernet test) the timeout to remove unhandled will be shorter, and the message will be debug instead of error, so you shouldn’t see them. But for now, it’s useful debugging information for me while I’m working on it
I plan to implement zone temps, just haven’t gotten to it (in part because I don’t currently have any such zones to test with, although I may have unearthed an elk temp probe intended for use with a zone input a few days ago - if that’s what it is I’ll wire it up when I get a chance and use that to develop / test that). I could probably implement untested support pretty easily actually… I’ll need to make some changes so the HASS sensor/elkm1 component can return different unit types and implement the messages etc in PyElk.
Thermostats may be harder to implement blindly than temp zones, but it’s also on the list, just further down it since it’s neither low hanging fruit nor something I can directly test
I also plan to figure out a way to log who arms/disarms/etc and correspondingly allow that to trigger things in HASS, though I haven’t started figuring out the HASS side of that yet much less implemented handling the corresponding messages in PyElk yet.
I wanted to add some features / extra details to the ‘more info’ on the alarm panel badge in HASS but this Polymer stuff is greek to me, that may have to wait. Want to be able to reflect “arm up” (readiness to arm - i.e. whether a zone is violated) status among other things. Doesn’t help that making changes to the UI stuff requires more hoop jumping (building) to test (although perhaps I can get away with the ‘custom UI’ in config dir stuff to test it).
@lmamakos
I may almost have temperature reading working, it’s a bit hacky and I need to clean it up before I commit it, but time permitting I should get it comitted to Github tonight.
I still don’t have any actual temp devices wired up, so I’ve only tested by injecting some fake ST packets into the queue and manually hard coding zones to be treated as temp zones (overriding what was detected from the Elk), so I can’t guarantee it will work for real, but hopefully it will …
@lmamakos If you want to try it out, both zone 1-16 temperature and keypad temperature should be supported now. Adding thermostat temperature reporting is dependent on getting thermostats put in, which I might do soon (I’ll probably both have just the reported temperature reported as a sensor you can choose to hide, as well as a thermostat entity complete with whatever functionality can be implemented).
I think the next thing I’m going to do is a minor rewrite to try and condense some of the duplicated code and such, unless you find any bugs related to the temperature stuff. Want to get things a bit more cleaned up before the job starts to get unmanageable.
Just tried the new code. My “zone” based temperature sensor is discovered, and shows up as a sensor, however the temperature is off (shows as -28 and should be about 72 degrees).
sensor.elk_temp_z_15 -28 unit_of_measurement: °F
icon: mdi:thermometer-lines
State: Unconfigured
Alarm: Disabled
friendly_name: Basement temp
Status: Normal
Definition: Temperature
I have 3 keypads; only two (2 and 3) have temperature sensors. Both of those also show up as incorrect as well. Also, all of the keypad sensor entities are marked “hidden”. The temperatures should be about 77 F
I will poke around a bit and see if I can see anything obvious.
If you’re taking suggestions, it would be nice to avoid creating the extra keypad entities; perhaps only for do so if there are names for the keypads in the Elk? LIkewise, it would be nice to fetch the name labels and maybe make them available as attributes or friendly names?
@lmamakos
I found the problem! I was grabbing 2 characters instead of 3 from the data and losing the hundreds column, and since my fake data I was testing with was raw data of less than 100, I didn’t notice. So everything was off by 100.
As for the keypads not even showing up, I don’t know why, unless they’re not assigned to an area? I guess I’ll change the auto-hiding logic to ONLY look for failure to get temp (I set it to -460 aka absolute zero first, and if it’s still showing as less than -100 I assume it’s not reporting temps). I was also checking if the keypad was set to area 0 (aka not set to any area) and hiding it then, but I’ll disable that, the temp check should be enough.
I think another problem is sending (and expecting) a “Requested Group” of 7 rather than 0 or 1 when generating the EVENT_TEMP_REQUEST message when scanning zones and keypads. So something like this, assuming understand what’s going on…
I’ve tried tweaking some of this, including making the change
diff --git a/PyElk/Keypad/__init__.py b/PyElk/Keypad/__init__.py
index 59a5de4..7b1be22 100644
--- a/PyElk/Keypad/__init__.py
+++ b/PyElk/Keypad/__init__.py
@@ -123,7 +123,7 @@ class Keypad(object):
PyElk.Event.Event.EVENT_TEMP_REQUEST_REPLY
"""
def unpack_event_temp_request_reply(self, event):
- data = int(event._data_str[4:6])
+ data = int(event._data_str[3:6])
data = data - 40
self._temp = data
self._updated_at = event._time
diff --git a/PyElk/Zone/__init__.py b/PyElk/Zone/__init__.py
index 15014cd..cdd628b 100644
--- a/PyElk/Zone/__init__.py
+++ b/PyElk/Zone/__init__.py
@@ -269,7 +269,7 @@ class Zone(object):
PyElk.Event.Event.EVENT_TEMP_REQUEST_REPLY
"""
def unpack_event_temp_request_reply(self, event):
- data = int(event._data_str[4:6])
+ data = int(event._data_str[3:6])
data = data - 60
self._temp = data
self._updated_at = event._time
But something else is acting squirrely… I’m not sure if it’s an environment problem on my end or not. I think I need to set up another HASS instance to experiment with so I can turn up logging and see what’s going on. I have the logging level configured to exclude routine messages since I get many events per second pumped through due to power measurements being taken every second.
@lmamakos
Oops, yeah that should be group 1 in the keypad section. I (theoretically) look for the group 0 (zone) / 1 (keypad) response plus group 7, because I found by experimentation (I didn’t see it documented) that the Elk responds with group 7 if you request a non-existent device. I guess I accidentally put the 7 in the wrong place. Only the event._data_str = '7' + format(k,'02') line should need to be changed to ‘1’. The second occurance is intentional (as I am scanning for events that match either what I asked for, or for the group 7 - though the previous error would make it only look for group 7 responses).
This part being wrong only breaks the initial scanning, and likely is what caused keypads to be hidden (since they would be -460F when being added to HASS).
Try it with just line 470 changed and see if that seems to work better.
I’ve updated both repos accordingly. I can’t test it directly of course…
Both the fix for [4:6] -> [3:6] and the fix for requesting group 7 when it should be 1 are in the latest commits / version on PyPi. You should be able to just update the files from the ha-elkm1 repo and HASS should download the newest version of PyElk from PyPi.
I’m just running PyElk-test.py now to see what happens…
Would it make sense to remove EVENT_TEMP_REQUEST_REPLY from event_scan_map? While it’s in there, the ST replies seem like they’re ignored. When I remove it, I get something like this at the end of a run. (Zones may be violated during this as there might be someone walking around the house…)
DEBUG:PyElk.Elk:elk_queue_process - checking events
DEBUG:PyElk.Elk:elk_queue_process - rescan in progress, skipping: 'ZC'
DEBUG:PyElk.Elk:elk_queue_process - rescan in progress, skipping: 'AS'
DEBUG:PyElk.Elk:elk_queue_process - rescan in progress, skipping: 'ZC'
DEBUG:PyElk.Elk:elk_queue_process - rescan in progress, skipping: 'AS'
DEBUG:PyElk.Elk:handle_line: 1EAS000000001111111100000000000E
DEBUG:PyElk.Elk:elk_queue_process - checking events
DEBUG:PyElk.Elk:elk_queue_process - rescan in progress, skipping: 'ZC'
DEBUG:PyElk.Elk:elk_queue_process - rescan in progress, skipping: 'AS'
DEBUG:PyElk.Elk:elk_queue_process - rescan in progress, skipping: 'ZC'
DEBUG:PyElk.Elk:elk_queue_process - rescan in progress, skipping: 'AS'
DEBUG:PyElk.Elk:elk_event_scan : timeout
DEBUG:PyElk.Elk:scan_keypads : got Event.EVENT_TEMP_REQUEST_REPLY
Traceback (most recent call last):
File "bin/PyElk-test.py", line 31, in <module>
ELK = PyElk.Elk(address=host, usercode=code, log=_LOGGER)
File "/Users/louie/homeassistant/lib/python3.5/site-packages/PyElk-0.1.1.dev9-py3.5.egg/PyElk/Elk.py", line 170, in __init__
self._rescan()
File "/Users/louie/homeassistant/lib/python3.5/site-packages/PyElk-0.1.1.dev9-py3.5.egg/PyElk/Elk.py", line 196, in _rescan
self.scan_keypads()
File "/Users/louie/homeassistant/lib/python3.5/site-packages/PyElk-0.1.1.dev9-py3.5.egg/PyElk/Elk.py", line 475, in scan_keypads
group = int(temp_reply._data[0])
AttributeError: 'bool' object has no attribute '_data'
Sorry for what appears to be random observations/guessing; trying to wrap my head around what’s going on inside.
Events in the (poorly named) event_scan_map should only be ignored if the scan process is running. This normally only happens during startup, and when the installer exit event occurs (exiting programming/installer mode on a keypad or disconnecting ElkRP - this triggers another scan process since there’s no way to know what was changed otherwise, if anything). The reason it’s set to ignore them during scan is otherwise you can get timeouts due to elk_queue_process getting to the events first, and now the scan process won’t know what happened and potentially fail (as it appears to have done when you removed it from the event_scan_map). I should probably fail gracefully in that case, but …
It looks like I didn’t put any debug reporting of when it reaches the Event.EVENT_TEMP_REQUEST_REPLY section of the elif chain in elk_queue_process, so it may appear to do nothing if you’re just running the test script. If you want you can try adding a logging line like you see on other elif sections to check if it’s truly reaching that code block when receiving ST events.
