ELK M1 Interface

Third party integrations
74

I did make this change which seems like an error, but still not working well. I stupidly updated hass and now some other weirdness is happening, which may be related to the upgrade… ah well.

diff --git a/light/elkm1.py b/light/elkm1.py
index 21e9b97..38bf26d 100644
--- a/light/elkm1.py
+++ b/light/elkm1.py
@@ -76,7 +76,7 @@ class ElkX10Device(Light):

     def trigger_update(self):
         """Target of PyElk callback."""
-        _LOGGER.debug('Triggering auto update of X10 ' + device.HOUSE_STR[device._house] + ' ' + str(self._device._number))
+        _LOGGER.error('Triggering auto update of X10 ' + device.HOUSE_STR[self._device._house] + ' ' + str(self._device._number))
         self.schedule_update_ha_state(True)

Now getting a bunch of errors like:

2017-06-09 00:23:44 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/Users/louie/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity.py", line 204, in async_update_ha_state
    "No entity id specified for entity {}".format(self.name))
homeassistant.exceptions.NoEntitySpecifiedError: No entity id specified for entity elk_zone_030
2017-06-09 00:24:11 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/Users/louie/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity.py", line 204, in async_update_ha_state
    "No entity id specified for entity {}".format(self.name))
homeassistant.exceptions.NoEntitySpecifiedError: No entity id specified for entity elk_zone_015

which I’m not sure are related to the input thread failing or not. Getting too late to pursue this further tonight…

75

@lmamakos
No time to really look at this yet, but good catch on trigger_update. Those references in that method to just device should be self._device, including the HOUSE_STR part.

        _LOGGER.debug('Triggering auto update of X10 ' + self._device.HOUSE_STR[self._device._house] + ' ' + str(self._device._number))
76

@BioSehnsucht - Duh, I should have noticed that. Well, I’ll use the excuse that it was late and the end of a long day.

I did make the last change as suggested, and this got things working again, including what appears to be correct behavior of switches for lights. When other activity (automation on the Elk) turns lamps on or off, their state seems to be successfully updated in hass, too.

I’ll watch this a bit more, but this change seems to have fixed the obvious issue.

77

@lmamakos

I’ve updated Git / etc with that fix for X10.

I’ve also added a new feature, tracking of which user armed/disarmed an area, and also what the last user code entered was for an area (and keypad, but keypad is not currently reflected beyond keypad temps in the HASS side of things)

alarm_control_panel.elk_area_N where N is the area, has the following new attributes:

last_user_code : Last valid user number entered for the area (I may rename this since it’s not actually the code …)
last_user_at : Timestamp in seconds when the code was entered (as you get from time.time() )

last_disarmed_by_user : The user number that most recently disarmed the area
last_disarmed_at : Timestamp (as above) when code was entered

last_armed_by_user : The user number that most recently armed the area
last_armed_at : Timestamp (as above) when code was entered

So potentially you can trigger things based on who armed/disarmed, or merely entered a code (i.e. user code has ‘access’ permissions, such as when using wiegand RFID badges, but didn’t actually arm/disarm an area)

78

In the interest of making things more clear I am thinking of renaming as follows :

last_user_code -> last_valid_user (the last valid user whose code was entered (as the value is the user number, not their access code))
last_user_at -> last_valid_user_at (because it is the last time a valid user code was entered)

last_disarmed_at -> last_disarmed_by_user_at
last_armed_at -> last_armed_by_user_at

I could potentially also change the ..._at attributes to be all in some format other than seconds since epoch, i.e. a human readable date and time, but leaving it as seconds since epoch is probably easier to trigger on (can compare raw values without conversions) ? I could always ad an extra version of each attribute with the human readable version, too …

79

So currently we should have control of outputs and X10 lighting devices, monitoring of input zones, arming/disarming of areas and tracking who armed/disarmed them etc, climate (thermostat) device support…

So to summarize a to do list before trying to get this merged into HASS proper (anyone can feel free to comment something they want that isn’t implemented yet):

  • Handle setting climate set points when in cool or heat mode properly (code was written assuming both could be set at any time, but it turns out HASS ‘more info’ for climate devices doesn’t handle more than a single set point currently, see https://github.com/home-assistant/home-assistant-polymer/issues/236)
  • Support reporting temperatures in Celsius instead of Fahrenheit (Elk is natively only working in F, so that’s all that is currently implemented)
  • Activate tasks (and also allow triggering events in HASS when they are activated naturally)
  • Expose counters and custom settings from the Elk
  • Enhanced configuration support to allow specifying which devices (zones, outputs, lights, … ) should be hidden or not hidden, etc, perhaps even totally ignoring specified ranges/devices (to speed up startup) - I’m not totally sure what this will look like yet
  • Look at enhancing both the HASS code and PyElk to support asyncio operation instead of threaded (this will make startup not stall waiting on the Elk integration, and allow the integration to ‘play nice’ with the rest of HASS in this regard)
  • Test and possibly improve handling of re-scanning devices after exiting installer programming mode / disconnection of ElkRP (right now it just tries to re-run the scan process as is, which might be okay once asyncio is implemented, but currently could cause a long delay while it runs)

Further features that maybe don’t need to be done before becoming part of HASS, or might not get done at all any time soon (I consider these lower priority):

  • Possibly implement Insteon lighting control via serial expander + PLM (would work similar to the X10 implementation, just a different set of events with different formatting, though I’d recommend people connect Insteon to HASS in almost any other way than through the Elk)
  • Possibly implement virtual keypads complete with function keys and so on (a good portion of the logic for this is in PyElk but implementing the UI requires learning how to do stuff with Polymer), either replacing (i.e. 1:1 virtual keypads for all actual keypads instead of 1:1 pseudo-virtual keypads using ‘alarm control panel’ to alarm areas as it is now) or in addition to current (both ‘alarm control panel’ and ‘virtual keypad’ entities and UIs)
  • Possibly implement displaying text on keypad LCD (likely implement it like a notification service)
  • Add ability to use the Elk’s TTS dictionary to speak words / phrases (phrases are easy, the Elk handles timing all the separate words, using more than one word by itself to manually form phrases is tricky but more flexible, since timing the space between words may not be easy to do reliably … could be implemented as some kind of notification service?)
1 Like
80

In addition to the set points, the control of the fan between “AUTO” and “ON” mode doesn’t seem to function at present. I’ve not had an opportunity to debug this by looking at messages that are emitted, etc.

The UI is also not ideal; like a switch. I realize this is likely a consequence of how the Climate stuff in Home Assistant works.

Yes, this would be great. As a reminder, I want this to be able to define some tasks to “chirp” the alarm speaker. There may be other operations that are not exposed by the serial API that this could give access to by defining simple automation tasks in the Elk.

This would be handy, in addition to avoiding the creation of a bunch of entities in Home Assistant that won’t get uised…

This would be appreciated, though I have no idea how hard it would be to implement…

Somewhat related to this is the ability to get some additional events that the Elk generates exposed to Home Assistant. These could be major events like the alarm going off. Or simple events that would be real handy like activation of function keys on the keypads. I use this extensively today to trigger some simple lighting (turn on the porch lights for 15 minutes so I can take the dog outside…).

I don’t know what the best style for exposing the hundreds of Elk events might be; perhaps a few basic types with most of the details in attributes? I don’t have enough experience building automations in Home Assistant to offer a good opinion here…

These would both be awesome!

Thanks again for all the really hard work to get this working so well!

81

@lmamakos I haven’t tested it exactly (didn’t want to set off the alarm, too lazy to disconnect it) but you should be able to see the alarm state in alarm_control_panel.*, with the Alarm attribute. It might be useful to change this to be something numeric to simplify reacting to it with automations, or perhaps I’ll just add another attribute that is the corresponding numeric setting for the text string displayed there.

Getting function keys & etc keypad related things I could easily do as far as just getting states into HASS, but actually giving it a pretty UI will require learning Polymer :smiley:

82

This is great news guys! I’m pretty proficient with Elk configurations, if there’s anything I can do to help, please let me know. I’m eager to see a native HASS/ELK integration!

83

Have you guys looked at the following?:

https://pydigger.com/pypi/PyElk

84

@cyberk The PyElk link, I’m the one who created PyElk. :smiley: My HASS integration uses PyElk.

They’re both on github:


Eventually I’ll submit a pull request to home assistant itself to get the stuff in ha-elkm1 repo into HASS itself, but not until it’s a bit more polished and using asyncio so that it plays nicer with everything else :slight_smile:

The other link for the official API isn’t directly useful since it’s for building programs in C. I’ve just been using mostly the ASCII protocol documentation directly, rather than trying to port the existing integrations out there (though I have looked at things like Elkington while researching how the protocol works).

85

I’m about halfway done figuring out how to make use of configuration options for including/excluding zones/outputs/keypads/etc in HASS both for UI purposes and for state tracking purposes (i.e., you can hide a given device from the UI but still track the state to run automations off of it or merely logbook the data, or have it completely removed from HASS entirely to remove clutter from all the unused things in the states view).

Tasks are practically implemented (but not tested), would have been updated on Github/PyPi already if I hadn’t gotten side tracked trying to figure out how I’m going to handle Counters and Custom Values. They’re both able to be set to arbitrary things (i.e. a number), so none of the toggling sorts of component classes (switches etc) work. There’s some generic input classes meant for making automation scripts but they only have boolean, selection (from a listed set of options), and slider. Technically slider would work, but then you’d have a range of 0 to 65535 to contend with and it would be hard to set a specific value… may have to implement my own UI elements in Polymer later, and new generic input component for entering raw text/number values, and just expose these only as sensors for now.

Getting these three (Tasks, Counters, Custom Values) implemented (or at least partially implemented) will wrap up most of the low hanging fruit. The other features will probably be more time consuming, but once I get those three plus the config update handled, and maybe try to get the thermostats to be more useful (not fixing the UI issue in auto mode, but making heat / cool work separately), I’m going to start working on the whole asyncio thing before I do anything else (UI stuff like virtual keypads, new input widgets, TTS, etc).

Once the Elk integration is a good asyncio citizen, I will move work into my fork of HASS and try submitting the changes as a PR into HASS itself, then tackle those more difficult issues from within my fork of HASS (with subsequent PRs), rather than a separate repo of just my components.

86

@lmamakos

Well, I’ve updated Github/PyPi, and tasks work, though the UI doesn’t seem to consistently update (when triggering from HASS, since they’re momentary and they should turn ‘off’ again). I haven’t gotten to the bottom of it yet. But you can toggle it off and on again and have it work still … I’ve tested it with a task that toggles an output every time it’s called, and I can see the output state change (as well as a zone state change, since I have it connected to a zone input) accordingly.

You can also now control which devices are included/excluded from being set up in HASS, and PyElk will try not to do unnecessary scanning of devices as well to an extent during startup. If you don’t specify anything, the behavior is the same as before.

Ex:

elkm1:
  host: socket://1.2.3.4:2101
  area:
    exclude: 5-8
  zone:
    exclude: 35-98, 105-183
  output:
    include: 1-10
    exclude: 5
  x10:
    include: a1-a16
  task:
    include: 1-4

Available device types you can specify for include/exclude (and later more options) are ‘zone’, ‘output’, ‘area’, ‘keypad’, ‘thermostat’, ‘x10’, ‘task’. All except x10 use numeric ranges, for x10 you can use ranges like ‘a1-b8’ etc, using the actual house code / device code combinations for X10. Alternatively to a range, you can specify a single number or for x10 housecode/device number combination (i.e. a10). You can combine as many single/range elements as you want as comma separated values.

The logic for include/exclude is as follows:
a) if no include specified, default include to all devices (i.e. for zones this is 1-208, for tasks 1-32), otherwise use specified include from configuration
b) mark all included devices to be included
c) mark all excluded devices to be excluded (not included)
d) try not to do unnecessary work in PyElk on excluded devices, and don’t load excluded devices at all in HASS (will add an option later to make them optionally hidden instead)

This allows you to select a few ranges to keep included, then punch holes in them to remove some things.

87

Thanks for all your work on this! There is a python3 wrapper in that official elk implementation. Someone on this thread submitted the pull request for it a while ago and I helped to get it pushed through on the Elk side.

setup.py Support generate python3 library with swig. 6 months ago

88

@cyberk D’oh. I knew I thought I’d seen some existing thing when I looked for it previously, but never found it again… because it was buried in that non-Python stuff. And I’d already glanced at that recently enough to not even give it a second look.

Well, at least I understand what the code is doing (generally) since I wrote it :expressionless: And it’s probably going to be easier to make this custom code asyncio capable that trying to wrap up their C code that was converted to Python … since I can simply make changes if necessary.

89

If there’s anything I can do to support you, please let me know. Are you writing this integration by connecting directly to the serial port on the elk, or are you using an M1XEP or C1M1?

90

@cyberk I’ve only tested via M1XEP, but it should work identically with C1M1, and you should be able to use serial port the same way (just specify the host as /dev/ttyUSB0 or whatever port instead of socket://IP:port)

91

I believe you’re 100% correct, from my understanding, the m1xep is just a serial to ethernet expander. My ISY was able to connect via the M1XEP and the C1M1 with no difference. The C1M1 also has a USB connection which apparently allows communication. The C1M1 also allows for proxy communication, and I can only assume that it’s a reverse-proxy type system.

Again, I can’t state how thankful I am that you are working on this, thank you, thank you, thank you.

92

Trying out the new version, seems a little flaky in some spots.

I wasn’t able to get the includes: thing to work; this throws an error which I’ll try to reproduced. Having no includes: or using excludes: seems to work just fine

In ha-elkm1/climate/elkm1.py, I think this change needs to be done, otherwise it seems like the reader thread fails? Or maybe that’s something else causing that problem…

diff --git a/climate/elkm1.py b/climate/elkm1.py
index 879d6f4..0a84e5b 100644
--- a/climate/elkm1.py
+++ b/climate/elkm1.py
@@ -38,7 +38,7 @@ def setup_platform(hass, config: ConfigType, add_devices: Callable[[list], None]
                 device = ElkClimateDevice(thermostat)
                 devices.append(device)
             else:
-                _LOGGER.debug('Skipping excluded Elk Thermostat: %s', device._number)
+                _LOGGER.debug('Skipping excluded Elk Thermostat: %s', thermostat._number)

     add_devices(devices, True)
     return True

Also, I think the most recent version of pyelk hasn’t been pushed to github…

I have been able to successfully invoke tasks in the Elk, too…

I’ll try to mess around a bit more with this tomorrow if I have some time. I really do like the new features; should improve the start-up time.

I’m not sure if there is brittleness if you try to refresh the browser and look at states while the scanning of the Elk is going on. It feels like the outcome is better if you leave it alone :slight_smile: Perhaps this is transmitting too many commands at the elk asynchronously while the scanning is happening?

93

@lmamakos
Oops, I think you may be right on that _LOGGER line. And you may be right that I forgot to push PyElk, I went ahead and pushed it, I’ll make the mentioned fix and bump PyElk version accordingly later.

For include/exclude config, for reference, my current config section is (IP anonymized):

elkm1:
  host: socket://1.2.3.4:2101
  area:
    exclude: 5-8
  zone:
    exclude: 35-98, 105-183
  output:
    include: 1-10
  x10:
    include: a1-a16
  task:
    include: 1-4

I did test more variations of this but not from within HASS (tested using the PyElk CLI test tool, and some standalone Voluptuous test code while first figuring out how to use Voluptuous for config parsing). Does it throw an actual error error, or just behave oddly? Currently you should be getting the detected config dumped into the error log regardless (just for testing, will provably remove on next update), from this line in components/elkm1.py:

    _LOGGER.error('Elk config : %s', elk_config)

As for refreshing the browser… things may be weird if it’s rescanning, but that should only happen if you connect / disconnect with ElkRP or enter/exit programming on a keypad. During normal startup, it currently stalls HASS until it finishes the initial scan, so the browser UI should just be unresponsive and say “reconnecting” or something until it’s done. Once I switch over to doing things the asyncio way, this will change somewhat, as you can get HASS started before all the Elk stuff has loaded in, in which case you’ll see missing devices and such until startup is complete.