ISY994 Z-Wave Sensor Enhancements: Testers Wanted

shbatm · June 30, 2019, 10:15am

After a few people have asked about Z-Wave issues in conjunction with my other improvements to the ISY994 integration, I took a stab at expanding out the Z-Wave categorization as well.

Instead of trying to rely on ISY Device Types which don’t work for Z-Wave, I added a patch to the PyISY module to pull in the devtype.cat field from the ISY Node REST data. This, coupled with the ISY SDK allows all Z-Wave devices to be categorized just like Insteon with the Type field.

In addition, this branch pulls in and captures any device properties that are pushed to the event stream, but don’t show up under the Node properties (these are the properties that show up if you call /rest/nodes/NODE_ADDRESS but don’t appear under /rest/nodes like Ramp Rate for lights, and Modes for climate devices). These will now show up as Device State Attributes for the different devices in Hass (visible in more_info windows and usable in templates).

I’ve tested what I can without having any Z-Wave devices tied to my ISY using the node files others have provided (and the fake-isy-994i project); but the Z-Wave part is still very experimental and I need help testing it.

If everything goes to plan, then next part would be to correctly assign device classes automatically.

Fixes are also included for Insteon Motion Sensor Gen 1 battery/dusk-dawn nodes, and leak sensor heartbeats.

Finally, this branch includes the climate changes and variable options that @Blueman2 and other have been testing.

To Install As A Custom Component (EDIT: Link Change)

I recommend using subversion to install this, as it will let you quickly export the files you need:

svn export https://github.com/shbatm/home-assistant/branches/isy994-climate/homeassistant/components/isy994 /your/hass/config/custom_components/isy994/

This requires PyISY-beta version >= 1.1.7 (EDIT: bug fix in 1.1.6) to be installed. It should be installed by Hass automatically when you add the custom component, but if you face any issues, you can manually install it with pip.

@danbutter, @Blueman2, @jayhova, @jon102034050, @tradel, @rccoleman, @OverloadUT – sorry for the shameless tag, but I thought you guys may be interested in this.

jon102034050 · June 30, 2019, 12:37pm

I’ll need a couple days to test things out as my isy currently only has an aeotec multi 6 paired with it. But, upon first boot with my test HASS instance, it appears to be a pretty good try. Let me know if you have any questions and I can test some things. It seems to add all these things as switches, which is a little odd. most are probably better off as binary sensors or sensors I imagine.

fyi - the temperature is 77.4, so the decimal maybe needs to be reworked there.

shbatm · June 30, 2019, 12:47pm

Thanks for testing!

I took my best guess at divvying up the different categories and assigning them to the different domains (sensors, binary_sensors, switches, etc.)… but without actual devices I know some are wrong. Just leave a note here with the devtype.cat number of the device from ISY/rest/nodes/ZWADDR and what kind of device it should be.

    <node flag="0" nodeDefId="UZW005D"><address>ZW026_118</address>
        <name>ZW 026 Multilevel Sensor</name>
        <family>4</family>
        <parent type="3">15747</parent>
        <type>4.16.1.0</type>
        <enabled>true</enabled>
        <deviceClass>0</deviceClass>
        <wattage>0</wattage>
        <dcPeriod>0</dcPeriod>
        <startDelay>0</startDelay>
        <endDelay>0</endDelay>
        <pnode>ZW026_1</pnode>
        <sgid>118</sgid>
        <devtype>
            <gen>4.16.1</gen>
            <mfg>134.3.6</mfg>
            <cat>118</cat>  <-- CHECK THIS NUMBER
        </devtype>
    </node>

You can test changes yourself by finding that number in isy994/__init__.py and moving it to the correct domain.

shbatm · June 30, 2019, 12:50pm

The State Attributes are just reported straight out of the ISY. To make them valuable for use in Hass I would highly recommend breaking them out as template sensors, there you can set the unit of measurement and correct any reporting errors (like needing to divide by 10).

jon102034050 · June 30, 2019, 12:52pm

Sure, that makes sense. Do you just want me to paste my ISY/rest/nodes here? it’s not very big and it’ll probably help to sort through them

jon102034050 · June 30, 2019, 12:53pm

<?xml version="1.0" encoding="UTF-8"?>
<nodes>
  <root>
  </root>
  <node flag="128" nodeDefId="UZW0003">
    <address>ZW002_1</address>
    <name>ZW 002 Multilevel Sensor</name>
    <family>4</family>
    <type>4.33.1.0</type>
    <enabled>true</enabled>
    <deviceClass>0</deviceClass>
    <wattage>0</wattage>
    <dcPeriod>0</dcPeriod>
    <startDelay>0</startDelay>
    <endDelay>0</endDelay>
    <pnode>ZW002_1</pnode>
    <sgid>1</sgid>
    <devtype>
      <gen>4.33.1</gen>
      <mfg>134.258.100</mfg>
      <cat>118</cat>
    </devtype>
  </node>
  <node flag="0" nodeDefId="UZW0004">
    <address>ZW002_104</address>
    <name>ZW 002 Binary Sensor 1</name>
    <family>4</family>
    <type>4.33.1.0</type>
    <enabled>true</enabled>
    <deviceClass>0</deviceClass>
    <wattage>0</wattage>
    <dcPeriod>0</dcPeriod>
    <startDelay>0</startDelay>
    <endDelay>0</endDelay>
    <pnode>ZW002_1</pnode>
    <sgid>104</sgid>
    <devtype>
      <gen>4.33.1</gen>
      <mfg>134.258.100</mfg>
      <cat>104</cat>
    </devtype>
    <property id="ST" value="100" formatted="On" uom="78" />
  </node>
  <node flag="0" nodeDefId="UZW0005">
    <address>ZW002_155</address>
    <name>ZW 002 Motion Sensor 1</name>
    <family>4</family>
    <type>4.33.1.0</type>
    <enabled>true</enabled>
    <deviceClass>0</deviceClass>
    <wattage>0</wattage>
    <dcPeriod>0</dcPeriod>
    <startDelay>0</startDelay>
    <endDelay>0</endDelay>
    <pnode>ZW002_1</pnode>
    <sgid>155</sgid>
    <devtype>
      <gen>4.33.1</gen>
      <mfg>134.258.100</mfg>
      <cat>155</cat>
    </devtype>
    <property id="ST" value="100" formatted="On" uom="78" />
  </node>
  <node flag="0" nodeDefId="UZW0006">
    <address>ZW002_157</address>
    <name>ZW 002 Tamper Alarm 1</name>
    <family>4</family>
    <type>4.33.1.0</type>
    <enabled>true</enabled>
    <deviceClass>0</deviceClass>
    <wattage>0</wattage>
    <dcPeriod>0</dcPeriod>
    <startDelay>0</startDelay>
    <endDelay>0</endDelay>
    <pnode>ZW002_1</pnode>
    <sgid>157</sgid>
    <devtype>
      <gen>4.33.1</gen>
      <mfg>134.258.100</mfg>
      <cat>157</cat>
    </devtype>
    <property id="ST" value="0" formatted="Off" uom="78" />
  </node>
  <node flag="0" nodeDefId="UZW0007">
    <address>ZW002_172</address>
    <name>ZW 002 Intrusion Alarm 1</name>
    <family>4</family>
    <type>4.33.1.0</type>
    <enabled>true</enabled>
    <deviceClass>0</deviceClass>
    <wattage>0</wattage>
    <dcPeriod>0</dcPeriod>
    <startDelay>0</startDelay>
    <endDelay>0</endDelay>
    <pnode>ZW002_1</pnode>
    <sgid>172</sgid>
    <devtype>
      <gen>4.33.1</gen>
      <mfg>134.258.100</mfg>
      <cat>172</cat>
    </devtype>
    <property id="ST" value="0" formatted="Off" uom="78" />
  </node>
  <node flag="0" nodeDefId="UZW0008">
    <address>ZW002_173</address>
    <name>ZW 002 Tamper Code Alarm 1</name>
    <family>4</family>
    <type>4.33.1.0</type>
    <enabled>true</enabled>
    <deviceClass>0</deviceClass>
    <wattage>0</wattage>
    <dcPeriod>0</dcPeriod>
    <startDelay>0</startDelay>
    <endDelay>0</endDelay>
    <pnode>ZW002_1</pnode>
    <sgid>173</sgid>
    <devtype>
      <gen>4.33.1</gen>
      <mfg>134.258.100</mfg>
      <cat>173</cat>
    </devtype>
    <property id="ST" value="0" formatted="Off" uom="78" />
  </node>
  <node flag="0" nodeDefId="UZW0009">
    <address>ZW002_153</address>
    <name>ZW 002 Glass Break Alarm 1</name>
    <family>4</family>
    <type>4.33.1.0</type>
    <enabled>true</enabled>
    <deviceClass>0</deviceClass>
    <wattage>0</wattage>
    <dcPeriod>0</dcPeriod>
    <startDelay>0</startDelay>
    <endDelay>0</endDelay>
    <pnode>ZW002_1</pnode>
    <sgid>153</sgid>
    <devtype>
      <gen>4.33.1</gen>
      <mfg>134.258.100</mfg>
      <cat>153</cat>
    </devtype>
    <property id="ST" value="0" formatted="Off" uom="78" />
  </node>
  <node flag="128" nodeDefId="UZW000A">
    <address>ZW003_1</address>
    <name>ZW 003 Dimmer Switch</name>
    <family>4</family>
    <type>4.17.1.0</type>
    <enabled>true</enabled>
    <deviceClass>0</deviceClass>
    <wattage>0</wattage>
    <dcPeriod>0</dcPeriod>
    <startDelay>0</startDelay>
    <endDelay>0</endDelay>
    <pnode>ZW003_1</pnode>
    <sgid>1</sgid>
    <devtype>
      <gen>4.17.1</gen>
      <mfg>388.18244.12338</mfg>
      <cat>109</cat>
    </devtype>
    <property id="ST" value="0" formatted="0%" uom="51" />
  </node>
  <group flag="12" nodeDefId="InsteonDimmer">
    <address>00:21:b9:02:1d:bd</address>
    <name>ISY</name>
    <family>6</family>
    <members>
      <link type="0">00:21:b9:02:1d:bd</link>
      <link type="0">ADR0001</link>
      <link type="0">ZW002_1</link>
      <link type="0">ZW002_104</link>
      <link type="0">ZW002_155</link>
      <link type="0">ZW002_157</link>
      <link type="0">ZW002_172</link>
      <link type="0">ZW002_173</link>
      <link type="0">ZW002_153</link>
      <link type="0">ZW003_1</link>
    </members>
  </group>
  <group flag="132" nodeDefId="InsteonDimmer">
    <address>ADR0001</address>
    <name>Auto DR</name>
    <family>6</family>
    <deviceGroup>16</deviceGroup>
    <pnode>ADR0001</pnode>
    <members>
    </members>
  </group>
</nodes>

shbatm · June 30, 2019, 12:59pm

Thanks. I actually got that one right on the category, but I had messed something else up in the sorter and pushed an update just before your first reply–update the files and try again.

jon102034050 · June 30, 2019, 1:03pm

awesome, looks much better! Will test a bit more and get back to you

Blueman2 · June 30, 2019, 3:19pm

Thanks for the continued work on this, @shbatm! I am just downloading and will do some testing today. I hope the z-wave module gets to you by Wednesday as well.

Is it necessary or beneficial for me to delete z-wave entries in the core.entity_registry file before testing?

EDIT: Just loaded up your new updates. It did pick up a few more items! It picked up my z-wave smoke/co alarm and reported battery level correctly! Other parts of the smoke alarm will not show up until they are triggered I think. I will test that further, but very nice!

One issue I continue to see is the failure to get any data for z-wave sensors such as Watts/KWHrs from Home Energy Monitors and Temperatures from my z-wave pool device. Most of the entities show up, but no values are given.

Here is the ISY node details of my pool controller, for example:

Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ] ----------------------------------------------
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ] ZW 044.01 Chlorinator
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ] ZW044_1 uid=44 type=4.16.1 mid=5 tid=20549 pid=1619
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ] Association Group ID 1
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x91  V1   MANUFACTURER_PROPRIETARY
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x73  V1   POWERLEVEL
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x72  V1   MANUFACTURER_SPECIFIC
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x86  V1   VERSION
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x81  V1   CLOCK
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x60  V2   MULTI_CHANNEL
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x70  V1   CONFIGURATION
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x85  V1   ASSOCIATION
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x25  V1   SWITCH_BINARY
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x27  V1   SWITCH_ALL
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x43  V1   THERMOSTAT_SETPOINT
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x31  V1   SENSOR_MULTILEVEL
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  x30  V1   SENSOR_BINARY
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ]    -  xEF  V0   MARK
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ] - Secure
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ] 
Sun 06/30/2019 08:37:13 AM : [ZW-SHOW         ] ----------------------------------------------

All the switches show up fine and work. The temperature probes show up, but no values are given. The Thermostat Setpoint does not show up.

Here is node ZW044_019_1 which has my pool temperature, result from REST query:

<?xml version="1.0" encoding="UTF-8" ?> 
- <nodeInfo>
- <node flag="128" nodeDefId="UZW0124">
  <address>ZW044_019_1</address> 
  <name>ZW 044.19 Water Temperature</name> 
  <family>4</family> 
  <parent type="3">19219</parent> 
  <type>4.33.0.0</type> 
  <enabled>true</enabled> 
  <deviceClass>0</deviceClass> 
  <wattage>0</wattage> 
  <dcPeriod>0</dcPeriod> 
  <startDelay>0</startDelay> 
  <endDelay>0</endDelay> 
  <pnode>ZW044_019_1</pnode> 
  <sgid>1</sgid> 
- <devtype>
  <gen>4.33.0</gen> 
  <mfg>5.20549.1619</mfg> 
  <cat>118</cat> 
  </devtype>
  </node>
- <properties>
  <property id="BARPRES" value="" formatted="" uom="0" /> 
  <property id="CLIHUM" value="" formatted="" uom="0" /> 
  <property id="CLITEMP" value="75" formatted="75°F" uom="17" /> 
  </properties>
  </nodeInfo>

Not sure if it matters, but there is also a ZW044_019_147 binary switch that is found. It is a phantom switch that is not real and never outputs any data. Not sure it that is causing problems or not.

And here is the REST query to my ISY for a Home Energy Monitor. HA picks this device up, but no value is populated.

 <?xml version="1.0" encoding="UTF-8" ?> 
- <nodeInfo>
- <node flag="0" nodeDefId="UZW014A">
  <address>ZW030_143</address> 
  <name>ZW 030 Energy Meter</name> 
  <family>4</family> 
  <parent type="1">ZW030_1</parent> 
  <type>4.33.1.0</type> 
  <enabled>true</enabled> 
  <deviceClass>0</deviceClass> 
  <wattage>0</wattage> 
  <dcPeriod>0</dcPeriod> 
  <startDelay>0</startDelay> 
  <endDelay>0</endDelay> 
  <pnode>ZW030_1</pnode> 
  <sgid>143</sgid> 
- <devtype>
  <gen>4.33.1</gen> 
  <mfg>134.2.9</mfg> 
  <cat>143</cat> 
  </devtype>
  <property id="ST" value="1107557" formatted="1107.557 Watts" uom="73" prec="3" /> 
  </node>
- <properties>
  <property id="CV" value="" formatted="" uom="0" /> 
  <property id="PF" value="" formatted="" uom="0" /> 
  <property id="PULSCNT" value="" formatted="" uom="0" /> 
  <property id="ST" value="1107557" formatted="1107.557 Watts" uom="73" prec="3" /> 
  <property id="TPW" value="26501200" formatted="26501.200 kWh" uom="33" prec="3" /> 
  </properties>
  </nodeInfo>

These devices do show up in HA, but their values just never get populated.

shbatm · June 30, 2019, 8:29pm

BTW, if you have ssh access to the Hass.io machine you should be able to install svn with sudo apt install subversion.

No, but you may have some cleanup to do afterwards if devices change domains.

Do any values show up under the attributes? e.g. When you click the sensor for the more info pop up or when you look in the States developer panel (see @jon102034050’s post.

You can add ‘[ignore]’ to the device name in the ISY to prevent this from showing up in Hass (or whatever you set ignore_string to in your isy994 config for Hass.

Same thing for this one: is any info showing up in the attributes? I’ll do a bit more troubleshooting on the value, but I need to make sure everything else is getting put somewhere like it should.

Blueman2 · June 30, 2019, 8:47pm

Wow, on ZW044.019 sure enough, the temperature is there under attributes! For the HEM devices (e.g. zw030) there is no info under attributes.

shbatm · June 30, 2019, 9:20pm

ISY doesn’t report a status for this device for some reason (notice in the REST dump there is no ST property anywhere). For this to be useful you’ll have to add a template sensor to extract the temperature from the attributes, then hide the original sensor.

I missed classifying type 143. Try changing this line:

        'zwave_cat': (['118'] +

To this:

        'zwave_cat': (['118', '143'] +

shbatm · July 1, 2019, 4:31am

New updates have been pushed to the isy994-zwave-x branch.

Changes:

Moved constants to their own file
Updated units of measurement for ISY SDK 5.0.4
Now tries to process state attributes and fix precision and get text values and friendly units where possible. Note: this translation is probably only going to work for ISYv5 firmware, v4 doesn’t use the numeric UOMs in the REST protocol.

Blueman2 · July 1, 2019, 6:02am

Updated to your latest push. For some reason, I have lost the Temperature attributes for ZW044_019 and for the battery attribute for the z-wave smoke detector that was found in the prior build. I am running 5.0.14 ISY firmware.

shbatm · July 1, 2019, 7:48am

Try the latest and greatest (and make sure you’ve grabbed the const.py file) and let me know if you’re still having issues.

Please check the logs too for any errors. Easiest for me is to use:

egrep 'ERROR|WARNING' /hass/config/home-assistant.log -A 20 -B 2

Blueman2 · July 1, 2019, 3:29pm

Just pulled down the latest files. Off topic, but I found a great PC based tool, TortoiseSVN, that simplifies updating for me. For some reason, Hassio does not have any package manager at all. No apt/apt-get/etc. Booted up fine, and absolutely no error/warning messages related to ISY994, which in itself is unusual. ~~But I have now lost even more attributes (no smoke/co alarm battery level anymore) and even lost a few z-wave devices (a motion detector)~~. I am happy to change log level to debug if that helps. Is that setting in your code, or can a put that in the configuration.yaml to set loglevel?

EDIT: Well, some really cool news!!! This latest build picked up the thermostat for my pool controller!!! Yay!!! It is off by factor of 10, and it can see settings but changes do not get sent to the thermostat, but it is there!

Also, the motion sensor was still there, it just changed domains and I missed it. ~~But Attributes are still missing that were there before.~~ edit: it just took attributes to change before they showed up. D’oh.

jon102034050 · July 1, 2019, 3:41pm

The latest build seems to pickup everything correctly for me (just using a simple test setup at the moment), but i’ve noticed things take a really long time to update. For example, if I toggle a zwave dimmer through the UDI admin interface, its instant. if i toggle it through the HASS UI, it takes ~5 seconds to actually do the action. On a similar note, my binary sensor for motion won’t update in HASS, but it works just fine in the UDI interface. Thoughts?

Blueman2 · July 1, 2019, 3:54pm

I just did a quick test of some of my z-wave switches, and they all seem to respond instantly. I have no z-wave dimmers, just switches, so maybe dimmers are impacted differently?

I also noticed this new build picks up status of several Polyglot nodes, for example Holiday node that tells what holidays are in the coming week. So definitely making some great progress.

shbatm · July 1, 2019, 7:58pm

At least we’re making some progress…

I have very little delay controlling and receiving updates. Something may be causing issues in your network. Try enabling debug logging and you may also try sending REST commands through a browser and see if you have the same delay (e.g. /rest/nodes/ADDR/cmd/DON and /DOF)

To enable debug logging:
Add to configuration.yaml

logger:
  default: info
  logs:
    custom_components.isy994: debug

Send me the rest node info for these. I’m curious why they would have changed domains with the last update.
I also wonder if the attributes just haven’t updated and that’s why they’re not showing.

Send the node dump for this one too (rest/nodes/ADDR). The debug logs as well for when you try and change something.

shbatm · July 1, 2019, 8:03pm

A note on polyglot nodes… Adding this is an unintended side effect at the moment. Because of their nature I don’t know any good ways to classify them. There may be some way to do the same thing as I did with variables (add by exception) but this is ‘out of scope’ for me for the moment.