How to test the OZW Beta without fully switching over

Tags: #<Tag:0x00007f73882586d8> #<Tag:0x00007f7388258598> #<Tag:0x00007f73882583e0>

Prerequisite: MQTT Broker (Server) and Integration


Test Migration for Supervised installs:

  1. Install the addon, but leave it off/disabled
  2. Open the zwave control panel and hit “Stop Network”
  3. Config/Activate the OZW Addon
  4. Activate the OZW (Beta) Integration from Integrations -> +

You are now using the OZW Beta integration!

Don’t like it and want to switch back? Easy.

  1. Remove the OZW Integration from Integrations -> Remove
  2. Stop/Disable the OZW Addon
  3. Open zwave control panel and hit “Start Network”

Congrats! You’ve switched back to the 1.4 built-in integration.


Core/Container Installs:

  1. docker pull (see below for full command) the ozwd container, do not start it.
  2. Open the zwave control panel and hit “Stop Network”
  3. Configure and start the ozwd container. (see below)
  4. Activate the OZW (Beta) Integration from Integrations -> +

You are now using the OZW Beta integration!

Don’t like it and want to switch back? Easy.

  1. Remove the OZW Integration from Integrations -> Remove
  2. Stop the ozwd container.
  3. Open zwave control panel and hit “Start Network”

Congrats! You’ve switched back to the 1.4 built-in integration.


Reminder, if you restart Home Assistant before switching back you’re going to have a bad time.
Make sure you switch back, or remove the zwave configuration from your configuration before restarting.


Full docker pull command: docker pull openzwave/ozwdaemon:allinone-latest

Config/Start the ozwd conatiner (runs interactive mode or foreground):

docker run -it --security-opt seccomp=unconfined \
--device=/dev/serial/by-id/usb-0658_0200-if00:/dev/zwave \
-v /opt/ozw:/opt/ozw/config \
-p 1983:1983 \
-p 5901:5901 \
-p 7800:7800 \
-e MQTT_SERVER="<MQTT SERVER IP/HOST>" \
-e MQTT_USERNAME="<MQTT SERVER USER>" \
-e MQTT_PASSWORD="<MQTT PWD>" \
-e OZW_NETWORK_KEY="<YOUR ZWAVE NETWORK KEY>" \
-e USB_PATH=/dev/zwave openzwave/ozwdaemon:allinone-build-150

Daemon mode (background mode):

docker run -d --security-opt seccomp=unconfined \
--device=/dev/serial/by-id/usb-0658_0200-if00:/dev/zwave \
-v /opt/ozw:/opt/ozw/config \
-p 1983:1983 \
-p 5901:5901 \
-p 7800:7800 \
-e MQTT_SERVER="<MQTT SERVER IP/HOST>" \
-e MQTT_USERNAME="<MQTT SERVER USER>" \
-e MQTT_PASSWORD="<MQTT PWD>" \
-e OZW_NETWORK_KEY="<YOUR ZWAVE NETWORK KEY>" \
-e USB_PATH=/dev/zwave openzwave/ozwdaemon:allinone-build-150

Modify as needed for your USB stick and other variables.

Docker Compose version:

version: '2'
services:
  mqtt:
    container_name: MQTT
    restart: unless-stopped
    image: eclipse-mosquitto
    privileged: true
    volumes:
      - /home/docker/mosquitto/config/:/mosquitto/config/
      - /home/docker/mosquitto/log:/mosquitto/log
      - /home/docker/mosquitto/data:/mosquitto/data
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "1883:1883"
      - "9001:9001"

  qt-openzwave:
    image: openzwave/ozwdaemon:allinone-build-150
    container_name: "qt-openzwave"
    security_opt:
      - seccomp:unconfined
    depends_on:
      - "MQTT"
    devices:
      - "/dev/serial/by-id/usb-0658_0200-if00:/dev/zwave"
    volumes:
      - /opt/ozw:/opt/ozw/config
    ports:
      - "1983:1983"
      - "5901:5901"
      - "7800:7800"
    environment:
      TZ: "America/Phoenix"
      MQTT_SERVER: "[redacted]"
      MQTT_USERNAME: "[redacted]"
      MQTT_PASSWORD: "[redacted]"
      USB_PATH: "/dev/zwave"
      OZW_NETWORK_KEY: "[redacted key]"
    restart: unless-stopped

How to obtain your network key

If your key is not in your configuration.yaml check core.config_entries.

Go into .storage/core.config_entries and search for network_key.

Troubleshooting OZW Beta

“All my devices are showing as unavailable!”

If topic: OpenZWave/1/status/ does not contain: "Status": "driverAllNodesQueried" all your devices will say unavailable. This can be caused by battery devices not reporting, try waking battery powered devices per the device manual.

Additional help can be found on the Home Assistant Discord in #zwave

Like what you see with the new integration and want to full switch?
See this guide for the steps to complete the switch.

17 Likes

Will this every time that you restart the ozw container or only the first time?

If It will then it’s going to get tedious to have to go around and wake up every battery powered device before you can use your zwave network devices after a restart.

Technically you should get a driverAwakeNodesQueried for sleeping nodes, after that initial startup and the cache is created that is.

I don’t have any battery nodes except a lock, and that is a different animal.

well, i guess i don’t have any other excuses now so i’ll have to try it out on my production system.

thanks… :roll_eyes::wink: :smile:

I’ve been using it on my production system for a couple weeks now.
0.113 should have quite a few improvements.

Well, I finally tried it out on a test instance of HA Container.

I had some kinds of expected results - all of my mains powered nodes are showing up as “complete” in my mqtt broker and all of my battery powered nodes are showing as either “static” or “probe” for the “nodeQueryStage”

But I have some (most) unexpected results -

  1. all of my battery powered devices are showing as either “static” or “probe” for the “nodeQueryStage”. That’s the expected part but even tho they aren’t reporting as “complete” the status is reporting “driverAllNodesQueried”,.

  2. even tho the status is reporting as “driverAllNodesQueried” none of my battery powered devices are showing up in the integration or in my states page as sensors.

  3. I have one “unknown” device (well, two if you count the one I had installed on my other non-production HUSBZB-1 z stick as a test). I’m not sure how to get the xml file for those devices to make them not “unknown”

4. I have two power strips that have 6 individually controllable outlets which also have power and energy reporting for each outlet but neither of the devices are showing up and none of those associated switches or sensors are showing up in the entities listing even tho both nodes report as complete. I’m not sure what needs to be done to get those to show up.

  1. is there any way to edit the cache so that the device/entity names can be used on multiple instances of HA using the same integration? It’s going to suck to have to edit all device & entity names in all of my HA instances (one production & three test instances).

as of right now I don’t see any other issues yet.

Any suggestions?

EDIT:

edited the above since after a restart some things have shown up.

These two are just the battery devices needing to be woken up for the interview, this is to rebuild the cache, once that’s done you should be good.

This is odd as Fishwaldo ported all the 1.4 XMLs to 1.6, so the devices, if they were detected in the old zwave should still be named in the new one.

Yes, if you are running the docker container on it’s own you can go into the /opt/ozw directory and modify the ozwcache_xxxxxx.xml file while the container is down, then spin it back up. If you are using the addon way, I’m told there’s no easy way to modify the cache with the addon :frowning:

It’s not recommended to rename the devices via the cache in case the cache gets rebuild for some reason, it’s best to just rename them in HA and let the device registry do the renaming for you.

They’re stored in the core. device_registry you could just copy that file over to all your instances.

If you get too stuck there’s plenty of smart people in #zwave on discord as well.

OK, I thought that was the case but this:

…seemed to be saying that none of the devices should be available (even mains powered ones) until that status was seen and that status only happened when all nodes reported in - even the battery powered ones.

Not a big deal. it was just more of a point of clarification.

agreed. that’s why I was asking.

So I dug in a bit deeper and found that the device that was wori=king in the old zwave install is a honeywell outlet and that apparently that manufactrurer xml isn’t in the old install either. The other is a minoston waterproof plug.

I’ve downloaded both xml files and p[laced them in the appropriate locations in the config directory and restarted both qt-ozw and HA but neither of them still show up correctly. I’m likelyy missing something else tho.

Is there anywhere that provides instructions on exactly what is required to update the manufacturer specific xml files?

Of course I am. I don’t need no stinkin’ add-ons! :wink:

I think that’s easier said than done.

Not only is it verboten! to edit that file, the zwave devices aren’t the only devices listed in there. So it’s going to be a surgical “copy/delete/paste” exercise and not a simple copying of the whole file.

And I would have to copy/delete/paste the zwave contents of the entity registry files for each entity as well.

Which is OK but definitely lends itself more to serious mistakes.

And then add to that the vague default naming of all the devices/entities it’s going to be a huge job just trying to correlate the new names with the existing names so all of my existing automations/scripts/lovelace views work without a massive editing of all of that stuff too.

I’m not too stuck yet and don’t think i will be (hopefully… :crossed_fingers:). I was just hoping this was going to be a less painful experience.

I should have known better… :frowning_face:

All I can say is that hopefully someone will devise a way for a reasonably easy migration path before the old one goes away. I’m moderately fluent in this stuff and I’m finding the idea of switching over with the requirement to completely manually reconfigure every device and entity pretty daunting.

Caveat to that is there’s technically 3 status that will bring it ‘online’:

  • driverAllNodesQueriedSomeDead - OZW has Initilized the Network and completed querying Devices on the Network. Some Nodes were detected as Dead and should be either removed or resurected after performing diagnostics on the affected nodes.
  • driverAllNodesQueried - OZW has Initilized the Network and completed querying all devices on the Network.
  • driverAwakeNodesQueried - OZW has Initilized the Network, and completed querying all “awake” or mains powered devices. Battery or Sleeping Devices may still be not fully initilized and need to be woken up before they are fully operational

Issue a refreshnodeinfo (see gist at bottom of post)

  1. Stop the container
  2. Modify the manufacturer.xml
  3. Start the container

Once modified THEN refreshnodeinfo should pick the devices up

Once the devices are in the device registry, if you rename the device, it will trickle down to the entities and rename them as well.


I renamed the device Alarm Siren and the entity and the entity_id's became domain.alarm_siren_<switch/etc>. So that’s not too difficult.

@cgarwood is working on making the device panel more intuitive, should help as well. The integration is still evolving, but hey we got those color temps sorted on RGBCT bulbs :stuck_out_tongue:

I completely migrated, and didn’t have to modify any of my automations or scripts, but everyone’s mileage will vary.

Here’s a make shift Control Panel it’s lacking a few commands, but the essential ones are there.

1 Like

The issue with the manufacturer xml files was that I forgot I needed to update the manufacturer_specific.xml file with the new device xml file.

I had just gotten that straightened out and noticed you had replied. So that part is good so far.

One other question related to that tho.

I’ve googled a bit and can’t find where the manufacturer files are stored for qt-ozw. I will submit a PR to update the files to include the two I’ve found missing but I’m not sure which repo I need to update.

it renames the entity friendly names but I think you then have to go in and edit the entity_id’s manually. At least that’s been my experience with the device/entity registry so far.

I just recently added the two Powernode 6 power strips and I had to do a whole bunch of copy/paste on the entity_ids.

But at least with the old zwave integration there is still the hidden “zwave.rename_node” service that you can use to rename the node and then you can use that to rename the entity_ids of the child entities.

ex1

Unless there is some equivalent command in qt-ozw?

I definitely don’t want my entity_ids to all be very generic - like :

          - switch.dsc26103_micro_switch_2nd_edition_switch
          - switch.dsc26103_micro_switch_2nd_edition_switch_2
          - switch.dsc26103_micro_switch_2nd_edition_switch_3
          - switch.dsc26103_micro_switch_2nd_edition_switch_4
          - switch.powernode_6_port_instance_1_switch
          - sensor.powernode_6_port_instance_1_electric_kwh
          - sensor.powernode_6_port_instance_1_electric_w
          - switch.powernode_6_port_instance_2_switch
          - sensor.powernode_6_port_instance_2_electric_kwh
          - sensor.powernode_6_port_instance_2_electric_w
          - switch.powernode_6_port_instance_3_switch
          - sensor.powernode_6_port_instance_3_electric_kwh
          - sensor.powernode_6_port_instance_3_electric_w
          ...etc
          - switch.powernode_6_port_instance_1_switch_2
          - sensor.powernode_6_port_instance_1_electric_kwh_2
          - sensor.powernode_6_port_instance_1_electric_w_2
          - switch.powernode_6_port_instance_2_switch_2
          - sensor.powernode_6_port_instance_2_electric_kwh_2
          - sensor.powernode_6_port_instance_2_electric_w_2
          - switch.powernode_6_port_instance_3_switch_2
          - sensor.powernode_6_port_instance_3_electric_kwh_2
          - sensor.powernode_6_port_instance_3_electric_w_2
          ...etc

All of my entity_ids are very specific to the use of the device so I’m going to have to update lots of stuff either way.

thanks for the control panel code. I’ll get it set up and see how it works.

openzwave/openzwave is the github repo for the XML files.

After you rename it in the device registry, you get a confirmation box, if you click “Ok” on that it should also prefix the entity_id's for you as well. The only time I’ve had that not work is when I refreshed a node and the prefixes were not all the same due to a new sensor showing up, but that was an easy fix.

I decided to try the OP method this morning.

Everything went well until I decided to keep OZW.

At that point I commented out my zwave config in YAML and restarted HA. Once rebooted, all OZW entities were unavailable. Do you guys have a clue what I did wrong?

I think I read somewhere that if ozw loses connection to the mqtt-broker, it will not reconnect automatically.
You could try to restart the ozw (and possibly the mqtt broker) as well.

1 Like

Thanks! Should I delete the zwave integration in the integration panel (in addition or instead of commenting out yaml)?

What happens if I restart without switching back?

If I want to start using this beta on a more permanent basis, what additional steps are needed?

Thanks!

Remove the old zwave configuration settings.

Stopping the network releases the USB stick so the ozwd container can use it, so if you restart without turning one of them off, they’ll fight over the USB stick and you’ll have a bad day.

Sorry if this was answered here before, but is there a way to migrate all existing entity/device names over to the new beta?
I followed the guide at the top of the page, everything started up without a hitch … and then I saw 55 devices and 280 entities I would have to rename. That’s a naww for me dawwwgg :joy:

…Thankfully the switchback went without a hitch as well.

They are working on creating some script or similar for an easier migration.

2 Likes

So, I bit the bullet and made the switch to the new OZW Beta yesterday.
All in all, it was pretty painless, although it of course took some time. I have about 60 devices and 500 entites (but the majority of entities are not actually in use), and it took about two hours before I had the home 80-90% up and running again. Then it took a while longer to go through and verify and fix all the little stuff that did not have consistant naming etc. One device needed to be removed and re added to OZW, but that also was pretty easy.

I basically followed the guide in the first post, but with a few more details outlined here.

First, I used this template in the template editor in developer tools to get a list of all my active zwave-devices in the old integration:

{% for state in states.zwave %}
{%- if state.attributes.node_id %}
{{ state.attributes.node_id }} - {{ state.entity_id  }} - {{ state.name  }} 
{%- endif %}
{%- endfor %}

I took this list offline just to be sure.

Then I made sure I set wake up interval to a few minutes on all the battery powered devices (I actually missed two, but that was entirely my own fault)

I then added the OZW Beta as described in the first post.

After the devices were added to the Integration, I did a rename of the device to the old name, according to the list of Node IDs I already had.

  • Just be aware that you should wait for all the entities to show up for each device before doing the rename. That allows the renaming to trickle down to the entites. If you do it to early, the renaminig wont work for new entites as they come in.

Here I also had to push the “wake up” button for the two missing devices. They were easy to find because I had the list of what should be there from the first step.

To test it further, I renamed a few of the entity_ids from the old ZW intergration to “switch.xxxxxx_old” and “light.xxxxxxx_old” and “binary_sensor.xxxxx_old” etc. And I then renamed the new OZW Beta entities to the old names.
I then verified that the automations worked as expected and that I could see the correct info in lovelace for the ones I tested.

I then made the leap. Removed the old ZW Integration. That effectively removed all the old entities, so it was just a matter of doing a change of the entity_id for the new ones. I guess about 150-200 entity_ids all in all that are actually used. Not the most funny part, but doable.

I did not change any options in qt-openzwave, all names and entity_ids are local to Home Assistant only.

One node did not work correctly (the switch would turn on, but not off again, for some weird reason) so I removed it and re added it using ozw-admin.

So now I can restart HA with a bit more confidence when I add new sensors and integrations etc.

7 Likes

So, I’ve tried to test the OZW a few times, but each time with little success.

Listening to OpenZWave/1/Status/ I get a single message reporting "Status": "driverReady". That’s it.

The OZW add-on log is flooded with these messages:

[20200819 10:03:37.738 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.741 CEST] [ozw.library] [info]: Info - Node: 3 Received SwitchMultiLevel report: level=0 
[20200819 10:03:37.741 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.744 CEST] [ozw.library] [debug]: Detail - Node: 3 Value Updated: old value=0, new value=0, type=byte 
[20200819 10:03:37.745 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.747 CEST] [ozw.library] [debug]: Detail - Node: 3 Changes to this value are not verified 
[20200819 10:03:37.747 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.749 CEST] [ozw.library] [debug]: Detail - Node: 3 Notification: ValueRefreshed CC: COMMAND_CLASS_SWITCH_MULTILEVEL Instance: 1 Index: 0 
[20200819 10:03:37.750 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.752 CEST] [ozw.notifications] [debug]: Notification pvt_valueRefreshed:  55148561 Thread:  0x7f8c9262cd48 
[20200819 10:03:37.754 CEST] [ozw.mqtt.publisher] [debug]: Publishing Event valueRefreshed: 55148561 
[20200819 10:03:37.802 CEST] [ozw.library] [debug]: Detail - Node: 0 Unsolicited message received while waiting for ACK. 
[20200819 10:03:37.802 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.805 CEST] [ozw.library] [debug]: Detail - Node: 3   Received: 0x01, 0x09, 0x00, 0x04, 0x00, 0x03, 0x03, 0x26, 0x03, 0x00, 0xd7 
[20200819 10:03:37.806 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.809 CEST] [ozw.library] [info]: Info - Node: 3 Received SwitchMultiLevel report: level=0 
[20200819 10:03:37.809 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.812 CEST] [ozw.library] [debug]: Detail - Node: 3 Value Updated: old value=0, new value=0, type=byte 
[20200819 10:03:37.812 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.816 CEST] [ozw.library] [debug]: Detail - Node: 3 Changes to this value are not verified 
[20200819 10:03:37.816 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.819 CEST] [ozw.library] [debug]: Detail - Node: 3 Notification: ValueRefreshed CC: COMMAND_CLASS_SWITCH_MULTILEVEL Instance: 1 Index: 0 
[20200819 10:03:37.819 CEST] [ozw.logging] [debug]: popping Log Mesages 
[20200819 10:03:37.822 CEST] [ozw.notifications] [debug]: Notification pvt_valueRefreshed:  55148561 Thread:  0x7f8c9262cd48 
[20200819 10:03:37.824 CEST] [ozw.mqtt.publisher] [debug]: Publishing Event valueRefreshed: 55148561 

Not sure where to go from here…

EDIT: OK, so I was probably just being impatient. I waited it out - took about an hour for only 4 nodes - but now I have "Status": "driverAllNodesQueried". Yay!