Switching Z-Wave JS Addons with Minimal Downtime! Z-Wave JS (Official) to Z-Wave JS UI (Community)

Introduction

This guide describes how to switch from the Z-Wave JS official add-on to the Z-Wave JS UI HA Community add-on, without losing any downtime caused by the re-interviewing of nodes.

In this guide, Z-Wave JS UI will henceforth be abbreviated as ZUI.

Who is this guide for?

This guide is for HAOS add-on users who have already installed the official Z-Wave JS add-on and the companion Z-Wave integration, and desire extra functionality that isn’t provided by HA, such as:

  • Adding and removing node associations
  • Network health diagnostics (testing and route map)
  • Testing device config firmware files
  • An alternate configuration UI
  • Advanced driver code execution
  • 7-Days of downloadable debug logs From add-on v0.4.0 you can save log files!
  • etc.

If you are not using HAOS, this guide is not for you.

When to use this guide?

The official instructions for switching add-ons are available in the integration documentation and are good enough for most people. This is a step-by-step version of those with the additional benefit of preserving the network information to avoid re-interviewing devices.

This guide is helpful when your network is very large, or you have many battery devices. If your network is small or you have very few battery devices, you can probably skip this and avoid all the hassle.

It helps to know a little bit about how the Z-Wave JS driver works. When you add a device to your network, the driver interviews the device for its identity and functionality. This information is permanently stored in a multiple cache files that are loaded each time the driver restarts. Without these cache files, the identity and functionality of the devices are lost after a restart until they are re-interviewed, and during that time HA is unable to control or observe the devices.

If you uninstall the official add-on those cache files are lost. In most cases this isn’t a big deal, re-interviewing can be painless. Mains-powered devices are always interviewed automatically, but battery devices are only interviewed when they wake up. That could be hours or days depending on their configuration. You might have a battery device in the attic that is difficult to access. Or your lock might be unreliable and fail interviews if the distance is too far. Etc.

The official add-ons does not provide a convenient way to export or import only the cache files, so this guide exists to cover that gap. As of v0.4.0 of the official add-on, the driver cache files are exposed to users, which makes it simpler than ever to migrate! If you don’t want to re-interview the devices when switching add-ons, this guide is for you. When completed, from the perspective of HA there will be no difference.

Prerequisites

The guide should work roughly the same on earlier or later versions of HAOS and HA Core, however screenshots and the locations of things may change. The versions are noted here simply to mark when this guide was written.

There are no requirement outside of HAOS, but the above indicated add-ons are needed.

If you do not understand the difference between integration and add-on, please refer to their definitions in the HA glossary.

Guide

Step 0.1: Install File editor add-on

Install the File editor add-on (see installation instructions for details).

The File editor add-on is used to download the Z-Wave JS cache files to your local PC.

Step 0.2: Install Terminal & SSH add-on

Install the Terminal & SSH add-on (see installation instructions)

The Terminal & SSH add-on is used to copy the Z-Wave JS cache files from the official add-on.

:spiral_notepad: In order to see this add-on in the Store to manually install you’ll need to enable Advanced mode in your user profile first. Clicking the link above will take you there regardless of the profile setting.

You can also use the Community Add-on: Advanced SSH & Web Terminal (see docs) instead if you already have it installed, but these examples will show the official SSH add-on. The same commands can be used.

Step 0.3: Make a backup

Make a full backup of your installation, or a partial backup of the Z-Wave JS add-on, just in case.

Step 1.0: Save important Z-Wave JS configuration settings

Go to Link to Add-on: dashboard – My Home Assistant
Or manually go to: Settings → Add-ons → Z-Wave JS

Select the Configuration tab.

Copy the USB device path (1) and all security keys (can ignore network_key as it is a duplicate of s0_legacy_key) (2) somewhere safe, like a text file. These settings are required when installing ZUI.

In the Options card select the ... and Edit in YAML to see this view for easy copy and paste:

Step 1.1: Optionally export device names and locations

Both HA and ZUI have the ability to configure node (device) names and locations (areas). While names and locations in ZUI can affect the names in HA, it is not true in the opposite direction: any custom names and areas in HA will not automatically migrate to ZUI.

If you would like to migrate and name/area customization to ZUI, you’ll need to make a backup file to import into ZUI later.

Even if you skip this step, your device names and areas in HA will not be affected after the migration. This step is purely to import them into ZUI. If you have no interest in settings names/locations in ZUI you can skip this step. You can also perform this anytime in the future.

First, copy and paste this template code into the Dev Tools template editor:

Open your Home Assistant instance and show your template developer tools.

{%- set ns = namespace(nodes=[], export={}) %}
{%- for dev_id in integration_entities('zwave_js') | map('device_id') | unique %}
  {%- set node_id = (device_attr(dev_id, 'identifiers') | first | last).split('-')[1] %}
  {%- set ns.nodes = ns.nodes + [(node_id | int, dev_id)] %}
{%- endfor %}
{%- for node_id, dev_id in ns.nodes | sort(attribute='0') %}
  {%- set node = {(node_id | string): {}} %}
  {%- set name = device_attr(dev_id, 'name_by_user') %}
  {%- set location = area_name(dev_id) %}
  {%- if name and location %}
    {%- set node = {(node_id | string): {"name": name, "loc": location}} %}
  {%- elif name %}
    {%- set node = {(node_id | string): {"name": name}} %}
  {%- elif location %}
    {%- set node = {(node_id | string): {"loc": location}} %}
  {%- endif %}
  {%- set ns.export = dict(ns.export.items(), **node) %}
{%- endfor %}

{{ ns.export | to_json }}

The output will be a JSON document with the node names and locations for the file format supported by ZUI. Copy and paste the JSON text in the Result panel and save it the contents into a file named nodes.json. Save this file for restoring into ZUI later.

Step 2: Disable the Z-Wave JS Integration

Open the Z-Wave integration page:
Open your Home Assistant instance and show an integration.

Or manually navigate the UI to: Settings → Devices & services → Z-Wave

Select ...Disable, then select DISABLE on the next prompt.

The Z-Wave JS add-on is automatically stopped as a result. Confirm this by going back to the add-on page and make sure it’s stopped, as indicated by the red dot. If it’s still running, click STOP. You may need to refresh the page to see the correct status.

Open your Home Assistant instance and show the dashboard of an add-on.

Step 3: Copy the Z-Wave JS cache files

Open the SSH web UI:
Open your Home Assistant instance and Open the ingress URL of an add-on.
Or manually navigate the UI to: Settings → Add-ons → Terminal & SSH, and select OPEN WEB-UI.

You will enter a terminal prompt. Enter the following command (copy text and try using middle-mouse button to paste):

apk add zip && zip -j /homeassistant/zjs.zip /addon_configs/core_zwave_js/cache/*.jsonl

This will create a zjs.zip zip file in your HA config directory which contains the Z-Wave JS cache files.

:spiral_notepad: The name of the *.jsonl files are unique for every Z-Wave network.

Step 4: Download the Z-Wave JS cache zip file

We need to download the zip file created above to restore to ZUI later. We can use the File editor add-on to do this.

Open the File editor:
Open your Home Assistant instance and Open the ingress URL of an add-on.
Or manually navigate the UI to: Settings → Add-ons → File editor and select OPEN WEB-UI.

Select the folder icon in the upper left (1) to show the files in your HA config directory.

Scroll down the directory listing and find the zjs.zip file we created previously. Select the ... next to the filename and Download.

Make note of where you saved the download as you will need it later.

Step 5: Install ZUI add-on

Install the Z-Wave JS UI add-on but do not configure it yet. The current version of the add-on used here is v3.18.0.

Open your Home Assistant instance and show the dashboard of an add-on.
Or manually go to: Settings → Add-ons → ADD-ON STORE, search for Z-Wave JS UI and select it.

  1. Select INSTALL to install it, this may take several minutes.
  2. Select START. You might need to refresh the add-on page to see it started.
  3. Select OPEN WEB UI to open the ZUI web UI, or click: Open your Home Assistant instance and Open the ingress URL of an add-on.

:warning: Do not configure any settings yet!

Step 6: Import cache files into ZUI

We will import the Z-Wave JS cache files we downloaded in step 4. This is the crucial step that restores the Z-Wave network information in order to avoid re-interviewing devices. Make sure you are still in the ZUI web UI from step 5.

  1. Select the “hamburger button” at the top to the left of Control Panel
  2. Select the Store tab (folder icon) to go to Store browser. It will look like a file browser.
  3. Select the large blue gear button at the bottom of the page
  4. Select the green Restore button

  1. Select the Zip file field and pick the zjs.zip file downloaded earlier.
  2. Select the RESTORE button.

After the restore, you should immediately see all three of the .jsonl cache files in the Store browser.

Make sure this is the case before proceeding. If you want, you can click on a file to inspect its contents.

Step 7: Configure ZUI

Make sure you are still in the ZUI web UI from steps 5 and 6.

Go to the Z-Wave settings:

  1. Click the hamburger button at the top to the left of Store
  2. Click the Settings tab (Gear icon)
  3. Expand the Z-Wave panel

Configure the Z-Wave settings:

  1. Set Serial Port by using the same device path from step 1. In most cases should be auto-detected and displayed in the pull-down menu. Tip: you should always use a /dev/device/by-id path to avoid issues with path changes.
  2. Set all the S0 and S2 keys via copy and paste from step 1. Be sure to match the keys exactly (S2 Authenticated to S2 Authenticated, etc.).

Optional settings:

  1. Set Log Level to Debug
  2. Set Log to file to On

The log settings are recommended for future troubleshooting of issues. If your storage is write-sensitive, such as an SD card, you may want to leave them disabled.

If you are using a VM and a 500-series controller, you may need to disable the “Soft Reset” setting. A Soft Reset of a 500-series will cause a USB reset, and the VM will lose connection. If pass-through is not configured for hot-plugging, the USB stick will never re-connect and Z-Wave JS will fail to start. Do not disable Soft Reset for 700/800-series controllers, they do not have this issue.

:spiral_notepad: Don’t forget to scroll to the bottom of the web UI and click SAVE. Saving will restart ZUI and it may take up to a minute or longer to load depending on the size of your Z-Wave network.

:warning: DO NOT change any of the settings in the Home Assistant settings panel. These are already configured to work with the Z-Wave integration.

Step 8.0: Verify configuration

Navigate to the Control Panel via the same hamburger menu. Eventually the node table rows will start filling up with all of the nodes.

A correct migration will show all of the nodes with valid manufacturer, product, and product code fields (1), and there should be no “Unknown” ones. The interview status should also be “Complete” (2).

Note that ZUI will switch to a compact mode if the window width is too narrow. Expand the window width to see the table view as shown above.

If you have some devices that are unknown or not Complete, then they would have been in that state before the migration. To fix that, you’ll need to wake up battery devices to complete the interviews, or try manually re-interviewing. While not required, it’s best if you have all the nodes in the Complete interview state before proceeding to the next step, as this is the main point of the guide.

8.1: Import node names and locations

If you exported your HA device (node) names and areas (locations) in step 1.1, now is the time to import them. From the Control Panel page in ZUI:

  1. Select the blue FAB (floating action button) in the bottom right
  2. Select the :magic_wand: (wand) icon, Advanced actions
  3. Select IMPORT under Backup
  4. Select YES when prompted to override existing
  5. You should see a green “Configuration import successfully” notification if it was successful
  6. Exit from the advanced actions

image

The names and location fields for the nodes should be updated to reflect the HA device names and areas

:spiral_notepad: In the future, always add nodes from ZUI to keep it and HA in-sync. When adding a node (either classic inclusion or SmartStart) ZUI allows you to pre-configure names and locations. If set, during inclusion HA will see the pre-configured name & location and assign them as the device name & area. If you include from HA and configure there, ZUI will not see the changes.

Step 9: Reconfigure the Z-Wave JS Integration

This process is done exactly as described in the official docs.

Go to the integrations dashboard:
Open your Home Assistant instance and show your integrations.
Or manually navigate the UI to: Settings → Devices & services

  1. Select + ADD INTEGRATION
  2. Search for Z-Wave and choose Z-Wave. Avoid “Add Z-Wave device” and choose Z-Wave if prompted a second time.
  3. :rotating_light: UNCHECK “Use the Z-Wave JS Supervisor add-on” (!!!CRITICAL!!!)

image

  1. Select SUBMIT

Enter ws://a0d7b954-zwavejs2mqtt:3000 as the URL (copy and paste the exact URL listed here, do not include any whitespace!) and select SUBMIT.

image

If successful the dialog will say “Device is already configured”. This is not an error, it means the existing Z-Wave network has successfully been found. Select CLOSE.

image

:spiral_notepad: The integration is still disabled though, so we need to turn it back on.

Go to the Z-Wave integration page:
Open your Home Assistant instance and show an integration.

Under Hubs it will show disabled by user.

Select the ENABLE button to re-enable it. Then click on the CONFIGURE button to validate (or go to Link to Z-Wave JS configuration – My Home Assistant).

Wait for the integration to load, but it should be nearly instant. If not, try reloading the page.

Ensure “Z-Wave Network Connected” (1) is shown and the the Server URL (2) is set to the above ZUI URL.

If you view your devices and entities, you’ll notice nothing has changed (e.g. device and entity names), as expected.

Step 10: Uninstall the Z-Wave JS add-on

The final step is to remove the official add-on. Both add-ons cannot be running and using the same Z-Wave controller at the same time.

Go to Link to Add-on: dashboard – My Home Assistant
Or manually go to: Settings → Add-ons → Z-Wave JS

Select UNINSTALL and UNINSTALL again in the popup dialog

The official add-on is now completely removed.

The End

Congratulations for making it to the end of this guide! You should now have a fully functioning ZUI add-on, with the ability to perform all the extra things you wanted to do, and nothing should have changed in HA.

If you notice any errors in this guide, please let me know with a comment.

58 Likes

Wow! Great guide!

Although I am too scared to mess, I have read every word. I wonder… having so much benefits, why isn’t zwavejs2mqtt not becoming the standard? Why maintain 2 containers and not bundle forces?

2 Likes

As I understand it, ZWaveJSMQTT was done by an outsider, ZWaveJS is officially supported by HA. Why the 3rd party version is so far out in front of the “supported” version is another question. The supported version is still receiving development so it may catch up, but nothing is promised.

Because the “official” version is based on the “3rd party” version and not the other way around.

You’re not asked for the URL until you click Submit on the first dialog (the one with the “Use Supervisor” check box). So you click Submit twice.

Other than that, I’ve verified that this procedure works - except that I didn’t do the part about modifying the backup archives to restore the cached device capabilities. I just let the JS MQTT add-on initiate a new round of interviews and they all completed successfully. After I re-enabled the integration, my devices and entities were all there.

Be careful, there’s a lot of tweaky stuff to enter. Read and follow every warning in this procedure, including the ones about ignoring a couple of error messages.

2 Likes

Per the road map the “official” addon has most of the same support zwavejs2mqtt has, it’s the HA frontend that needs some lovin’

1 Like

You are the fucking master, I currently have 62 zwave devices and 910 entities, I was working with ZwaveJS as is the official complement, but at a certain time gave me quite long runtimes, such as turning on a group of lights, it took between 40 to 90 seconds :sleepy:, now with your tutorial I have been able to migrate successfully to Z-Wave JS to MQTT :green_heart:, although it is a bit preliminary to say that it goes better, I have noticed that with the example of the lights, it takes only 3 to 5 seconds to turn on all the lights, another issue that happened to me with ZwaveJS is that when I gave orders to the curtains and wanted to turn on the light of a room, there was also a lag in execution times that reached up to 2 minutes in some opportunities… now that doesn’t happen, everything goes in a matter of seconds… :smiling_face_with_three_hearts: but as I mentioned I have just been using Z-Wave JS to MQTT for a couple of hours. :grin:

I will update my comment in case of any event.
Regards and thanks again. :star_struck: :beers: :beers:

I’m just hitting some similar snags that are hit here

Frankly I was winging it a bit as I installed Zwave, and now trying to take a bit more logical approach…
I’ve followed these instructions and am trying to get ZwaveJS2MQTT set up again

I did have it working but nothing was being revealed to HA.
I assumed this was because I had a clash on Port 3000 so am trying to get this running on Port 3000 now - however it tells me it’s already in use…

How can I either a) find out what’s using port 3000, or b) get Zwavejs2MQTT running on a different port and things still show to HA?

EDIT. Lol, literally just managed to get this working immediately after posting by starting afresh and setting port 3001 everywhere that it asked for a port during setup

Still interested to know how to find out what is running on port 3000

@freshcoast

Thanks a TON for writing this up. I followed your instructions and got moved over seamlessly! You really saved me a ton of time and I’m so appreciative!!

1 Like

Worked great… I first tried copying the files using winrar. I could not copy/paste, had to download 7-zip for that. Everything else was great. Thank You!!!

Just out of curiosity: If you delete the integration (not the add-on!) and then add it again pointing to the new add-on do all entities stay the same? If so: How? My best guess is that entities we be re-created using the names in the add-on which might be different (esp. when entities were renamed before).

Is is not possible to reconfigure the integration to point to a different IP:Port or Docker container instead of deleting / re-adding?

You lose all entities and device IDs if you uninstall the integration. That means all renames and other customization will be permanently lost, and automations that use Device triggers/actions/conditions will be broken.

You can’t use the “re-configure” button in the integration if it cannot connect to the websocket server. So if you uninstall the core add-on, the server is no longer available, and thus it won’t allow re-configuration (an issue was submitted). The main concern of this guide is with backing up and restoring the driver cache files so you avoid having to perform re-interviews. Re-configuring the integration is still trivial with the existing method. It’s slightly less convenient then using the “re-configure” button, but accomplishes exactly the same thing.

Thanks for the insight! Depending on your setup and how the Z-Wave JS integration interprets node info in Z-Wave JS UI, it might be the case that all entities will be named differently. For larger installations that would mean having a before / after table and then go through all YAML / UI config and change the entities (or rename them back one by one). Either that doesn’t happen to much (as it hasn’t come up) or people don’t mind - but should be aware. :slight_smile:

Edit: Reading again through the FAQ (Z-Wave - Home Assistant) it appears that if the integration is disabled, second integration installed, then the first one removed, it does not screw up the entities. I just don’t get why - I would expect to have all entities coming up with _2 at the end.

If you follow this guide, or the integration docs, none of your entities change, which is the whole point. You’re not installing a “second” integration and removing the “first”, you are just re-configuring the existing one.

Ah thanks, I think I misunderstood first. With regard to Step 10 I just want to understand what happens: So when the Z-Wave JS Integration is added while the original one is disabled (not deletd), the config is being overwritten with the new config. If it’s then re-enabled, it retains all info on the original entities etc.

Does that only work because the cache files were transferred or how does the integration match its entities in HA with the devices and endpoints in the Z-Wave JS integration?

Yes, it just overwrites the existing config. The unique ID of the integration is determined by the network’s home ID. If it encounters a home ID matching the existing integration, it overwrites the settings. Everything else is preserved.

Ah, I was wondering why adding a 2nd integration for another Z-Wave network does work! So I guess for the entities there’s also a key with something like network ID, node ID, …? In any case good to know, also if you transfer the add-on to a different host…

I appreciate the guide, and it worked flawlessly. In truth, all I really thought I needed to know was where the cache files were, and this certainly had that information. But I’m glad I followed this guide anyway, because it made clear the addon disabling/enabling steps that are needed, too.

I’ve been quite frustrated with the Z-Wave JS addon. No matter what, every time I exclude a device (which I’ve been doing since I upgraded my controller), the whole thing hangs and my only choice is to restart the addon. Then I can add the device, and the next one has the exact same problem. I’ve done 30+ devices like this, wasting 5-10 minutes every time.

The MQTT version works much better and has feedback of how long until it times out–and it actually times out and continues working. I am quite glad this guide exists, because doing it all from scratch again was not something I wanted to deal with. This guide literally took me 5 minutes to follow, and everything worked the same as before–except excluding nodes and adding them actually works reliably!

I keep getting Failed to connect when trying to add it to my integrations. I do see all my zwave devices via the web ui but can’t pass the integration process to add it to hassio. I used several url including the “ws://a0d7b954-zwavejs2mqtt:3000”

Edit: It finally worked using my local ip to access my hassio prior to adding the integration. It did not work while using my hassio dashboard via duckdns url.

@freshcoast
Thank you very much for this guide.
Switching beetween Z-Wave JS to Z-Wave JS UI was done in less than five minutes without any problem.
Just a little doubt at Step 4. Clicking on Start didn’t started the add-on (the dot switched quickly from red to green and then red again) but as the Web UI became available I could proceed with the click to the hambuger menu and everything was OK.
Sorry for my English :slightly_smiling_face:

1 Like