Switching Z-Wave JS Addons with Minimal Downtime! Z-Wave JS (Official) to Z-Wave JS UI (Community)

Introduction

This guide describes how to switch from the Z-Wave JS official add-on to the Z-Wave JS UI community add-on, without losing any downtime caused by the re-interviewing of nodes.

The only requirement outside of HAOS is the ability to extract and modify tar files, using a file archive tool like 7-Zip.

:spiral_notepad: The idea of using modified backups to restore the Z-Wave driver cache files was inspired by Mark#1028 on Discord.

In this guide, Z-Wave JS UI will henceforth be abbreviated as ZUI.

Who is this guide for?

This guide is for HAOS add-on users who have already installed the official Z-Wave JS add-on and Z-Wave integration, and would like more functionality that isn’t yet provided by HA, such as:

  • Adding and remove node associations
  • Testing device config firmware files
  • 7-Days of downloadable debug logs
  • An alternate configuration UI
  • Advanced driver code execution
  • etc.

If you are not using HAOS, this guide is not for you.

When to use this guide?

The official instructions for switching add-ons are available in the integration documentation and are good enough for most people. This guide is a step-by-step version of that with an additional benefit of preserving the network information to avoid re-interviewing devices.

This guide is helpful when your network is very large, or you have many battery devices. If your network is small or you have very few battery devices, you can probably skip this.

It helps to know a little bit about how the Z-Wave JS driver works. When you add a device to your network, the driver interviews the device for its identity and functionality. This information is permanently stored in a set of cache files that are loaded each time the driver restarts. Without these cache files, the identity and functionality of the devices are lost until they are re-interviewed, and during that time HA is unable to control or observe the devices.

If you simply uninstall the official add-on and install ZUI, those cache files are lost. In most cases this isn’t a big deal, re-interviewing can be painless. Mains-powered devices are always interviewed automatically, but battery devices are only interviewed when they wake up. That could be hours or days depending on their configuration. You might have a battery device in the attic that is difficult to access. Or your lock might be unreliable and fail interviews if the distance is too far. Etc.

These official add-ons does not provide a convenient way to export or import only the cache files, so this guide exists to cover that gap. If you don’t want to re-interview the devices when switching add-ons, this guide is for you. When completed, from the perspective of HA there will be no difference.

Prerequisites

The guide should work roughly the same on earlier or later versions of HAOS and HA, however screenshots and the locations of things may change. The versions are noted here simply to mark when this guide was written.

If you do not understand the difference between integration and add-on, please read their entries in the HA glossary.

Guide

Step 0: Make a full backup

Make a full backup before you do anything, just in case.

Step 1: Save important Z-Wave JS configuration

Go to: Settings → Add-ons → Z-Wave JS → Configuration

Copy the USB path and S0 and S2 keys from the Z-Wave JS add-on configuration into a text file.

Step 2: Disable the Z-Wave JS Integration

Go to: Settings → Devices & Services → Z-Wave JS ... → Disable.
Click DISABLE in the prompt.

The Z-Wave JS add-on is automatically stopped as a result. Confirm this by going back to the add-on page and make sure it’s stopped, as indicated by the Red dot. If it’s still running, click STOP.

Step 3: Make a partial backup of the official add-on

Go to: Settings → System → Backups
Click “+ CREATE BACKUP”
Set Backup name: ZJS
Set Backup type: Partial backup
Select Add-ons / Z-Wave JS
Click CREATE

Step 4: Download the partial backup

We are going to extract the important cache files from the add-on backup to import into ZUI later.

Go to: Settings → System → Backups
Click the previously created “ZJS” backup
Click ...
Click Download backup
Save the file and open with your archive program.

The filename will be ZJS.tar. Make note of the download folder.

Step 5: Extract the Z-Wave JS driver cache files from the partial backup

Open the ZJS.tar backup file from the download folder.

The cache files are located in the directory .\core_zwave_js.tar.gz\core_zwave_js.tar\data\cache\ within the tar file, as seen here with 7-Zip:

Extract all three .jsonl files to a temporary directory, I’m using C:\temp\ha. You should have the following files extracted:

  • xxxxxxxx.jsonl
  • xxxxxxxx.metadata.jsonl
  • xxxxxxxx.values.jsonl

The xxxxxxxx filename prefix is the Z-Wave network ID, which is unique for your network.

Step 6: Install ZUI add-on

Install the add-on but do not configure it yet.

Go to: Settings → Add-ons → ADD-ON STORE
Search for Z-Wave JS UI and select it.
Click INSTALL to install it, and wait some time.
Click START. You might need to refresh the add-on page to see it started.
Click “OPEN WEB UI” to open the ZUI page.

:warning: Do not configure any settings yet!

Step 7: Upload cache files to ZUI

We will import the ZUI cache files we extracted in step 5. This step restores the Z-Wave network information in order to avoid re-interviewing devices. Make sure you are already at the ZUI page from step 6.

Click the hamburger button at the top to the left of Control Panel
Click the Store tab (folder icon) to go to Store browser. It will look like a file browser.
Click the large blue gear button at the bottom of the page
Click the orange Upload button

Click the “File” field and choose the first .jsonl cache file.
Click the UPLOAD button.

Repeat the upload steps for the two remaining .jsonl cache files. When finished, you should immediately see all three .jsonl files in the Store browser.

Make sure this is the case before proceeding.

Step 8: Configure ZUI

Make sure you are already in the ZUI web UI from steps 6 and 7.

Click the hamburger button at the top to the left of Store
Click the Settings tab (Gear icon)
Expand the Z-Wave panel

  • Set Serial Port by copying and pasting the path from step 1. Tip: you should always use a /dev/device/by-id path to avoid issues with path changes.
  • Set all the S0 and S2 keys via copy and paste from step 1
  • Set Log Level to Debug
  • Set Log to file to On

The log settings are optional and not required, but I recommend them for future troubleshooting uses.

Scroll to the bottom of the web UI and click SAVE. This will restart ZUI. This may take up to a minute or longer.

Navigate to the Control Panel via the same hamburger menu. Eventually the rows will start filling up with nodes.

If all went well, all of the nodes should have valid manufacturer, product, and product id fields, and there should be no “Unknown” ones. The interview status should also be “Complete”.

Note that ZUI will switch to a Compact mode if the window width is to narrow. Expand the window width to see the table view as above.

If you have some devices that are unknown or not Complete, then they were in that state before the migration. To fix that, you’ll need to wake up battery devices to complete the interviews, or try manually re-interviewing. While not required, it’s best if you have all the nodes in the expected state before proceeding to the next step, as this is the main point of the guide.

Step 9: Reconfigure the Z-Wave JS Integration

This process is done exactly as described in the official docs.

Go to Settings → Devices & Services
Click “+ ADD INTEGRATION”
Search for Z-Wave and choose Z-Wave. Avoid “Add Z-Wave device” and choose Z-Wave if prompted a second time.

:rotating_light: UNCHECK “Use the Z-Wave JS Supervisor add-on” (!!!VERY IMPORTANT!!!)

image

Click SUBMIT
Enter ws://a0d7b954-zwavejs2mqtt:3000 as the URL (exactly as listed here!)
Click SUBMIT

If successful the dialog will say “Device is already configured”. This is not an error, it means the existing Z-Wave network has been found.
Click CLOSE

image

Click SHOW at the top right to show the disabled Z-Wave integration.
Click ENABLE within the Z-Wave card to re-enable it.

Wait for the integration to load, but it should be nearly instant. If not, try reloading the page.

Click CONFIGURE in the Z-Wave integration card. Check for “Network Connected” and the ensure the Server URL is set to the correct ZUI URL.

If you view your devices and entities, you’ll notice nothing has changed. As expected.

Step 10: Uninstall the Z-Wave JS add-on

The final step is to remove the official add-on, so it doesn’t accidentally startup and fight for control over the Z-Wave controller.

Go to Settings → Add-ons
Click Z-Wave JS
Click UNINSTALL

The official add-on is now removed and you are left with the ZUI add-on only.

The End

Congratulations for making it to the end of this guide! You should now have a functioning ZUI add-on, with the ability to perform all the extra things you wanted to do, and nothing should have changed in HA.

If you notice any errors in this guide, please let me know with a comment.

18 Likes

Wow! Great guide!

Although I am too scared to mess, I have read every word. I wonder… having so much benefits, why isn’t zwavejs2mqtt not becoming the standard? Why maintain 2 containers and not bundle forces?

1 Like

As I understand it, ZWaveJSMQTT was done by an outsider, ZWaveJS is officially supported by HA. Why the 3rd party version is so far out in front of the “supported” version is another question. The supported version is still receiving development so it may catch up, but nothing is promised.

Because the “official” version is based on the “3rd party” version and not the other way around.

You’re not asked for the URL until you click Submit on the first dialog (the one with the “Use Supervisor” check box). So you click Submit twice.

Other than that, I’ve verified that this procedure works - except that I didn’t do the part about modifying the backup archives to restore the cached device capabilities. I just let the JS MQTT add-on initiate a new round of interviews and they all completed successfully. After I re-enabled the integration, my devices and entities were all there.

Be careful, there’s a lot of tweaky stuff to enter. Read and follow every warning in this procedure, including the ones about ignoring a couple of error messages.

2 Likes

Per the road map the “official” addon has most of the same support zwavejs2mqtt has, it’s the HA frontend that needs some lovin’

1 Like

You are the fucking master, I currently have 62 zwave devices and 910 entities, I was working with ZwaveJS as is the official complement, but at a certain time gave me quite long runtimes, such as turning on a group of lights, it took between 40 to 90 seconds :sleepy:, now with your tutorial I have been able to migrate successfully to Z-Wave JS to MQTT :green_heart:, although it is a bit preliminary to say that it goes better, I have noticed that with the example of the lights, it takes only 3 to 5 seconds to turn on all the lights, another issue that happened to me with ZwaveJS is that when I gave orders to the curtains and wanted to turn on the light of a room, there was also a lag in execution times that reached up to 2 minutes in some opportunities… now that doesn’t happen, everything goes in a matter of seconds… :smiling_face_with_three_hearts: but as I mentioned I have just been using Z-Wave JS to MQTT for a couple of hours. :grin:

I will update my comment in case of any event.
Regards and thanks again. :star_struck: :beers: :beers:

I’m just hitting some similar snags that are hit here

Frankly I was winging it a bit as I installed Zwave, and now trying to take a bit more logical approach…
I’ve followed these instructions and am trying to get ZwaveJS2MQTT set up again

I did have it working but nothing was being revealed to HA.
I assumed this was because I had a clash on Port 3000 so am trying to get this running on Port 3000 now - however it tells me it’s already in use…

How can I either a) find out what’s using port 3000, or b) get Zwavejs2MQTT running on a different port and things still show to HA?

EDIT. Lol, literally just managed to get this working immediately after posting by starting afresh and setting port 3001 everywhere that it asked for a port during setup

Still interested to know how to find out what is running on port 3000

@freshcoast

Thanks a TON for writing this up. I followed your instructions and got moved over seamlessly! You really saved me a ton of time and I’m so appreciative!!

1 Like

Worked great… I first tried copying the files using winrar. I could not copy/paste, had to download 7-zip for that. Everything else was great. Thank You!!!

Just out of curiosity: If you delete the integration (not the add-on!) and then add it again pointing to the new add-on do all entities stay the same? If so: How? My best guess is that entities we be re-created using the names in the add-on which might be different (esp. when entities were renamed before).

Is is not possible to reconfigure the integration to point to a different IP:Port or Docker container instead of deleting / re-adding?

You lose all entities and device IDs if you uninstall the integration. That means all renames and other customization will be permanently lost, and automations that use Device triggers/actions/conditions will be broken.

You can’t use the “re-configure” button in the integration if it cannot connect to the websocket server. So if you uninstall the core add-on, the server is no longer available, and thus it won’t allow re-configuration (an issue was submitted). The main concern of this guide is with backing up and restoring the driver cache files so you avoid having to perform re-interviews. Re-configuring the integration is still trivial with the existing method. It’s slightly less convenient then using the “re-configure” button, but accomplishes exactly the same thing.

Thanks for the insight! Depending on your setup and how the Z-Wave JS integration interprets node info in Z-Wave JS UI, it might be the case that all entities will be named differently. For larger installations that would mean having a before / after table and then go through all YAML / UI config and change the entities (or rename them back one by one). Either that doesn’t happen to much (as it hasn’t come up) or people don’t mind - but should be aware. :slight_smile:

Edit: Reading again through the FAQ (Z-Wave - Home Assistant) it appears that if the integration is disabled, second integration installed, then the first one removed, it does not screw up the entities. I just don’t get why - I would expect to have all entities coming up with _2 at the end.

If you follow this guide, or the integration docs, none of your entities change, which is the whole point. You’re not installing a “second” integration and removing the “first”, you are just re-configuring the existing one.

Ah thanks, I think I misunderstood first. With regard to Step 10 I just want to understand what happens: So when the Z-Wave JS Integration is added while the original one is disabled (not deletd), the config is being overwritten with the new config. If it’s then re-enabled, it retains all info on the original entities etc.

Does that only work because the cache files were transferred or how does the integration match its entities in HA with the devices and endpoints in the Z-Wave JS integration?

Yes, it just overwrites the existing config. The unique ID of the integration is determined by the network’s home ID. If it encounters a home ID matching the existing integration, it overwrites the settings. Everything else is preserved.

Ah, I was wondering why adding a 2nd integration for another Z-Wave network does work! So I guess for the entities there’s also a key with something like network ID, node ID, …? In any case good to know, also if you transfer the add-on to a different host…

I appreciate the guide, and it worked flawlessly. In truth, all I really thought I needed to know was where the cache files were, and this certainly had that information. But I’m glad I followed this guide anyway, because it made clear the addon disabling/enabling steps that are needed, too.

I’ve been quite frustrated with the Z-Wave JS addon. No matter what, every time I exclude a device (which I’ve been doing since I upgraded my controller), the whole thing hangs and my only choice is to restart the addon. Then I can add the device, and the next one has the exact same problem. I’ve done 30+ devices like this, wasting 5-10 minutes every time.

The MQTT version works much better and has feedback of how long until it times out–and it actually times out and continues working. I am quite glad this guide exists, because doing it all from scratch again was not something I wanted to deal with. This guide literally took me 5 minutes to follow, and everything worked the same as before–except excluding nodes and adding them actually works reliably!

I keep getting Failed to connect when trying to add it to my integrations. I do see all my zwave devices via the web ui but can’t pass the integration process to add it to hassio. I used several url including the “ws://a0d7b954-zwavejs2mqtt:3000”

Edit: It finally worked using my local ip to access my hassio prior to adding the integration. It did not work while using my hassio dashboard via duckdns url.

@freshcoast
Thank you very much for this guide.
Switching beetween Z-Wave JS to Z-Wave JS UI was done in less than five minutes without any problem.
Just a little doubt at Step 4. Clicking on Start didn’t started the add-on (the dot switched quickly from red to green and then red again) but as the Web UI became available I could proceed with the click to the hambuger menu and everything was OK.
Sorry for my English :slightly_smiling_face:

1 Like