Switching Z-Wave JS Addons with Minimal Downtime! Z-Wave JS (Official) to Z-Wave JS UI (Community)

:warning: :warning: Z-Wave JS to MQTT was renamed to Z-Wave JS UI. This guide has not been updated to reflect that change, nor sother HA UI changes. The steps are roughly the same though, so for now just replace instances of “Z-Wave JS to MQTT” with “Z-Wave JS UI”.

Intro

This guide describes how to switch from the official Z-Wave JS add-on to the community Z-Wave JS to MQTT add-on, without losing any downtime caused by the re-interviewing of nodes. Move from one add-on to the other without missing a beat!

This guide does not require any docker knowledge or command line usage, everything is done via the HA UI. However, it does require some ability to extract and modify tar files, using a file archive tool like 7-Zip.

The idea of using modified backups to restore the z-wave driver cache files was inspired by Mark#1028 on Discord. Thanks Mark#1028!

Who is this guide for?

HAOS add-on users who have already installed the official Z-Wave JS add-on, and would like more functionality that isn’t yet provided by HA. Functionality such as:

  • OTA Firmware upgrades of devices
  • Adding and remove node associations
  • Testing device config firmware files
  • 7-Days of downloadable debug logs
  • An alternate configuration UI
  • Advanced driver code execution
  • etc.

If you are still using the legacy z-wave integration, or are not using HAOS, this guide is not for you.

Otherwise, read-on into the “Why” section to see why this guide might be useful to you or not.

Why?

You say:

But freshcoast, the official Z-Wave JS docs already have a simple set of instructions on how switch add-ons. Why do I need this guide?

It’s true, those instructions are here, and I’ve recommended them before. This guide is mostly a re-hash of that (I encourage you read it) with extra detail. However, tthere is one missing piece of information that can make or break some installations.

It helps to know a little bit about how the Z-Wave JS driver works. When you add a device to your network, the driver interviews said device for important information: its identity and functionality. This information is permanently stored in a set of cache files that are loaded each time the driver restarts. Without these cache files, the identity and functionality of the devices are lost until they are re-interviewed, and during that time HA is unable to control or observe the devices.

If you simply uninstall the official add-on, and install zwavejs2mqtt, you’ve lost those cache files. In most cases this isn’t a big deal, re-interviewing can be painless. Mains-powered devices are always interviewed automatically, but battery devices are only interviewed when the wake up. That could be hours or days depending on their configuration. You might have a battery device in the attic that is difficult to access. Or your lock might be unreliable and fail interviews if the distance is too far. Etc.

These add-ons do not provide a convenient way to export or import those cache files, so this guide exists to cover that gap. If you don’t want to be required to re-interview the devices when switching add-ons, this guide is for you. After completing the guide, from the perspective of HA, there will be no noticeable change. All devices and entity customization will persist.

Prerequisites

The guide should work the same on earlier or later versions of HAOS and HA, the versions are noted here simply to mark when this guide was written. Note that HA versions prior to 2021.12.10 can wipe out your device/entity customization if you are not careful (that has since been fixed).

If you don’t know how to work with tar and tar.gz files yourself, then install 7-Zip, otherwise you’re on your own…

Guide

Step 0: Make a full backup

Of course, you should make a full backup before you do anything, just in case.

Step 1: Save important Z-Wave JS configuration

Copy the USB path and S0 and S2 keys from the Z-Wave JS add-on configuration into a text file. Ignore the network_key field, it’s a duplicate of s0_legacy.

Go to: Configuration → Add-ons, Backups & Supervisor → Add-ons → Z-Wave JS → Configuration

Step 2: Disable the Z-Wave JS Integration

Go to: Configuration → Devices & Services → … → Disable

The official Z-Wave JS add-on is automatically stopped as a result.

Step 3: Make a partial backup of the official add-on

Go to: Configuration → Add-ons, Backups & Supervisor → Backups
Click “+ CREATE BACKUP”
Set Backup name: zwave-js
Set Backup type: Partial backup
Select Add-ons / Z-Wave JS
Click CREATE

Step 4: Install zwavejs2mqtt

I won’t go into great detail here. You can follow the add-on instructions. Here is my recommended quick-setup:

Install add-on (wait for a bit)
Click Start
Click Open Web UI when available
Click the hamburger menu
Click Settings
Go to Z-Wave:

  • Set Serial Port by copying and pasting the path from the Z-Wave JS add-on
  • Set all the S0 and S2 keys via copy and paste
  • Set Log Level to Debug
  • Set Log to file to On

Go to UI:

  • Use tabs for navigation

Click Save then wait for a little bit for zwavejs2mqtt to restart.

Navigate to the z2m Control Panel. Eventually the rows will start filling up with nodes. Notice the names of the nodes and interview states. If it looks something like this:

with some node names unknown and the Interview status at ProtocolInfo, that’s fine, we’ll fix it next. If all your nodes are at Complete, congratulations! You can skip the backup modifications and restore steps, and go straight to reconfiguring the integration.

Step 5: Make a partial backup of the zwavejs2mqtt add-on

Go to Configuration → Add-ons, Backups & Supervisor → Z-Wave JS to MQTT
Click Stop.

Go to Configuration → Add-ons, Backups & Supervisor → Backups
Click “+ CREATE BACKUP”
Set Backup name: z2m
Set Backup type: Partial backup
Select Add-ons / Z-Wave JS to MQTT
Click CREATE

Step 6: Download both backups

Go to Configuration → Add-ons, Backups & Supervisor → Backups
Click on backup “zwave-js”
Select Add-ons / Z-Wave JS
Click … → Download backup
The file zwave_js.tar will be downloaded.

Go to Configuration → Add-ons, Backups & Supervisor → Backups
Click on backup “z2m”
Select Add-ons / Z-Wave JS to MQTT
Click … → Download backup
Click … → Download backup
The file z2m.tar will be downloaded.

Step 7: Transfer cache files

This is probably the hardest step. As mentioned, it requires you to have the ability to extract the cache files from the tarfile, and copy them into another tarfile. 7-Zip makes this easy, or you can do it by hand (AKA you’re on your o wn).

Open both zwave_js.tar and z2m.tar in separate 7-Zip windows.

In zwave_js.tar, navigate to the path: zwave_js.tar\.\core_zwave_js.tar.gz\coreo_zwave_js.tar\data\cache

In z2m.tar, navigate to the directory: z2m.tar\.\a0d7b954_zwavejs2mqtt.tar.gz\a0d7b954_zwavejs2mqtt.tar\data\store

Select the *.json* files in the zjs.tar file and drag them into the z2m.tar window. Say yes to any prompts asking to overwrite or modify.

Close the tar files. Say “yes” to all the prompts asking to update the z2m.tar file.

At this point, you have copied the cache files that are complete, into the zwavejs2mqtt backup, making them identical from the Z-Wave JS driver point of view.

Step 8: Restore the Z-Wave JS to MQTT partial backup

Go to Configuration → Add-ons, Backups & Supervisor → Backups
Click … → Upload backup
Choose the modified z2m.tar file
Select Add-ons / Z-Wave JS to MQTT
Click RESTORE
Click RESTORE again

Step 9: Start the Z-Wave JS to MQTT add-on

Step 9: Restart z2m

Go to Configuration → Add-ons, Backups & Supervisor
Click Z-Wave JS to MQTT
Click Start
Click Open Web UI
Wait for Z2m to fully load
Verify all nodes’ Interview status is “Complete” and the nodes have names.

If yours looks like the above screenshot, then congratulations, you were successful.

If you have some devices that are unknown or not Complete, then they probably were in that state in the first place. You’ll need to wake up battery devices to complete the interviews. While not required, it’s best if you have all the nodes in the expected state before proceeding to step 10.

Step 10: Reconfigure the Z-Wave JS Integration

This is done as described in the official docs.

Go to Configuration → Devices & Services
Click “+ ADD INTEGRATION”
Choose Z-Wave JS
UNCHECK “Use the Z-Wave JS Supervisor add-on” (!!!VERY IMPORTANT!!!)
Click SUBMIT
Enter ws://a0d7b954-zwavejs2mqtt:3000 as the URL (exactly)
Click SUBMIT
Ignore the error “Aborted / Device is already configured”
Click CLOSE

Click Show disabled integrations
Click ENABLE for Z-Wave JS
Wait for the integration to load, you may need to reload the page to see the status change
Click configure, check for “Network Connected” and the new URL referenced zwavejs2mqtt.

If you view your devices and entities, you’ll notice nothing has changed. As expected.

Step 11: Uninstall the Z-Wave JS add-on

Be sure to remove the old add-on, so it doesn’t accidentally startup and fight for control over the Z-Wave controller.

Go to Configuration → Add-ons, Backups & Supervisor
Click Z-Wave JS
Click UNINSTALL

Fin

Congratulations for making it to the end of this guide! You should now have a functioning Z-Wave JS to MQTT add-on, with the ability to perform all the extra things you wanted to do, and nothing should have changed in HA. Go crazy and update that firmware! :tada:

If you notice any errors, please let me know. Feel free to ask any clarifying questions.

14 Likes

Wow! Great guide!

Although I am too scared to mess, I have read every word. I wonder… having so much benefits, why isn’t zwavejs2mqtt not becoming the standard? Why maintain 2 containers and not bundle forces?

1 Like

As I understand it, ZWaveJSMQTT was done by an outsider, ZWaveJS is officially supported by HA. Why the 3rd party version is so far out in front of the “supported” version is another question. The supported version is still receiving development so it may catch up, but nothing is promised.

Because the “official” version is based on the “3rd party” version and not the other way around.

You’re not asked for the URL until you click Submit on the first dialog (the one with the “Use Supervisor” check box). So you click Submit twice.

Other than that, I’ve verified that this procedure works - except that I didn’t do the part about modifying the backup archives to restore the cached device capabilities. I just let the JS MQTT add-on initiate a new round of interviews and they all completed successfully. After I re-enabled the integration, my devices and entities were all there.

Be careful, there’s a lot of tweaky stuff to enter. Read and follow every warning in this procedure, including the ones about ignoring a couple of error messages.

2 Likes

Per the road map the “official” addon has most of the same support zwavejs2mqtt has, it’s the HA frontend that needs some lovin’

1 Like

You are the fucking master, I currently have 62 zwave devices and 910 entities, I was working with ZwaveJS as is the official complement, but at a certain time gave me quite long runtimes, such as turning on a group of lights, it took between 40 to 90 seconds :sleepy:, now with your tutorial I have been able to migrate successfully to Z-Wave JS to MQTT :green_heart:, although it is a bit preliminary to say that it goes better, I have noticed that with the example of the lights, it takes only 3 to 5 seconds to turn on all the lights, another issue that happened to me with ZwaveJS is that when I gave orders to the curtains and wanted to turn on the light of a room, there was also a lag in execution times that reached up to 2 minutes in some opportunities… now that doesn’t happen, everything goes in a matter of seconds… :smiling_face_with_three_hearts: but as I mentioned I have just been using Z-Wave JS to MQTT for a couple of hours. :grin:

I will update my comment in case of any event.
Regards and thanks again. :star_struck: :beers: :beers:

I’m just hitting some similar snags that are hit here

Frankly I was winging it a bit as I installed Zwave, and now trying to take a bit more logical approach…
I’ve followed these instructions and am trying to get ZwaveJS2MQTT set up again

I did have it working but nothing was being revealed to HA.
I assumed this was because I had a clash on Port 3000 so am trying to get this running on Port 3000 now - however it tells me it’s already in use…

How can I either a) find out what’s using port 3000, or b) get Zwavejs2MQTT running on a different port and things still show to HA?

EDIT. Lol, literally just managed to get this working immediately after posting by starting afresh and setting port 3001 everywhere that it asked for a port during setup

Still interested to know how to find out what is running on port 3000

@freshcoast

Thanks a TON for writing this up. I followed your instructions and got moved over seamlessly! You really saved me a ton of time and I’m so appreciative!!

1 Like

Worked great… I first tried copying the files using winrar. I could not copy/paste, had to download 7-zip for that. Everything else was great. Thank You!!!

Just out of curiosity: If you delete the integration (not the add-on!) and then add it again pointing to the new add-on do all entities stay the same? If so: How? My best guess is that entities we be re-created using the names in the add-on which might be different (esp. when entities were renamed before).

Is is not possible to reconfigure the integration to point to a different IP:Port or Docker container instead of deleting / re-adding?

You lose all entities and device IDs if you uninstall the integration. That means all renames and other customization will be permanently lost, and automations that use Device triggers/actions/conditions will be broken.

You can’t use the “re-configure” button in the integration if it cannot connect to the websocket server. So if you uninstall the core add-on, the server is no longer available, and thus it won’t allow re-configuration (an issue was submitted). The main concern of this guide is with backing up and restoring the driver cache files so you avoid having to perform re-interviews. Re-configuring the integration is still trivial with the existing method. It’s slightly less convenient then using the “re-configure” button, but accomplishes exactly the same thing.

Thanks for the insight! Depending on your setup and how the Z-Wave JS integration interprets node info in Z-Wave JS UI, it might be the case that all entities will be named differently. For larger installations that would mean having a before / after table and then go through all YAML / UI config and change the entities (or rename them back one by one). Either that doesn’t happen to much (as it hasn’t come up) or people don’t mind - but should be aware. :slight_smile:

Edit: Reading again through the FAQ (Z-Wave - Home Assistant) it appears that if the integration is disabled, second integration installed, then the first one removed, it does not screw up the entities. I just don’t get why - I would expect to have all entities coming up with _2 at the end.

If you follow this guide, or the integration docs, none of your entities change, which is the whole point. You’re not installing a “second” integration and removing the “first”, you are just re-configuring the existing one.

Ah thanks, I think I misunderstood first. With regard to Step 10 I just want to understand what happens: So when the Z-Wave JS Integration is added while the original one is disabled (not deletd), the config is being overwritten with the new config. If it’s then re-enabled, it retains all info on the original entities etc.

Does that only work because the cache files were transferred or how does the integration match its entities in HA with the devices and endpoints in the Z-Wave JS integration?

Yes, it just overwrites the existing config. The unique ID of the integration is determined by the network’s home ID. If it encounters a home ID matching the existing integration, it overwrites the settings. Everything else is preserved.

Ah, I was wondering why adding a 2nd integration for another Z-Wave network does work! So I guess for the entities there’s also a key with something like network ID, node ID, …? In any case good to know, also if you transfer the add-on to a different host…

I appreciate the guide, and it worked flawlessly. In truth, all I really thought I needed to know was where the cache files were, and this certainly had that information. But I’m glad I followed this guide anyway, because it made clear the addon disabling/enabling steps that are needed, too.

I’ve been quite frustrated with the Z-Wave JS addon. No matter what, every time I exclude a device (which I’ve been doing since I upgraded my controller), the whole thing hangs and my only choice is to restart the addon. Then I can add the device, and the next one has the exact same problem. I’ve done 30+ devices like this, wasting 5-10 minutes every time.

The MQTT version works much better and has feedback of how long until it times out–and it actually times out and continues working. I am quite glad this guide exists, because doing it all from scratch again was not something I wanted to deal with. This guide literally took me 5 minutes to follow, and everything worked the same as before–except excluding nodes and adding them actually works reliably!

I keep getting Failed to connect when trying to add it to my integrations. I do see all my zwave devices via the web ui but can’t pass the integration process to add it to hassio. I used several url including the “ws://a0d7b954-zwavejs2mqtt:3000”

Edit: It finally worked using my local ip to access my hassio prior to adding the integration. It did not work while using my hassio dashboard via duckdns url.

@freshcoast
Thank you very much for this guide.
Switching beetween Z-Wave JS to Z-Wave JS UI was done in less than five minutes without any problem.
Just a little doubt at Step 4. Clicking on Start didn’t started the add-on (the dot switched quickly from red to green and then red again) but as the Web UI became available I could proceed with the click to the hambuger menu and everything was OK.
Sorry for my English :slightly_smiling_face:

1 Like