ZWave dead after update

[ImaginaryTango (Hal)]

I’ve been busy and have seen update notices for HAOS and HACS for a while and finally updated both yesterday - and things seemed okay, but they’re not. Here’s my version info:

I’ve been using ZWaveJS for a while. If I remember, it’s pretty much the only way to deal with some ZWave devices at this point. Before the update, everything was working fine. Now I’m having trouble with most ZWave devices. (I haven’t been able to test all and some I can’t easily test, like the outside water shutoff that I don’t want to mess with during the winter.) ZWaveJS has also disappeared from my HA sidebar on my browser. It shows on my iPhone. On my iPad, it’s there, but when I went to it, I got an error that the page wasn’t there.

On my desktop, in Chrome, I went through to the Add-Ons to try to check on ZWaveJS. I start it and it stops again. I saw it was stopped and checked and there were no log entries. I restarted it and it stopped again and I got this:

I haven’t changed anything in my setup, including not touching the hardware in weeks (maybe months). So the USB issue is, I would assume, the Aeotec Z Stick 7. Config info on that:

What information do I need to provide that I haven’t provided here and what do I need to do to get ZWave up and running again? I depend on it for a number of automations and for a few controls, including our thermostats.

[freshcoast (Z-Wave AI Bot)]

The error message indicates another process has locked the USB device. Are you running the Z-Wave JS UI add-on at the same time?

https://www.home-assistant.io/integrations/zwave_js/#can-i-switch-between-the-official-z-wave-js-add-on-and-the-z-wave-js-ui-add-on

Noticed this minor detail:

@ImaginaryTango

ZWaveJS has also disappeared from my HA sidebar on my browser.

If you have a “Z-Wave” entry in your sidebar, then it means are you are running Z-Wave JS UI at the same time.

[ImaginaryTango (Hal)]

I have the ZWJSUI Add-On as well. Neither one will start now. I remember thinking this through when I re-installed HA after a while. (It was a big mess for a number of reasons, and not issues with HA itself, so I basically had to create a new system and re-add everything.) It’s been a while, so I don’t have it fresh in my head. Here’s my add-ons, with both versions of ZWJS:

I believe, when I set everything up on this system (which, I think, was early in 2022, maybe it was in 2021), there was a good reason to NOT use the normal ZWave add-on. But as for ZWJS and ZWJSUI, I don’t remember the issues or situation with them. (Am I remembering correctly that the UI version needed the other version to work, but that I should start the UI version and never the normal version?)

I tried starting ZWJSUI and opened the web UI and got this error:

The ZWJSUI logs showed a loop - starting, failing, redoing:

And, on the sidebar issue, I do have that option checked as On for ZWJSUI.

[freshcoast (Z-Wave AI Bot)]

Again, you can only run one of the add-ons. You’ll need to pick one and use it exclusively.

[ImaginaryTango (Hal)]

So do I remove ZWJS if I use ZWJSUI? Does it matter if one is there if it’s not started?

And do I run the risk of losing any information if I delete one in favor of the other?

[freshcoast (Z-Wave AI Bot)]

If your server URL is set to host core-zwave-js then it means you aren’t using Z-Wave JS UI. Follow the instructions I linked to switch from core to Z-Wave JS UI.

If your server URL is set to host a0d7b954-zwavejs2mqtt, then it means you are using Z-Wave JS UI and you can uninstall the core add-on to avoid the conflict.

If you switch add-ons, you might have to manually wake up battery powered nodes so they complete their interviews and become functional.

[ImaginaryTango (Hal)]

I think that’s the page I looked at originally. I need some clarification here. Is there a core (with more to the name than just core, like ZWave Core, or something like that) that is a different add-on than ZWJS? I’m just trying to get a structure of all this in my head that makes sense with what little I remember from early 2022.

Looking at what you wrote, yes, a0d7b954-zwavejs2mqtt, looks like the page my iPad complained about not finding. I remember it started with what seemed like a random hex string like that, so I must have been using ZWJSUI.

I don’t mind dealing with waking up devices. That’s about 6-8 devices. Not hard to deal with.

What’s odd is that I’ve had both of these add-ons present for months without issue, so I’m guessing I just got lucky with that and the upgrade does something new. (It seems to be working okay in our barn - but I’ll fix it there, too.)

From what I’m reading, it sounds like if they’re both running, they’re both trying to grab the same device, so that’s a conflict. But do they both use the same configuration files, or d they only work with the data in the Z-stick?

[ImaginaryTango (Hal)]

Removed ZWJS and tried to start ZWJSUI - it’s still doing the same things. I rebooted, just to clear things out. No difference. Here’s the most recent log:

[freshcoast (Z-Wave AI Bot)]

@ImaginaryTango

Is there a core (with more to the name than just core, like ZWave Core, or something like that) that is a different add-on than ZWJS?

Z-Wave JS is the official (core) add-on. Z-Wave JS UI is a community add-on. https://www.home-assistant.io/integrations/zwave_js/#whats-the-benefit-of-using-z-wave-js-ui-add-on-over-the-official-add-on

Looking at what you wrote, yes, a0d7b954-zwavejs2mqtt, looks like the page my iPad complained about not finding. I remember it started with what seemed like a random hex string like that, so I must have been using ZWJSUI.

Did you go into the integration settings and confirm if that’s the configured server URL?

What’s odd is that I’ve had both of these add-ons present for months without issue, so I’m guessing I just got lucky with that and the upgrade does something new. (It seems to be working okay in our barn - but I’ll fix it there, too.)

Yes, it was just luck. Maybe there was some change in the OS that resulted in different timing of things, making making one add-on start before the other. Who knows.

From what I’m reading, it sounds like if they’re both running, they’re both trying to grab the same device, so that’s a conflict. But do they both use the same configuration files, or d they only work with the data in the Z-stick?

Add-ons do not share configuration files. The Z-Wave network is saved on the stick, so the driver information can be rebuilt, which is what the node interviews do.

Removed ZWJS and tried to start ZWJSUI - it’s still doing the same things. I rebooted, just to clear things out. No difference.

Double check that the official add-on is not running again. This would occur if your integration was configured to use the official add-on. You can follow the instructions to switch to Z-Wave JS UI outlined in the docs, to make sure the official add-on is not being installed automatically.

[ImaginaryTango (Hal)]

@freshcoast

Z-Wave JS is the official (core) add-on. Z-Wave JS UI is a community add-on.

Thank you for clarifying that for me. It helps me remember, a little better, just what was going on back when I was setting things up.

@freshcoast

Did you go into the integration settings and confirm if that’s the configured server URL?

I didn’t remember the exact URL - there were other things going on around me at the time and I forgot to take a screenshot of that. My iPad is in the barn, forgot to bring it back, so I can check tomorrow. Odd that I got that error on my iPad but not my iPhone and not in a browser. When I went to Settings->Devices And Services->Integrations->Z-Wave, I got this when I clicked on “Configure:”

All the devices are there when I click “Devices,” but they’re not operable.

@freshcoast

Yes, it was just luck. Maybe there was some change in the OS that resulted in different timing of things, making making one add-on start before the other. Who knows.

The one in the barn is still working and I think it’s updated. Right now I’m not going to press my luck. I’ll get the house system working again, then I’ll remove ZWJS from the barn system.

Personally, I think the one factor that changed had to do with migrating geese and the phase of the Moon… In other words, I get what you mean. It could be any small factor that created the issue.

@freshcoast

Add-ons do not share configuration files. The Z-Wave network is saved on the stick, so the driver information can be rebuilt, which is what the node interviews do.

Does that include things like authentication keys? The ZWJS add-on has a place to enter those, ZWJSUI does not. (Or maybe it does when you first set it up - don’t remember.) I do mention those because I have some door locks that need those. But since settings are saved on the stick, that explains to me why the settings stay the same from add on to add on.

@freshcoast

Double check that the official add-on is not running again. This would occur if your integration was configured to use the official add-on. You can follow the instructions to switch to Z-Wave JS UI outlined in the docs, to make sure the official add-on is not being installed automatically.

For now, I just did a quick check and went to add-ons to see what was included. No sign of ZWJS:

It’s late now so I’ll go through the instructions carefully tomorrow. The more I look things over, the more I think I had set everything up to use ZWJSUI in the first place and probably left ZWJS on because I wasn’t completely sure I wouldn’t be using it later. (The more I think about it, the more I think I must have set it up to use ZWJSUI, since I was used to having it in the sidebar and my memory is clear that I went at least one more step than normal for whatever I used. In other words, I know I didn’t use the default add-on.

Is it possible the official add-on, ZWJS, could be installed without it showing up in the add-ons?

What concerns me is what I see in the ZWJSUI logs. These entries are not sequential - you can see that a few posts up, but these are the ones that seem to be dealing with the error or something missing.

2023-01-03 04:49:22.195 WARN Z-WAVE: Zwavejs driver is not ready yet, statistics will be enabled on driver initialization

2023-01-03T09:49:27.450Z DRIVER Failed to initialize the driver: ZWaveError: Timeout while waiting for an ACK
from the controller (ZW0200)
at Driver.sendMessage (/opt/node_modules/zwave-js/src/lib/driver/Driver.ts
:3990:23)
at ZWaveController.identify (/opt/node_modules/zwave-js/src/lib/controller
/Controller.ts:713:33)
at Driver.initializeControllerAndNodes (/opt/node_modules/zwave-js/src/lib
/driver/Driver.ts:1208:26)
at Immediate. (/opt/node_modules/zwave-js/src/lib/driver/Driver
.ts:1015:16)

2023-01-03 04:49:27.451 INFO Z-WAVE-SERVER: Server closed
2023-01-03 04:49:27.458 INFO Z-WAVE: Client closed

[freshcoast (Z-Wave AI Bot)]

According to the screenshot, your Z-Wave integration is indeed configured to talk to the Z-Wave JS UI add-on. So simply uninstalling the official add-on should be good enough. If the add-on is no longer in the installed list, then it has been removed.

I would try stopping Z-Wave JS UI, physically removing the USB stick and re-insert it, and then start Z-Wave JS UI.

What concerns me is what I see in the ZWJSUI logs. These entries are not sequential - you can see that a few posts up, but these are the ones that seem to be dealing with the error or something missing.

The application logs are in the localtime zone, while the driver logs are in UTC.

[ImaginaryTango (Hal)]

@freshcoast

I would try stopping Z-Wave JS UI, physically removing the USB stick and re-insert it, and then start Z-Wave JS UI.

Is there a problem pulling the stick out while it’s powered up? I take it I don’t need to power down to pull it. (I know that’s not always necessary, but old habits from early computing days die hard!)

[freshcoast (Z-Wave AI Bot)]

No, it’s like any other USB device.

[ImaginaryTango (Hal)]

Got it - thanks for all your patience!

It took a little more to make it work than just unplugging and replugging it in.

A little background - and, yes, the Insteon stuff ties in with Z-Wave issues! I have HA running on a Pi in my “tech closet.” Meaning it’s near a small media server, two RAIDs, several fiber-to-ethernet converters for signals to our barn and from the Starlink dish over 800 feet from the house - plus a number of other devices. When I started with home automation, I was using Insteon and Z-Wave. At that time I used one of the Insteon PLMs that had to plug into an AC outlet and connected to my ISY controller (I think that’s from Universal Devices) by a special serial cable. All the literature on it stressed that it had to be AWAY from other electronic devices because of sensitivity. (Sounded like a weak transceiver to me.)

When I switched to HA, I got a USB PLM for Insteon. While it was easy to setup, even on an extended USB cord so it was within 10’ of another Insteon device, and line-of-sight with that device, it could not control it at all! I fixed that by plugging in the AC powered PLM 4’ away from all my various devices in that closet and the USB PLM was able to talk to the AC powered PLM, which sends out signals through our AC wiring as well as wireless. I suspect the wireless signals on my Insteon setup are wasted and only the over-the-AC ones are effective.

Due to the previous issues with interference and my overall issue that Insteon PLMs seem to have very weak signals, I got a USB hub I could plug into the Pi and I put the hub up high, above and a bit away from all the devices. That worked for my Insteon PLM (which is working now - it didn’t have any issues at all) and I figured if it helped the poor Insteon radio system, it wouldn’t hurt the Aeotec ZStick, so I put that in the same hub.

I took the ZStick out and put it back in and it didn’t make a difference. (Yes, I did restart ZWJSUI to check.) It didn’t work in any other ports in the hub (it was only 4 ports), but when I put it into the Pi directly, it worked, even though the Pi is surrounded by other computers and devices. I don’t know why the ZStick suddenly stopped working there when the Insteon stick still is, but I’m going to buy another hub and replace that one. In my eyes, it’s now questionable. (Or maybe there’s an issue with how ZWJSUI finds the ZWave device? Maybe a bug was introduced that creates problems if the device is not in the built-in hub in the computer?)

I hope I’m not going on too much, but I figure that I never know who is seeing a post and the extra information might help someone else.

[pdwonline (When a goat is on the net)]

Help… my Zwave is dead too after updating zwaveJS this morning. I am using HA and Zwave JS on docker. This is the error I found:

2024-01-05 10:04:04.870 INFO Z-WAVE: Connecting to /dev/ttyACM0
2024-01-05 10:04:04.879 INFO Z-WAVE: Zwavejs usage statistics ENABLED
2024-01-05 10:04:04.880 WARN Z-WAVE: Zwavejs driver is not ready yet, statistics will be enabled on driver initialization
2024-01-05 10:04:05.926 INFO Z-WAVE: Setting user callbacks
2024-01-05T09:04:07.597Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try in 100 ms.
2024-01-05T09:04:08.703Z CNTRLR   Failed to execute controller command after 2/3 attempts. Scheduling next try in 1100 ms.
2024-01-05T09:04:10.818Z DRIVER   Failed to initialize the driver: ZWaveError: Timeout while waiting for an ACK from the controller (ZW0200)
at Driver.sendMessage (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:5256:23)
at ZWaveController.identify (/usr/src/app/node_modules/zwave-js/src/lib/controller/Controller.ts:897:37)
at Driver.initializeControllerAndNodes (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:1432:26)
at Immediate.<anonymous> (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:1226:16)
2024-01-05 10:04:10.820 INFO Z-WAVE: Controller status: Driver: Failed to initialize the driver: ZWaveError: Timeout while waiting for an ACK from the controller (ZW0200)
at Driver.sendMessage (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:5256:23)
at ZWaveController.identify (/usr/src/app/node_modules/zwave-js/src/lib/controller/Controller.ts:897:37)
at Driver.initializeControllerAndNodes (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:1432:26)
at Immediate.<anonymous> (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:1226:16) (ZW0100)

Rebooted, replugged the USB device (Aeotec Gen5), nothing seems to work…

[freshcoast (Z-Wave AI Bot)]

Try disabling Soft Reset in the add-on settings. If you’re using a VM you’ll need this (possibly even if not). You might need to pull and re-insert the USB stick and reboot the host again, if using HA in a VM.

You should also be using the /dev/serial/by-id path, not /dev/ttyACM0, to avoid any path-change issues.

[pdwonline (When a goat is on the net)]

This is how I got this working again:
Before restarting the NUC that runs the docker containers, I paused Home Assistant.
then I rebooted my NUC. After reboot, I checked ZwaveJS and triggered a Rebuild all nodes.
That took some time with a lot of Zwave traffic (apprx. 70 Zwave nodes). Then I started the docker container for HA. All working now…

[PeteRage]

Glad you got it sorted. Prior to updating zwave or Ha docker images I copy the config to a new folder for the new version and update the docker comfiguration.

For example create a new folder called ha_2024_1_0 and copy the existent HA config in. Same thing for zwavejsui. This way if you get it a failed update, it’s easy to revert to the old versions if you don’t have time to sort it out.

Typically I only update one component at a time and usually start with zwavejs.

[pdwonline (When a goat is on the net)]

@PeteRage

revert to the old versions

I was thinking to use Git for that, but need to sort out how to configure that first…