All controls (switches, shutters, etc) stop working after a while

nfh · December 1, 2021, 8:08am

After the latest updates for Core and supervisor (not sure if it is a coincidence or the root cause), all my zwave controls stop working after a while (after a reboot).

The zwave network itself seems to be working, as I still get regular data packets from the energy monitor meter. Nonetheless, any switch (for lights, for example) does not do anything.

I have to reboot in order to get full control of my house again, but that does not last for long (not sure how long does it take until the controls are unresponsive).

I know it is a long shot from this description, but is anyone experiencing the same problem? Any clues of what could be causing this?

FYI: I’m using the zwavejs2mqtt addon

schford · December 1, 2021, 11:08am

I too am having this issue. Would love any insights.

schford · December 1, 2021, 11:18am

Apologies for the brain dump - if I restart the WS Server from ZWAVE JS to MQTT then the integration works again with control of lights etc.

I wonder if the issue is the latest ZWAVE JS to MQTT rather than home assistant itself.

nfh · December 1, 2021, 3:11pm

Yep, the same is happening with my home assistant installation.
I just restarted the addon and it everything is working without having to restart Home Assistant or the RPI.
I guess this is something related to the addon itself (or to some issue that was added on the latest updates of the core or supervisor).

I still did not move to the official zwaveJS integration as not only I did not want to go through all the entities to name them (I think that will be lost as soon as I switch the zwaveJS in use), but also I do not want to loose the webUI from zwavejs2mqtt. Nonetheless, I prefer stability over everything else, so maybe I’ll get my hands on this weekend.

mwav3 · December 2, 2021, 11:37am

There appears to be a bug in the underlying zwavejs server update. Zwavejs2mqtt is built on zwavejs and includes it, so I don’t think switching between the two will fix this.

After I updated my zwavejs2mqtt, which bumped zwavejs to version to 8.8.2, my zwave network crashed every night, and restarting zwavejs2mqtt was the only way to fix it. I run the docker version of zwavejs2mqtt, but the zwavejs2mqtt addon version was also updated recently. Zwavejs had a major code refactor/rewrite which appears to have created the issue. See release notes below:

I rolled back zwavejs2mqtt to the older version in docker and the problem went away. Unfortunately if you use the addon version, your only option to roll back is to restore an old snapshot.

The developers of zwavejs are aware of the issue and are working on a fix. Here’s related posts on github

github.com/zwave-js/node-zwave-js

Network Breaks Nightly Until Zwave-JS Restart Since Friday

opened 03:05PM - 01 Dec 21 UTC

kars85

### Is your problem within Home Assistant (Core or Z-Wave JS Integration)? YE…S, BUT a Home Assistant developer has told me to come here ### Is your problem within ZWaveJS2MQTT? YES, BUT a ZWaveJS2MQTT developer has told me to come here ### Checklist - [X] I have checked the [troubleshooting section](https://zwave-js.github.io/node-zwave-js/#/troubleshooting/index) and my problem is **not** described there. - [X] I have read the [changelog](https://github.com/zwave-js/node-zwave-js/blob/master/CHANGELOG.md) and my problem was **not** mentioned there. ### Describe the bug What causes the bug? Every evening (sunset - 4AM approx), automations fail to execute and Zwave devices become unavailable What do you observe? Unable to control/manage nodes. Early testing appears that if I restart the zwavejs2mqtt container, nodes become available again, but network healing is unstable. What did you expect to happen? Stable behavior. When restarting the container and initiating a heal, the heal rate is ~50%. I am attaching log archives going back to Friday when the behavior began (can't remember if Silly was enabled that far back, but I turned Silly on shortly thereafter over the weekend. Todays 2021-12-01.log**.old** is before I restarted the container this morning and began troubleshooting, so it should contain information on the network "crash". The 2021-12-01.log is a fresh log starting with a container startup, network graph view, followed by a network heal (~50% success on the heal), then me shutting down the container. I've also attached my Zooz controller's NVM backup along with my homeid.json ### Device information All devices ### How are you using `node-zwave-js`? - [X] `zwavejs2mqtt` Docker image (latest) - [ ] `zwavejs2mqtt` Docker image (dev) - [ ] `zwavejs2mqtt` Docker manually built _(please specify branches)_ - [ ] `ioBroker.zwave2` adapter _(please specify version)_ - [ ] `HomeAssistant zwave_js` integration _(please specify version)_ - [ ] `pkg` - [ ] `node-red-contrib-zwave-js` _(please specify version, double click node to find out)_ - [ ] Manually built from GitHub _(please specify branch)_ - [ ] Other _(please describe)_ ### Which branches or versions? zwavejs2mqtt: 6.0.2 zwave-js: 8.8.2 home id: 3745883352 home hex: 0xdf45a4d8 ### Did you change anything? no ### If yes, what did you change? _No response_ ### Did this work before? Yes (please describe) ### If yes, where did it work? Prior to Friday all automations and control worked as expected. ### Attach Driver Logfile [zwavejs_2021-11-30.log.gz](https://github.com/zwave-js/node-zwave-js/files/7634944/zwavejs_2021-11-30.log.gz) [zwavejs_2021-11-29.log.gz](https://github.com/zwave-js/node-zwave-js/files/7634945/zwavejs_2021-11-29.log.gz) [zwavejs_2021-11-28.log.gz](https://github.com/zwave-js/node-zwave-js/files/7634946/zwavejs_2021-11-28.log.gz) [zwavejs_2021-11-27.log.gz](https://github.com/zwave-js/node-zwave-js/files/7634947/zwavejs_2021-11-27.log.gz) [zwavejs_2021-12-01.log](https://github.com/zwave-js/node-zwave-js/files/7634949/zwavejs_2021-12-01.log) [zwavejs_2021-12-01.old.log](https://github.com/zwave-js/node-zwave-js/files/7634951/zwavejs_2021-12-01.old.log) [df45a4d8.json.txt](https://github.com/zwave-js/node-zwave-js/files/7634953/df45a4d8.json.txt) [NVM_2021-12-01.bin.txt](https://github.com/zwave-js/node-zwave-js/files/7634985/NVM_2021-12-01.bin.txt)

nfh · December 2, 2021, 2:42pm

Thanks Tim for the update on this.

Let’s hope there are news about this anytime soon. I had the addon set with autoupdate, so unfortunately I don’t have a way to revert to the previous version (in this case snapshots are not created automatically).

schford · December 2, 2021, 4:43pm

yes thanks for the update.

CO_4X4 · December 2, 2021, 5:10pm

I’m having the same issue. The Z-Wave network first slows down significantly (often taking 3-10 seconds to toggle a device) to being totally unresponsive until I reload the add-in or restart HA. I’ve been tying to diagnose and fix this so I’m glad I ran across this thread so I know that it may be fixed soon!

mwav3 · December 2, 2021, 9:27pm

Not sure if this will go anywhere, but I created a feature request post to at least implement the ability to downgrade an addon if something like this happens. Click below to vote for it if you agree

mwav3 · December 3, 2021, 3:19pm

ZwaveJS released a new version 8.8.3 that rolled back the code changes to the outgoing message handling that created the bug mentioned above.

This zwavejs update has also been pulled into Zwavejs2MQTT version 6.0.3

I can confirm all is working OK so far in my docker version of zwavejs2mqtt. For anyone running the addon version of zwavejs2mqtt, you will need to wait until these updates make its way into that - you can track that progress on the addon Github page below

I submitted an issue to hopefully move the addon update along

github.com/hassio-addons/addon-zwavejs2mqtt

Bug in underlying zwaveJS server fixed, addon needs update to current version of zwavejs2mqtt

opened 03:32PM - 03 Dec 21 UTC

mwav3

# Problem/Motivation ZwaveJS 8.8.0 had a major rewrite for outgoing message h…andling. This resulted in bugs that were crashing users Zwave networks as discussed in these threads/issues: https://community.home-assistant.io/t/all-controls-switches-shutters-etc-stop-working-after-a-while/362139 https://github.com/zwave-js/node-zwave-js/issues/3811 https://github.com/zwave-js/node-zwave-js/issues/3829 ZwaveJS released 8.8.3 to revert the outgoing message handling code changes due to these bugs, and that update has been incorporated into ZwaveJS2MQTT 6.0.3 https://github.com/zwave-js/node-zwave-js/releases/tag/v8.8.3 The current addon version of zwavejs2mqtt contains the older version of zwavejs2mqtt with the underlying zwavejs bugs and needs to be updated to the latest stable release of zwavejs2mqtt 6.0.3 ## Expected behavior Stable zwave network without crashing ## Actual behavior network crashes due to bugs mentioned above ## Steps to reproduce Steps are well documented in the corresponding github issues and community forum link above. Generally, the network fails when multiple commands are sent to the Zwave network at once (IE a "goodnight" shutoff/lock routine) ## Proposed changes This has already been fixed in the latest stable release of zwavejs2mqtt, and the new release https://github.com/zwave-js/zwavejs2mqtt/releases/tag/v6.0.3 just needs to be pulled into the addon version.

Update- the zwavejs2mqtt addon has been updated. Installing the latest version of the addon should fix these issues.

schford · December 4, 2021, 3:21pm

Im still having issues


2021-12-04 16:19:02.587 DEBUG SOCKET: New connection 4F4fLBmn_Mmoc-VVAAAN
2021-12-04 16:19:02.700 INFO APP: GET /api/auth-enabled 304 5.976 ms - -
2021-12-04 16:19:02.712 DEBUG SOCKET: Event INITED emitted to 4F4fLBmn_Mmoc-VVAAAN
2021-12-04 16:19:15.946 DEBUG SOCKET: Event ZWAVE_API emitted to Ynyssnc0z9k1hVcXAAAL
2021-12-04 16:19:15.950 INFO ZWAVE: Calling api writeValue with args: [
{ nodeId: 2, commandClass: 37, endpoint: 0, property: 'targetValue' },
true,
{},
[length]: 3
]
2021-12-04 16:19:15.951 INFO ZWAVE: Writing true to 2-37-0-targetValue
2021-12-04 16:19:20.668 ERROR ZWAVE: Error while writing true on 2-37-0-targetValue: Failed to send the message after 3 attempts (ZW0202)
2021-12-04 16:19:20.672 ERROR ZWAVE: Unable to write true on 2-37-0-targetValue

mwav3 · December 4, 2021, 6:19pm

Maybe it’s a different issue. It appears to just fail to communicate the command to Node 2. A few questions to hopefully come up with a solution:

Any improvement with the update? Ie is everything crashing or is it just sometimes a command doesn’t go through, or takes awhile? Also, was everything fine before these updates, or did you have some problems before?

What is node 2 exactly (manufacturer/model)?

Can you do a “heal node” ( not entire network heal) of just node 2? Any improvement with the healing of that node? Did it heal ok or error?

How far away from the controller is node 2?

Also it looks like these are just the zwavejs2mqtt logs. There are 2 sets of logs. Can you post the underlying zwavejs server logs? See instructions below for getting those.

https://zwave-js.github.io/zwavejs2mqtt/#/troubleshooting/generating-logs

nfh · December 16, 2021, 7:31pm

Funny enough, after upgrading to this month’s HASSIO release, as well as upgrading the operating system, I’m getting again the same odd behavior (zwave devices that send data get it right, but all controls are unresponsive).
I’m now updating zwave2mqtt to a minor version that was released today hoping that it will solve this once more (I cannot tell for sure immediately, as now the controls get unresponsive after nearly 24h).

ImNotSpam · December 20, 2021, 1:43pm

Jumping on this wagon. A couple of weeks ago, I updated to 0.31 and after a while zwave became very slow. Reverted to 0.30 and things are great. Last night, went to 0.32.1 and everything slowed down again. Unfortunately, I can’t update home assistant without updating z-wave too. So it looks like I’m stuck where I am, home assistant 2021.11.5 and Z-Wave JS to MQTT 0.30.0.

nfh · December 20, 2021, 3:50pm

Well, I tried to revert back to the previous version of zwavejs2mqtt from a recent backup that I created a few days ago just to find that when I restore it, the addon fails to start because the “addon image does not exist”. Too bad we cannot trust the backup system.

And in the meantime I have to restart my host every 3 hour in order to be able to use any ZWave device… I mean, once again devices that broadcast zwave messages (such as the energy meter) get their data transmitted, but devices that get input (such as a switch or a shutter controller) stop working completely.

I have the latest version of everything (HASSIO, operating system and zwavejs2mqtt).

nfh · December 21, 2021, 9:22am

Well, after reverting to hassio 2021.11.x the problem remains, so I guess this is once again a problem related to the add-on.
Mental reminder: start doing a full backup of the system before updating the add-on, as partial backups do not seem to do the trick to restore a previous version.

ImNotSpam · December 28, 2021, 5:03pm

I use the Google Drive Backup and Samba Backup addons. Do a couple full backups per week. If you’re not doing that and don’t have the local storage, maybe the google one would be a good option.

In any case, I’m still holding out. Hoping I can find something that says the issue is resolved.

EDIT: Fingers crossed, it looks like this may be fixed. Zwavejs2mqtt addon 0.32.2 updates zwavejs2mqtt to 6.2.0. That bumps zwavejs to 8.9.1 and I believe the bug was fixed in 8.9.0. We’ll see… This all assumes I’m reading the notes correctly.

EDIT 2: Initially had a bit of a hiccup with slow response times. Seems to be working fine though.

Malum-in-se · January 15, 2022, 7:26pm

I too am have this problem. I’m on zwavejs2mqtt 6.1.1 and zwave js 8.9.0-beta.1

I know it’s a websocket crash because my zwavejs2mqtt runs in a debian lxc container and my ha core venv runs in a separate container.

I can systmctl status both programs and they are u0 and running. When I check the logs on HA it will kick an error it can’t reach the server.

However if I leave HA amd reboot the zwavejs2mqtt lxc everything functions again for a time.

dropper · July 12, 2022, 5:06pm

I am having this problem, as well. I had originally used the zwave JS server and had this issue, so I moved to the MQTT version and the problem persists.

zwavejs2mqtt: 6.13.0
zwave-js: 9.5.0

I even upgraded my HUSBZB-1 to the latest firmware I could find (on the GitHub), with no change. I have separated my stick with a USB extension (designed for RF), just in case that was an issue.

System will run fine for a while, and then HA doesn’t seem to get any messages from MQTT. I then restart MQTT and all if fine.

nfh · July 13, 2022, 10:46am

In my case the problem is long gone.
If I’m not mistaken it was solved after one zwavejs2mqtt update.