Upgrades today have caused constant unresponsive warnings, resulting in repetitive driver restarts and z wave interruptions

Yes it’s being worked on:
:rescue_worker_helmet: Version 12: Problems communicating with the controller :rescue_worker_helmet: · Issue #6341 · zwave-js/node-zwave-js (github.com)

This specific issue you’ve linked to is not something that is “being worked on”. It is a set of instructions telling users how to fix problems related to migrating to v12.

Apologies. There are four so far I’ve read this morning in both JS and JsUI that seem to be interconnected. This seemed to be the most concise summary. Which one is the root?
(and why were all confused)

No apology needed, just clarifying as it is indeed all confusing. I don’t think there is a single root issue, and not everyone has a problem even.

Here’s my take on the current situation.

First, I think many of the reported problems are related to the new soft-reset behavior and 500-series controllers. Configuring your VM (if using one) and using the by-id paths, as described in the linked issue, will go a long way. If you are unable to perform those fixes, then both add-ons have the ability to manually disable soft-reset as of right now. You can find reports that doing either of those have solved some people’s problems.

Second, I think users are affected in different ways by the new behavior for handling unresponsive controllers.

Some subset of users seem to have a supposedly unresponsive 500-series controller (according to the logs) despite either fixing the above soft-reset config requirements or disabling soft-reset entirely. So there seems to an unsolved problem there. Yet not everyone is affected by this. I don’t think you will know until you try.

If you have a 700-series, you may be affected by a possible 7.19 firmware issue that supposedly causes unresponsive controllers. The workaround here is also the automatic soft-reset recovery. This has been reported to Silicon Labs, but I personally haven’t heard of any update. I’m staying on 7.18 for now.

There is also a fix in v12.0.2 related to unresponsive controllers (jammed). The newest version of the official add-on has this version of the driver, but the ZUI add-on does not yet (at this very moment).

So prior to upgrading any add-ons, if you’re using a VM and a 500-series controller, I’d says it’s essential to get USB pass-through working completely. Second, use the /dev/serial/by-id paths when available whether VM or bare-metal. These are already provided by HAOS. If you are using Docker (or Snap?), make sure your OS is providing the path for you. There are some versions of systemd which broke this, updating can solve that. If none of those are possible, as a last resort you can disable soft-reset too.

So all I can really advise is to make sure you have a backup of any working add-on so you can rollback if needed, and be aware that HA 2023.10 does having breaking requirements (needs official add-on 0.1.91, ZUI add-on 2.0.0, or standalone ZUI 9.0.0, all of which use driver v12), so keep that in mind if you plan an HA update this week.

3 Likes

We have an eta on 12.0.2? Sounds like that’s where ZUI users want to be before next week.

Only the add-on maintainer can answer that, community add-ons are a one person show. Probably busy with the HA release.

HA release is tomorrow, not next week.

I don’t really know how prevalent the issue fixed in 12.0.2 is.

1 Like

I have access to the repo, so I’m going to try to push the release as long as we can get CI to pass

EDIT: pushed!

2 Likes

Will this create an increment update in HA ZUI from 2.0.1 i’m currently seeing available?

1 Like

Z-Wave JS UI 2.0.2 was released 12hrs ago according to Release v2.0.2 · hassio-addons/addon-zwave-js-ui · GitHub

I don’t know how often HA checks for a new addon?

The add-on was updated, but the add-on repository still needs to be updated for HA to know about it. (which hasn’t happened yet).

2 Likes

My zwave suddenly isn’t working and I’ve got the following error in my logs:

Failed to set the Z-Wave JS add-on options: not a valid value for dictionary value @ data['options']. Got {'device': '/dev/serial/by-id/usb-Silicon_Labs_HubZ_Smart_Home_Controller

Is this related?

edit: I was able to resolve this. Somehow the zwave addon got set to not start at boot. I did not change this, but once I started the addon and re-set the option to start at boot - everything seems fine.

1 Like

Lets hope it happens before supervisor 2023.10.1 thats now showing available automatically updates.

There’s no relationship with Supervisor, unless that is requiring a minimum HA version.

Yup, just updated Supervisor and old Zwave JS UI still works fine…

JS ui 2.0.2 is now showing available.
Once installed along with
supervisor 2023.10
and soft reset disabled
and controller unplugged and replugged
and host rebooted (raspberry pi and no VM involved)

Issue is the same, constant driver restarts and loss of zwave while everything is rescanned.

I do have logs, from before i disabled soft reset, after i disabled it and replugged the (aeotec gen5+) controller, and also after the JS UI 2.0.2 update… but watching the live logs, everything looks exactly the same.
Update, logs below.
I am aware I have one dead node on the network (node 112). After trying everything else, I can only assume that’s causing this catastrophic failure.
After restore, everything working perfectly again, with known dead node causing no issues

2023-10-04T19:10:46.144Z CNTRLR « [Node 004] ping successful
2023-10-04T19:10:48.106Z CNTRLR   [Node 036] The node is alive.
2023-10-04T19:10:48.154Z CNTRLR   Retrieving priority route to node 36...
2023-10-04T19:10:48.156Z CNTRLR   [Node 036] The node is ready to be used
2023-10-04T19:10:48.171Z CNTRLR « [Node 036] ping successful
2023-10-04T19:10:51.193Z CNTRLR   [Node 073] The node is alive.
2023-10-04T19:10:51.290Z CNTRLR   Retrieving priority route to node 73...
2023-10-04T19:10:51.295Z CNTRLR   [Node 073] The node is ready to be used
2023-10-04T19:10:51.327Z CNTRLR « [Node 073] ping successful
2023-10-04T19:10:54.918Z CNTRLR   [Node 112] The node did not respond after 1 attempts, it is presumed dead
2023-10-04T19:10:54.921Z CNTRLR   [Node 112] The node is dead.
2023-10-04T19:10:54.922Z CNTRLR   All nodes are ready to be used
2023-10-04T19:10:54.927Z CNTRLR   [Node 112] ping failed: The node did not acknowledge the command (ZW0204)
2023-10-04T19:10:54.944Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try i
                                  n 100 ms.
2023-10-04T19:10:55.148Z DRIVER     no handlers registered!
2023-10-04T19:11:42.208Z CNTRLR » [Node 112] pinging the node...
2023-10-04T19:12:00.154Z CNTRLR   The controller is unresponsive
2023-10-04T19:12:00.163Z DRIVER   Attempting to recover unresponsive controller...
2023-10-04T19:12:00.170Z CNTRLR   The controller does not support soft reset or the soft reset feature has been 
                                  disabled with a config option or the ZWAVEJS_DISABLE_SOFT_RESET environment va
                                  riable.
2023-10-04T19:12:00.175Z DRIVER   Recovering unresponsive controller failed. Restarting the driver...
2023-10-04T19:12:00.185Z CNTRLR   [Node 014] Deleting SUC return route failed: Timeout while waiting for a callb
                                  ack from the controller (ZW0200)
2023-10-04T19:12:00.188Z CNTRLR   [Node 014] Assigning SUC return route failed: The driver is not ready or has b
                                  een destroyed (ZW0103)
2023-10-04T19:12:01.249Z DRIVER   ███████╗ ██╗    ██╗  █████╗  ██╗   ██╗ ███████╗             ██╗ ███████╗
                                  ╚══███╔╝ ██║    ██║ ██╔══██╗ ██║   ██║ ██╔════╝             ██║ ██╔════╝
                                    ███╔╝  ██║ █╗ ██║ ███████║ ██║   ██║ █████╗   █████╗      ██║ ███████╗
                                   ███╔╝   ██║███╗██║ ██╔══██║ ╚██╗ ██╔╝ ██╔══╝   ╚════╝ ██   ██║ ╚════██║
                                  ███████╗ ╚███╔███╔╝ ██║  ██║  ╚████╔╝  ███████╗        ╚█████╔╝ ███████║
                                  ╚══════╝  ╚══╝╚══╝  ╚═╝  ╚═╝   ╚═══╝   ╚══════╝         ╚════╝  ╚══════╝
2023-10-04T19:12:01.250Z DRIVER   version 12.0.2
2023-10-04T19:12:01.251Z DRIVER   
2023-10-04T19:12:02.281Z CONFIG   version 12.0.2
2023-10-04T19:12:03.767Z CNTRLR   querying Serial API capabilities...
2023-10-04T19:12:03.783Z CNTRLR   received API capabilities:
                                    firmware version:    1.2
                                    manufacturer ID:     0x86
                                    product type:        0x01
                                    product ID:          0x5a
                                    supported functions: 
                                    · GetSerialApiInitData (0x02)
                                    · SetApplicationNodeInformation (0x03)
                                    · ApplicationCommand (0x04)
                                    · GetControllerCapabilities (0x05)
                                    · SetSerialApiTimeouts (0x06)
                                    · GetSerialApiCapabilities (0x07)
                                    · SoftReset (0x08)
                                    · GetProtocolVersion (0x09)
                                    · SerialAPIStarted (0x0a)
                                    · SerialAPISetup (0x0b)
                                    · SetRFReceiveMode (0x10)
                                    · UNKNOWN_FUNC_SET_SLEEP_MODE (0x11)
                                    · FUNC_ID_ZW_SEND_NODE_INFORMATION (0x12)
                                    · SendData (0x13)
                                    · SendDataMulticast (0x14)
                                    · GetControllerVersion (0x15)
                                    · SendDataAbort (0x16)
                                    · FUNC_ID_ZW_R_F_POWER_LEVEL_SET (0x17)
                                    · FUNC_ID_ZW_GET_RANDOM (0x1c)
                                    · GetControllerId (0x20)
                                    · UNKNOWN_FUNC_MEMORY_GET_BYTE (0x21)
                                    · UNKNOWN_FUNC_MEMORY_PUT_BYTE (0x22)
                                    · UNKNOWN_FUNC_MEMORY_GET_BUFFER (0x23)
                                    · UNKNOWN_FUNC_MEMORY_PUT_BUFFER (0x24)
                                    · EnterBootloader (0x27)
                                    · UNKNOWN_FUNC_UNKNOWN_0x28 (0x28)
                                    · GetNVMId (0x29)
                                    · ExtNVMReadLongBuffer (0x2a)
                                    · ExtNVMWriteLongBuffer (0x2b)
                                    · ExtNVMReadLongByte (0x2c)
                                    · ExtExtWriteLongByte (0x2d)
                                    · NVMOperations (0x2e)
                                    · undefined (0x2f)
                                    · undefined (0x37)
                                    · undefined (0x38)
                                    · UNKNOWN_FUNC_ClearNetworkStats (0x39)
                                    · UNKNOWN_FUNC_GetNetworkStats (0x3a)
                                    · GetBackgroundRSSI (0x3b)
                                    · undefined (0x3c)
                                    · UNKNOWN_FUNC_RemoveNodeIdFromNetwork (0x3f)
                                    · GetNodeProtocolInfo (0x41)
                                    · HardReset (0x42)
                                    · FUNC_ID_ZW_REPLICATION_COMMAND_COMPLETE (0x44)
                                    · FUNC_ID_ZW_REPLICATION_SEND_DATA (0x45)
                                    · AssignReturnRoute (0x46)
                                    · DeleteReturnRoute (0x47)
                                    · RequestNodeNeighborUpdate (0x48)
                                    · ApplicationUpdateRequest (0x49)
                                    · AddNodeToNetwork (0x4a)
                                    · RemoveNodeFromNetwork (0x4b)
                                    · FUNC_ID_ZW_CREATE_NEW_PRIMARY (0x4c)
                                    · FUNC_ID_ZW_CONTROLLER_CHANGE (0x4d)
                                    · AssignPriorityReturnRoute (0x4f)
                                    · FUNC_ID_ZW_SET_LEARN_MODE (0x50)
                                    · AssignSUCReturnRoute (0x51)
                                    · FUNC_ID_ZW_REQUEST_NETWORK_UPDATE (0x53)
                                    · SetSUCNodeId (0x54)
                                    · DeleteSUCReturnRoute (0x55)
                                    · GetSUCNodeId (0x56)
                                    · UNKNOWN_FUNC_SEND_SUC_ID (0x57)
                                    · AssignPrioritySUCReturnRoute (0x58)
                                    · FUNC_ID_ZW_EXPLORE_REQUEST_INCLUSION (0x5e)
                                    · undefined (0x5f)
                                    · RequestNodeInfo (0x60)
                                    · RemoveFailedNode (0x61)
                                    · IsFailedNode (0x62)
                                    · ReplaceFailedNode (0x63)
                                    · UNKNOWN_FUNC_UNKNOWN_0x66 (0x66)
                                    · UNKNOWN_FUNC_UNKNOWN_0x67 (0x67)
                                    · FirmwareUpdateNVM (0x78)
                                    · GetRoutingInfo (0x80)
                                    · UNKNOWN_FUNC_LOCK_ROUTE_RESPONSE (0x90)
                                    · GetPriorityRoute (0x92)
                                    · SetPriorityRoute (0x93)
                                    · UNKNOWN_FUNC_UNKNOWN_0x98 (0x98)
                                    · FUNC_ID_APPLICATION_SLAVE_COMMAND_HANDLER (0xa1)
                                    · UNKNOWN_FUNC_UNKNOWN_0xB4 (0xb4)
                                    · UNKNOWN_FUNC_WATCH_DOG_ENABLE (0xb6)
                                    · UNKNOWN_FUNC_WATCH_DOG_DISABLE (0xb7)
                                    · UNKNOWN_FUNC_WATCH_DOG_KICK (0xb8)
                                    · UNKNOWN_FUNC_UNKNOWN_0xB9 (0xb9)
                                    · UNKNOWN_FUNC_RF_POWERLEVEL_GET (0xba)
                                    · UNKNOWN_FUNC_GET_LIBRARY_TYPE (0xbd)
                                    · UNKNOWN_FUNC_SEND_TEST_FRAME (0xbe)
                                    · UNKNOWN_FUNC_GET_PROTOCOL_STATUS (0xbf)
                                    · FUNC_ID_ZW_SET_PROMISCUOUS_MODE (0xd0)
                                    · FUNC_ID_PROMISCUOUS_APPLICATION_COMMAND_HANDLER (0xd1)
                                    · UNKNOWN_FUNC_UNKNOWN_0xD2 (0xd2)
                                    · UNKNOWN_FUNC_UNKNOWN_0xD3 (0xd3)
                                    · UNKNOWN_FUNC_UNKNOWN_0xD4 (0xd4)
                                    · undefined (0xee)
                                    · UNKNOWN_FUNC_UNKNOWN_0xEF (0xef)
2023-10-04T19:12:03.784Z CNTRLR   querying version info...
2023-10-04T19:12:03.795Z CNTRLR   received version info:
                                    controller type: Static Controller
                                    library version: Z-Wave 6.07
2023-10-04T19:12:03.796Z CNTRLR   querying protocol version info...
2023-10-04T19:12:03.806Z CNTRLR   received protocol version info:
                                    protocol type:             Z-Wave
                                    protocol version:          6.7.0
                                    appl. framework build no.: 97
2023-10-04T19:12:03.807Z CNTRLR   querying serial API setup capabilities...
2023-10-04T19:12:03.821Z CNTRLR   supported serial API setup commands:
                                  · GetSupportedCommands
                                  · SetTxStatusReport
                                  · SetPowerlevel
                                  · GetPowerlevel
                                  · GetMaximumPayloadSize
2023-10-04T19:12:03.822Z CNTRLR   querying controller IDs...
2023-10-04T19:12:03.832Z CNTRLR   received controller IDs:
                                    home ID:     0xfb046add
                                    own node ID: 1
2023-10-04T19:12:04.087Z CNTRLR   supported Z-Wave features: 
                                    · SmartStart
2023-10-04T19:12:04.088Z CNTRLR   querying controller capabilities...
2023-10-04T19:12:04.118Z CNTRLR   received controller capabilities:
                                    controller role:      primary
                                    is the SUC:           true
                                    started this network: true
                                    SIS is present:       true
                                    was real primary:     true
2023-10-04T19:12:04.119Z CNTRLR   Enabling TX status report...
2023-10-04T19:12:04.168Z CNTRLR   Enabling TX status report successful...
2023-10-04T19:12:04.190Z CNTRLR   finding SUC...
2023-10-04T19:12:04.234Z CNTRLR   This is the SUC
2023-10-04T19:12:04.235Z CNTRLR   querying additional controller information...
2023-10-04T19:12:04.375Z CNTRLR   received additional controller information:
                                    Z-Wave API version:         8 (legacy)
                                    Z-Wave chip type:           ZW050x
                                    node type                   Controller
                                    controller role:            primary
                                    controller is the SIS:      true
                                    controller supports timers: false
                                    nodes in the network:       1, 4, 8, 9, 11, 12, 14, 15, 17, 18, 19, 20, 21, 
                                  22, 23, 24, 34, 36, 38, 39, 41, 42, 43, 44, 47, 48, 49, 51, 53, 55, 56, 60, 62
                                  , 64, 65, 66, 67, 68, 69, 70, 71, 73, 78, 79, 80, 81, 82, 94, 99, 110, 111, 11
                                  2, 113, 114, 117, 118, 125
2023-10-04T19:12:04.792Z CNTRLR   [Node 001] Embedded device config loaded
2023-10-04T19:12:04.799Z CNTRLR   [Node 004] Embedded device config loaded
2023-10-04T19:12:04.810Z CNTRLR   [Node 008] Embedded device config loaded
2023-10-04T19:12:04.819Z CNTRLR   [Node 009] Embedded device config loaded
2023-10-04T19:12:04.844Z CNTRLR   [Node 011] Embedded device config loaded
2023-10-04T19:12:04.884Z CNTRLR   [Node 012] Embedded device config loaded
3 Likes

Hm… this is strange… I have no dead nodes and the same problem.
Anyhow, thanks for confirmation that 2.0.2 does not solve the problem, this saved quite some time to retry…
Shame it is not fixed, as this is effectively prohibiting us from updating HA to 2023.10, as version 2.0+ of Zwave JS UI is required for zwave network :frowning:

Fix might be coming, stay updated here too: Z-Wave JS failure · Issue #3234 · home-assistant/addons · GitHub

(Looks like there is some issue with Aeotec Gen5 sticks firmware triggering new features in zwave js, maybe the dead node some people have mentioned is just co-incidence)

This sounds good.
All 3 points match:
-having the z-wave issue
-having the Aeotec Gen5 stick
-having failed nodes that cannot be removed

Hopefully the fix is there soon.

1 Like

Your theory holds.

I do NOT have dead nodes, have an Aeotec 5+ on latest firmware so supports smartstart, on a Pi4, have upgraded to JsUI 2.02 and 2023.10, soft reset is Off

and Do not have the issue.

My difference is I do not have any weird dead nodes I cannot remove - I only have the standard once in a while drop off and back with a ping dead nodes :joy:

That said it’s (the JsUI console) actually notably more responsive… So I’ll give them that.

1 Like

Also not having fdead nodes and seeng some successes I also made another retry to update and it went smooth this time. No issues whatsoever!