Invalid payload errors and eventually non-functional Z-wave network

Configuration Z-Wave
1

My HAOS z-wave network is becoming non-functional after a few days. Here are some of the stats.

HA core: 2024.11.3
HAOS: 13.2
ZST39 LR Firmware 1.50 USB stick on 1’ USB extender cable
64 devices
Also host a Zigbee stick (also on 1’ USB extender cable)
Host system: Beelink Mini S12 Pro Mini PC, 12th Intel Alder Lake-N100, 16 GB Memory, 500 GB SSD.

I see random disconnects with 800 Z-wave stick in the log over time. Then log indicates jammed sessions and finally non-functional. Once this happens only a cold start will resolve the problem as reboots are ineffective. Contacted Zooz but they sent me to Github. Not sure how to post properly there. Pretty sure I have the latest firmware on the stick. I am told “no one” is having this problem but somehow I don’t believe that. The “invalid payload” messages don’t seem to reference a node, so I assume the driver is reporting that.

While all the RSSI I’ve seen in the debug log indicate everything should be well within range, I relocated the host in the middle of my house with no device more than 60’ away now and undergoing a z-wave network rebuild at this time to see if that has any effect.

(segment of startup log)
Closing server...
Server closed
[10:20:35] INFO: e[32mService restart after closinge[0m
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
cont-init: info: running /etc/cont-init.d/config.sh
[05:28:16] INFO: e[32mNo 'network_key' detected, setting it to 's0_legacy_key' for backwards compatibilitye[0m

[05:28:16] INFO: e[32mFlushing config to disk due to creation of new key(s)...e[0m
[05:28:17] INFO: e[32mSoft-reset set to automatice[0m
[05:28:17] INFO: e[32mVirtual Machine not detected, enabling soft-resete[0m
cont-init: info: /etc/cont-init.d/config.sh exited 0
cont-init: info: running /etc/cont-init.d/structure.sh
cont-init: info: /etc/cont-init.d/structure.sh exited 0
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun zwave_js (no readiness notification)
s6-rc: info: service legacy-services successfully started
[05:28:17] INFO: e[32mSuccessfully send discovery information to Home Assistant.e[0m
2024-12-09T10:28:18.151Z DRIVER   ███████╗        ██╗    ██╗  █████╗  ██╗   ██╗ ███████╗          ██╗ ███████╗
                                  ╚══███╔╝        ██║    ██║ ██╔══██╗ ██║   ██║ ██╔════╝          ██║ ██╔════╝
                                    ███╔╝  █████╗ ██║ █╗ ██║ ███████║ ██║   ██║ █████╗            ██║ ███████╗
                                   ███╔╝   ╚════╝ ██║███╗██║ ██╔══██║ ╚██╗ ██╔╝ ██╔══╝       ██   ██║ ╚════██║
                                  ███████╗        ╚███╔███╔╝ ██║  ██║  ╚████╔╝  ███████╗     ╚█████╔╝ ███████║
                                  ╚══════╝         ╚══╝╚══╝  ╚═╝  ╚═╝   ╚═══╝   ╚══════╝      ╚════╝  ╚══════╝
2024-12-09T10:28:18.152Z DRIVER   version 13.10.3
2024-12-09T10:28:18.152Z DRIVER   
2024-12-09T10:28:19.174Z CONFIG   version 13.10.3
2024-12-09T10:28:19.363Z CNTRLR   querying Serial API capabilities...
2024-12-09T10:28:20.436Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try i
                                  n 100 ms.
2024-12-09T10:28:21.166Z CNTRLR   Failed to execute controller command after 2/3 attempts. Scheduling next try i
                                  n 1100 ms.
2024-12-09T10:28:21.166Z DRIVER   Dropping message because it contains invalid data
2024-12-09T10:28:21.175Z DRIVER   Dropping message with invalid payload:
                                  0x012c00a800000100031e9f0328004e63a1b360ab8c0c3887532ce3ba320001e68a3f4f68728a
                                  d90200a8007f7ff2
2024-12-09T10:28:21.183Z DRIVER   Dropping message with invalid payload:
                                  0x012c00a800000100051e9f03ac0047f5a1e6e07765055669fe0055de628124d418e3debde5a4
                                  122e00b2007f7f5d
2024-12-09T10:28:21.190Z DRIVER   Dropping message with invalid payload:
                                  0x012800a800000100591a9f037000ab13c69d23d398917726413aaf849515362fcb651d7900ad
                                  007f7f34
2024-12-09T10:28:21.197Z DRIVER   Dropping message with invalid payload:
                                  0x012800a800000100361a9f0306009bad54a32c9b882d6c30fcabd6115142c606717c559900af
                                  007f7f6a
2024-12-09T10:28:21.204Z DRIVER   Dropping message with invalid payload:
                                  0x012800a800000100041a9f031c00252c3e8fb577b33f762138d2105586680a3ad9c36ab200ac
                                  007f7f5e
2024-12-09T10:28:21.212Z DRIVER   Dropping message with invalid payload:
                                  0x012800a800000100521a9f03c100ffcf6b12d573380ba4687eabd01b669ca0e2febcacb300aa
                                  007f7f2a
2024-12-09T10:28:21.219Z DRIVER   Dropping message with invalid payload:
                                  0x012800a800000100521a9f03c400e2abf7c2c3d5c5d67186b9d3cdce24480519df36eae800aa
                                  007f7fb8
2024-12-09T10:28:21.553Z DRIVER   Dropping message with invalid payload:
                                  0x012800a800000100051a9f03b0002243a9c0911869fd22899b3e7978672660d6a3916cf100b2
                                  007f7fbd
2024-12-09T10:28:22.073Z DRIVER   Dropping message with invalid payload:
                                  0x012400a8000001004d169f03c10090f030cb64c3b32d9d1c6ded51c775c2ea7400a9007f7fc1
2024-12-09T10:28:22.280Z CNTRLR   received API capabilities:
                                    firmware version:    1.50
                                    manufacturer ID:     0x027a
                                    product type:        0x04
                                    product ID:          0x0610
                                    supported functions: 
                                    · GetSerialApiInitData (0x02)
                                    · SetApplicationNodeInformation (0x03)
                                    · GetControllerCapabilities (0x05)
                                    · SetSerialApiTimeouts (0x06)
                                    · GetSerialApiCapabilities (0x07)
                                    · SoftReset (0x08)
                                    · GetProtocolVersion (0x09)
                                    · SerialAPIStarted (0x0a)
                                    · SerialAPISetup (0x0b)
                                    · SetRFReceiveMode (0x10)
                                    · SendNodeInformation (0x12)
                                    · SendData (0x13)
                                    · SendDataMulticast (0x14)
                                    · GetControllerVersion (0x15)
                                    · SendDataAbort (0x16)
                                    · FUNC_ID_ZW_GET_RANDOM (0x1c)
                                    · GetControllerId (0x20)
                                    · UNKNOWN_FUNC_MEMORY_GET_BYTE (0x21)
                                    · UNKNOWN_FUNC_MEMORY_PUT_BYTE (0x22)
                                    · UNKNOWN_FUNC_MEMORY_GET_BUFFER (0x23)
                                    · UNKNOWN_FUNC_MEMORY_PUT_BUFFER (0x24)
                                    · EnterBootloader (0x27)
                                    · UNKNOWN_FUNC_UNKNOWN_0x28 (0x28)
                                    · GetNVMId (0x29)
                                    · ExtNVMReadLongBuffer (0x2a)
                                    · ExtNVMReadLongByte (0x2c)
                                    · NVMOperations (0x2e)
                                    · ClearTxTimers (0x37)
                                    · GetTxTimers (0x38)
                                    · ClearNetworkStats (0x39)
                                    · GetNetworkStats (0x3a)
                                    · GetBackgroundRSSI (0x3b)
                                    · SetListenBeforeTalkThreshold (0x3c)
                                    · ExtendedNVMOperations (0x3d)
                                    · RemoveSpecificNodeIdFromNetwork (0x3f)
                                    · GetNodeProtocolInfo (0x41)
                                    · HardReset (0x42)
                                    · FUNC_ID_ZW_REPLICATION_COMMAND_COMPLETE (0x44)
                                    · FUNC_ID_ZW_REPLICATION_SEND_DATA (0x45)
                                    · AssignReturnRoute (0x46)
                                    · DeleteReturnRoute (0x47)
                                    · RequestNodeNeighborUpdate (0x48)
                                    · ApplicationUpdateRequest (0x49)
                                    · AddNodeToNetwork (0x4a)
                                    · RemoveNodeFromNetwork (0x4b)
                                    · AddPrimaryController (0x4d)
                                    · AssignPriorityReturnRoute (0x4f)
                                    · SetLearnMode (0x50)
                                    · AssignSUCReturnRoute (0x51)
                                    · RequestNetworkUpdate (0x53)
                                    · SetSUCNodeId (0x54)
                                    · DeleteSUCReturnRoute (0x55)
                                    · GetSUCNodeId (0x56)
                                    · SendSUCNodeId (0x57)
                                    · AssignPrioritySUCReturnRoute (0x58)
                                    · ExploreRequestInclusion (0x5e)
                                    · ExploreRequestExclusion (0x5f)
                                    · RequestNodeInfo (0x60)
                                    · RemoveFailedNode (0x61)
                                    · IsFailedNode (0x62)
                                    · ReplaceFailedNode (0x63)
                                    · undefined (0x65)
                                    · RequestProtocolCCEncryption (0x68)
                                    · GetRoutingInfo (0x80)
                                    · LockUnlockLastRoute (0x90)
                                    · GetPriorityRoute (0x92)
                                    · SetPriorityRoute (0x93)
                                    · UNKNOWN_FUNC_UNKNOWN_0x98 (0x98)
                                    · VirtualNodeSetNodeInfo (0xa0)
                                    · VirtualNodeSendNodeInfo (0xa2)
                                    · VirtualNodeSetLearnMode (0xa4)
                                    · GetVirtualNodes (0xa5)
                                    · IsVirtualNode (0xa6)
                                    · BridgeApplicationCommand (0xa8)
                                    · SendDataBridge (0xa9)
                                    · SendDataMulticastBridge (0xab)
                                    · GetLibrary (0xbd)
                                    · SendTestFrame (0xbe)
                                    · GetProtocolStatus (0xbf)
                                    · StartWatchdog (0xd2)
                                    · StopWatchdog (0xd3)
                                    · SetMaximumRoutingAttempts (0xd4)
                                    · SetMaxSmartStartInclusionRequestInterval (0xd6)
                                    · undefined (0xd7)
                                    · undefined (0xd8)
                                    · Shutdown (0xd9)
                                    · GetLongRangeNodes (0xda)
                                    · GetLongRangeChannel (0xdb)
                                    · SetLongRangeChannel (0xdc)
                                    · SetLongRangeShadowNodeIDs (0xdd)
                                    · Proprietary_DE (0xde)
                                    · Proprietary_DF (0xdf)
                                    · Proprietary_E7 (0xe7)
                                    · Proprietary_E8 (0xe8)
2024-12-09T10:28:22.281Z CNTRLR   querying additional controller information...
2024-12-09T10:28:22.291Z CNTRLR   received additional controller information:
                                    Z-Wave API version:         1 (official)
                                    Z-Wave chip type:           EFR32ZG23 / ZGM230S
                                    node type                   Controller
                                    controller role:            primary
                                    controller is the SIS:      true
                                    controller supports timers: false
                                    Z-Wave Classic nodes:       1, 3, 4, 5, 9, 12, 13, 14, 15, 16, 17, 19, 20, 2
                                  1, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 41, 43, 44, 45,
                                   46, 47, 48, 49, 52, 53, 54, 56, 57, 58, 60, 61, 64, 65, 66, 67, 68, 72, 74, 7
                                  5, 76, 77, 78, 79, 80, 82, 85, 86, 89, 94, 95
2024-12-09T10:28:22.291Z CNTRLR   querying version info...
2024-12-09T10:28:22.298Z CNTRLR   received version info:
                                    controller type: Bridge Controller
                                    library version: Z-Wave 7.22
2024-12-09T10:28:22.298Z CNTRLR   querying protocol version info...
2024-12-09T10:28:22.305Z CNTRLR   received protocol version info:
                                    protocol type:             Z-Wave
                                    protocol version:          7.22.1
                                    appl. framework build no.: 43981
                                    git commit hash:           63636639623665613165616566313162
2024-12-09T10:28:22.305Z CNTRLR   querying controller capabilities...
2024-12-09T10:28:22.310Z CNTRLR   received controller capabilities:
                                    controller role:      Primary
                                    is the SUC:           true
                                    started this network: true
                                    SIS is present:       true
                                    was real primary:     true
2024-12-09T10:28:22.310Z CNTRLR   querying serial API setup capabilities...
2024-12-09T10:28:22.317Z CNTRLR   supported serial API setup commands:
                                  · GetSupportedCommands
                                  · SetTxStatusReport
                                  · SetLongRangeMaximumTxPower
                                  · SetPowerlevel
                                  · GetLongRangeMaximumTxPower
                                  · GetPowerlevel
                                  · GetMaximumPayloadSize
                                  · GetLongRangeMaximumPayloadSize
                                  · SetPowerlevel16Bit
                                  · GetPowerlevel16Bit
                                  · GetRFRegion
                                  · SetRFRegion
                                  · SetNodeIDType
2024-12-09T10:28:22.317Z CNTRLR   querying max. payload size...
2024-12-09T10:28:22.322Z CNTRLR   maximum payload size: 46 bytes
2024-12-09T10:28:22.322Z CNTRLR   supported Z-Wave features: 
                                    · SmartStart
2024-12-09T10:28:22.322Z CNTRLR   Querying configured RF region...
2024-12-09T10:28:22.327Z CNTRLR   The controller is using RF region USA (Long Range)
2024-12-09T10:28:22.327Z CNTRLR   Querying configured max. Long Range powerlevel...
2024-12-09T10:28:22.332Z CNTRLR   The max. LR powerlevel is 14.0 dBm
2024-12-09T10:28:22.332Z CNTRLR   Querying configured Long Range channel information...
2024-12-09T10:28:22.337Z CNTRLR   received Z-Wave Long Range channel information:
                                    channel:                         A
                                    supports auto channel selection: true
2024-12-09T10:28:22.337Z CNTRLR   Performing soft reset...
2024-12-09T10:28:22.344Z CNTRLR   Waiting for the controller to reconnect...
2024-12-09T10:28:22.515Z CNTRLR   reconnected and restarted
2024-12-09T10:28:22.516Z CNTRLR   Starting hardware watchdog...
2024-12-09T10:28:22.517Z CNTRLR   Switching serial API to 16-bit node IDs...
2024-12-09T10:28:22.518Z CNTRLR   querying Z-Wave Long Range capabilities...
2024-12-09T10:28:22.534Z CNTRLR   Switching to 16-bit node IDs successful
2024-12-09T10:28:22.558Z CNTRLR   received Z-Wave Long Range capabilities:
                                    max. payload size: 148 bytes
                                    nodes:             
2024-12-09T10:28:22.559Z CNTRLR   Switching serial API to 16-bit node IDs...
2024-12-09T10:28:22.564Z CNTRLR   Switching to 16-bit node IDs successful
2024-12-09T10:28:22.564Z CNTRLR   querying controller IDs...
2024-12-09T10:28:22.569Z CNTRLR   received controller IDs:
                                    home ID:     0xe4c0be0e
                                    own node ID: 1
2024-12-09T10:28:22.570Z CNTRLR   Enabling TX status report...
2024-12-09T10:28:22.575Z CNTRLR   Enabling TX status report successful...
2024-12-09T10:28:22.575Z CNTRLR   finding SUC...
2024-12-09T10:28:22.579Z CNTRLR   This is the SUC
2024-12-09T10:28:22.686Z CNTRLR   [Node 001] Embedded device config loaded


(segment of Z-wave log)


2024-12-09T11:50:22.981Z DRIVER » [Node 060] [REQ] [SendDataBridge]
                                  │ source node id:   1
                                  │ transmit options: 0x05
                                  │ callback id:      100
                                  └─[Security2CCNonceReport]
                                      sequence number:  151
                                      SOS:              true
                                      MOS:              false
                                      receiver entropy: 0x41d207b9ac731e1572c712a782e04214
2024-12-09T11:50:22.988Z SERIAL « [ACK]                                                                   (0x06)
2024-12-09T11:50:22.991Z SERIAL « 0x010401a90152                                                       (6 bytes)
2024-12-09T11:50:22.991Z SERIAL » [ACK]                                                                   (0x06)
2024-12-09T11:50:22.991Z DRIVER « [RES] [SendDataBridge]
                                    was sent: true
2024-12-09T11:50:23.008Z SERIAL « 0x012400a8000001003c169f03a6007a313dc96dc9f984e9d8c6f0ef1a3d1b29f50 (38 bytes)
                                  0b0007f7fbc
2024-12-09T11:50:23.009Z DRIVER   Dropping message with invalid payload
2024-12-09T11:50:23.010Z DRIVER « [Node 060] [REQ] [BridgeApplicationCommand]
                                  │ RSSI: -80 dBm
                                  └─[Security2CCMessageEncapsulation] [INVALID]
                                      error: Duplicate command (sequence number 166)
2024-12-09T11:50:23.010Z SERIAL » [ACK]                                                                   (0x06)
2024-12-09T11:50:23.054Z SERIAL « 0x011d00a96400000600b37f7f7f7f01010300000000020200007f7f7f7f7fe6    (31 bytes)
2024-12-09T11:50:23.054Z SERIAL » [ACK]                                                                   (0x06)
2024-12-09T11:50:23.055Z DRIVER « [REQ] [SendDataBridge]
                                    callback id:            100
                                    transmit status:        OK, took 60 ms
                                    routing attempts:       2
                                    protocol & route speed: Z-Wave, 40 kbit/s
                                    routing scheme:         LWR
                                    ACK RSSI:               -77 dBm
                                    ACK channel no.:        1
                                    TX channel no.:         1
2

Same day, same boot. At the moment non-functional. See this in the log.

2024-12-09T12:18:58.804Z CNTRLR « [Node 045] received CentralScene notification {
                                      "nodeId": 45,
                                      "ccId": "Central Scene",
                                      "ccCommand": "0x03",
                                      "payload": "0x108001"
                                  }
2024-12-09T12:27:14.628Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try i
                                  n 100 ms.
2024-12-09T12:39:37.610Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:54.001Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:54.105Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:54.466Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:54.656Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:54.688Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:54.720Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:55.222Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:56.625Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:56.637Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:57.905Z DRIVER   Dropping message with invalid payload
2024-12-09T12:46:58.056Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:02.649Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:03.056Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:12.976Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:13.241Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:13.505Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:17.172Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:17.306Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:17.440Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:17.515Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:17.615Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:17.641Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:17.756Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:17.915Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:17.943Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.092Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.109Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.269Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.304Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.349Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.386Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.419Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.455Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.467Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.491Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.563Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.581Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.594Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.611Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:18.675Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:23.670Z DRIVER   Dropping message with invalid payload
2024-12-09T12:47:25.006Z CNTRLR   The controller is jammed
2024-12-09T12:47:29.143Z CNTRLR   The controller is no longer jammed
2024-12-09T12:47:29.307Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try i
                                  n 100 ms.
2024-12-09T12:47:29.457Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try i
                                  n 100 ms.
2024-12-09T12:47:29.470Z CNTRLR « [Node 005] received S2 nonce without an active transaction, not sure what to d
                                  o with it
2024-12-09T12:47:29.608Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try i
                                  n 100 ms.
2024-12-09T12:47:29.823Z CNTRLR « [Node 005] received S2 nonce without an active transaction, not sure what to d
                                  o with it
2024-12-09T12:47:30.314Z CNTRLR « [Node 043] Received update for a Supervision session
2024-12-09T12:47:34.827Z DRIVER   Dropping message with invalid payload
2024-12-09T12:50:15.635Z DRIVER   Dropping message with invalid payload:
                                  0x010e00a800000100400000af007f7fb7
2024-12-09T12:51:41.225Z DRIVER   Dropping message with invalid payload
2024-12-09T12:51:41.943Z DRIVER   Dropping message with invalid payload
2024-12-09T12:51:42.638Z DRIVER   Dropping message with invalid payload
2024-12-09T12:51:45.172Z CNTRLR « [Node 004] received S2 nonce without an active transaction, not sure what to d
                                  o with it
2024-12-09T12:51:45.667Z DRIVER   Dropping message with invalid payload
2024-12-09T12:51:47.322Z DRIVER   Dropping message with invalid payload
2024-12-09T12:51:47.579Z DRIVER   Dropping message with invalid payload
2024-12-09T12:51:49.116Z DRIVER   Dropping message with invalid payload
2024-12-09T12:53:51.672Z DRIVER   Dropping message with invalid payload
2024-12-09T12:56:34.675Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try i
                                  n 100 ms.
2024-12-09T12:56:34.688Z CNTRLR « [Node 005] received S2 nonce without an active transaction, not sure what to d
                                  o with it
2024-12-09T12:56:34.811Z CNTRLR « [Node 005] received S2 nonce without an active transaction, not sure what to d
                                  o with it
2024-12-09T12:56:34.815Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try i
                                  n 100 ms.
2024-12-09T12:56:34.833Z CNTRLR « [Node 005] received S2 nonce without an active transaction, not sure what to d
                                  o with it
2024-12-09T12:56:45.180Z DRIVER   Dropping message with invalid payload
2024-12-09T12:56:49.291Z DRIVER   Dropping message with invalid payload
2024-12-09T12:56:49.936Z DRIVER   Dropping message with invalid payload
2024-12-09T12:57:02.422Z CNTRLR   [Node 004] Timed out while waiting for a response from the node (ZW0201)
2024-12-09T12:57:03.223Z CNTRLR   [Node 004] Transport Service RX session #14 failed
2024-12-09T12:59:28.912Z CNTRLR « [Node 049] Received update for a Supervision session
2024-12-09T13:00:10.165Z DRIVER   Dropping message with invalid payload
2024-12-09T13:02:25.542Z CNTRLR « [Node 049] Received update for a Supervision session
2024-12-09T13:06:38.656Z CNTRLR   Failed to execute controller command after 1/3 attempts. Scheduling next try i
3

And a visual log of the collapse of the 800 yesterday evening.

FWIW, The rebuild of the z wave network is still in progress

1208_800_status
1208_800_status2247×131 48.6 KB

4

Could be that one of your devices has failed and is spewing garbage into the network. Troubleshooting will be a challenge, basically shutdown all powered nodes, watch the logs and start turning things on from the controller out. Wait for the problem to materialize in the logs. Good luck.

5

That crossed my mind. I thought I knew what node the culprit was and removed it but no effect. I could live with that method of resolution if I knew for certain it was one of the nodes and not the Z-wave stick itself. I say that because the PREVIOUS firmware had this exact bug (documented).

6

Did you recently upgrade the controller firmware? I’m having similar errors after updating a ZAC93 to firmware 1.50.

My network functions normally, but I’m unable to do a z-wave inclusion on a Leviton DZ6HD dimmer (current firmware - 1.22).

HA core: 2024.12.2
Z-Wave JS UI 3.18.0
HAOS: 14.0
ZAC93 Firmware 1.50

I also rolled back without effect:
HA core: 2024.11.3
Z-Wave JS UI 3.17.0

I have a support request to Zooz for a controller firmware rollback.

I was able to include a HomeSeer HSM200 sensor into the network in the same room. I’ve tried with the controller in it’s usual location, as well as bringing it within 5 feet. I can directly include / exclude / ota update the device with a 700 series controller and HomeSeer Z-Flash.

2024-12-10 22:09:45.801 INFO Z-WAVE: Calling api startInclusion with args: [
  2,
  { forceSecurity: false, name: 'Pot Lights', location: 'Bedroom (S)' },
  [length]: 2
]
2024-12-11T03:09:45.806Z CNTRLR   Starting inclusion process with strategy Insecure...
2024-12-11T03:09:45.827Z CNTRLR   The controller is now ready to add nodes
2024-12-10 22:09:45.830 INFO Z-WAVE: Controller status: Non-secure inclusion started
2024-12-10 22:09:45.832 INFO Z-WAVE: Success zwave api call startInclusion true
2024-12-10 22:09:52.545 INFO APP: GET /health/zwave 301 1.596 ms - 162
2024-12-11T03:10:01.875Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:03.398Z DRIVER   Dropping message because it contains invalid data
2024-12-11T03:10:03.400Z DRIVER   Dropping message because it contains invalid data
2024-12-11T03:10:03.408Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:03.415Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:03.421Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:03.426Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:03.430Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:03.434Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:03.439Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:03.443Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:03.448Z DRIVER   Dropping message with invalid payload
2024-12-11T03:10:15.804Z CNTRLR   stopping inclusion process...
2024-12-11T03:10:15.826Z CNTRLR   The inclusion process was stopped
2024-12-10 22:10:15.829 INFO Z-WAVE: Controller status: Inclusion stopped
2024-12-10 22:10:22.642 INFO APP: GET /health/zwave 301 1.416 ms - 162
7

I have had a lot of issues with the ZAC93. On firmware v1.50 I cannot add any devices at all. On v1.20 it takes many many tries to add a device, if it even works.