Having some Z-Wave issues. All advice welcome please. 🙏🏻

[jamespreedy]

Hi folks. Really sorry for the essay, but wanted to include as much as I could in case it’s useful.

I want to start by saying that I really want to love Z-Wave. When it works for me, it’s awesome, but I’m having some issues which I just can’t shake.

I used a previous post as reference for the kind of information that might be useful for some advice - if I’ve missed anything useful, please let me know and I’ll update this post.

I’m running an Intel NUC i3 with 16gb RAM and a 200Gb SSD. I’m running Home Assistant OS, and Z-WaveJS2MQTT. HA, my supervisor, OS, addons, HACS - everything is up to date.

Before evening knowing it was a thing, I’d extended the USB. I’m not sure I got this 100% however. The NUC is in a server rack in a server cupboard. Clearly not a great place for my Z-Wave controller and as you might expect I was having all sorts of related issues. I bought the below from Amazon, and extended it into the loft space in our roof. This was a great move, and everything was instantly better. That said, it’s 10m long and USB 3.0. Still, much better than it was before.

UGREEN USB Extension Cable, USB 3.0 Male to Female Active Repeater Cable with...

I’m running a Gen 5 Z-Stick ZW090 with FW: v1.2, SDK: v6.7.0 and I initially built the mesh inside out, starting from the controller in the middle, and using no security at all (no locks or similar). I have 81x devices in total, mostly Aeotec where possible - they seem great.

In no particular order.

• 4x Range Extenders (700 series) - ZW189-C15
• 1x Doorbell 6 - ZW162
• 5x Nano Shutters v3 - ZW141
• 10x Range Extenders (500 series) - ZW117
• 6x Nano Switches - ZW116
• 2x Nano Dimmer - ZW111
• 22x Multisensor 6’s - ZW100
• 11x Smart Switch 6’s - ZW096
• 4x Heavy Duty Switches - ZW078
• 5x DIN Rail mounted Qubino 3 Phase Smart Meters - ZMNHXD
• 6x Steinel XLED Home 2 Flood Lights
• 1x Popp & Co Mains Water Stop - POPE701479
• 1x Philio Panic Button - PSR03-C

All devices except the panic button are mains powered. All devices have had firmware updates where possible (most Aeotecs, plus the Qubinos).

The network is much more stable than it ever used to be. I dont generally see anything “dead” or unavailable. I’ve healed the network before as I sometimes find that when updating the Z-WaveJS2MQTT addon that everything shows in the addon dashboard as being unavailable. Restarting the addon, HA and even the host wouldn’t bring them back to life, but I discovered that by chance, disconnecting the Z-Stick for a little time, plugging it back in, and rebooting the addon would work like magic at bringing it all back to life.

Speed seem pretty good to my untrained eye too. Multisensor 6 motion => HA => Hue Hub => Lights. There’s no particularly noticeable lag an I’m happy with how quickly the lights come on as you move around the house. I use the multisensors for motion, temp, humidity and light levels. Aside from some clearly anomalous readings (which I now filter out by passing them through a range then an outlier filter), I’m pretty happy with the data I get from them. Obviously motion is immediate, and the temp, humidity and light level come through every 300 seconds.

To keep an eye on the devices, I built some sensors showing the time when the devices last updated a value I want to keep a track on - for example in a multisensor 6, the last time we saw an updated value for the motion, temp, humidity or light levels. This seems to be pretty solid with only the odd outlier from time to time - mostly keeping under 300 seconds as you would hope.

That “side” of the system is great. However I’m really struggling with the other side - the devices that I have in place to update my energy dashboard and do some regular power monitoring.

These are the …

• Smart Switches
• Heavy Duty Switches
• Nano Switches
• Qubino Smart Meters

In general, these are all configured in the same way. A change in power draw of more than X watts (say 5W for example), or a change in power draw of more than Y watts (say 20%), will trigger the device to update its readings. If there’s no change, then it will update power (W) and consumption (kWh) every 300 seconds too.

Again, I keep an eye on these with a last changed sensor - in most cases, this is just reporting when they last changed their W and kWh values.

Some of these again seem really well behaved …

… whilst others really aren’t.

They seem fine for a period of time, reporting on change or 300 seconds correctly, whichever comes first. Then, for whatever reason, they go completely off the rails, and wont report any changes at all. In addition, there’s nothing being reported in the Z-Wave add-on logs that seems out of the ordinary - the devices aren’t throwing errors, they just aren’t reporting any changes.

They don’t go dead, and will ping immediately. Refreshing values immediately updates the data values, and we see those changes reflected in my “last changed sensors”. They heal without issue, and also re-interview without issue.

I’m not claiming anything here, but anecdotally, it seems that if I go into their config, and change some settings (perhaps change 300 sec report time to 150, then wait for it to update, then change it back to 300), that it seems to remember what it’s supposed to be doing and gets back into the rhythm of things.

The Qubino smart meters are a bit less configurable, but are set to update on either power change (e.g. 5W), or time interval (e.g. 300 secs). They don’t look great, and my “change something in the Z-Wave dashboard config for that device” trick doesn’t seem to have any effect.

If nothing else, the power consumption in kWh for these devices should be creeping up over time - they are monitoring devices like our Air Source Heat Pump and our Mechanical Ventilation and Head Recovery + cooling. Again, they just seem to be stuck until they aren’t.

I’m really disappointed that I can’t get this to be as rock solid as Z-Wave should be - I know it can be amazingly good for some people. I feel like I’ve tried everything I can and have found similar sounding issues on here and other forums, but never want to assume it’s the same thing - no-one ever has the same house and the same setup.

I’ve spent good $$$ on this setup and on the house design to make this a reality for us and I fear that if I cant make it work for us that I’ll end up with a load of extra 2.4Ghz devices on my Wifi trying to do the same thing worse.

I don’t want to moan, but was wondering if anyone had seen anything like this, or had any advice I could please try? Appreciate anything you might have in advance …

[finity]

Until someone smarter than me comes around and gives you real advice…

If all else fails maybe the load on the system is being an issue?

I have only 30ish devices and don’t have any problems like that using the same aeotec gen 5 stick.

You might be able to try to get another stick and run two networks and see if that smooths things out.

Especially since you aren’t seeing anything in the logs to tell you something is wrong and it seems like you’ve got all the other known culprits taken care of.

[jamespreedy]

@finity - No, all good - I’ll take any advice at this point!
How would you approach the 2x sticks approach? Run Z-WaveJS2MQTT in Docker on a Pi somewhere else in the house, and phone home back to my HA instance over MQTT?

I’ll be interested to hear how bad people think my USB 3 extender cable is. I also have a lot of range extenders - this was partly as I had issues at the beginning when the Z-stick was in the server rack, but also a lot of my devices are in walls, under cabinets, in DIN rail cabs etc etc. Not sure if you can have too many of them.

I also saw this from @PeteRage, I love the idea of using my sensors to give the devices a poke. Will give this a try today, but clearly would prefer to fix the underlaying issue rather than issue a digital cattle prod …

How to recognize when devices are offline?

I do a combination of things. Most importantly I verify that the device is up and responding. Here are my templates / automations. Every 10 minutes and at HA startup it polls the switch state. When a response to the poll occurs, regardless of whether it’s state changes a zwave_js.value_updated event fires. This guarantees that the switch is alive and responding. That event triggers an update to the last_updated template sensor that captures the current time. Then there is a latency sens…

[Edwin_D]

@jamespreedy

I’ll be interested to hear how bad people think my USB 3 extender cable is.

Using an USB cable in between is smart, but an active USB 3 one not so much. I’ve seen USB3 interfere with lots of transmissions, depending on how well everything was shielded. I can see the length being tricky too, so maybe there are active usb2 ones too? Or use a smaller passive to get the stick away from interference and use routers to cover the rest?

[jamespreedy]

@Edwin_D - thanks mate, I’ve been thinking similar. I ordered this this morning, will try it next week when it arrives. I don’t think the swap would hurt and I’m always game to improve the setup, but I dont feel like it’s the root of my problems. After all, the multisensors are able to deliver their updates in a timely way - just can’t see why energy sensor classes can’t keep to the 300 sec update period.

Amazon Basics USB 2.0 Active Extension Cable Type A Male to A Female - 9.75...

[Edwin_D]

You are sure that they should have changed value in that period do you? Because the update may come in and be ignored because the value is the same as before. But you have a point there.

You could also look at the security the devices are included with. Granted, my bad experiences were with Homey, not HA, but S0 security could be extremely flakey since it requires a lot more data to get through safely, whereas unsecure and S2 have way less data so way less that can fail. Also I’m not sure if z-wave routers can pass along data for devices in different security classes.

Since most of my devices do not support S2 and I do not own locks and stuff, I opted for all unsecure.

[jamespreedy]

@Edwin_D. Yeah I checked that one by increasing the power consumed on those devices / plugs by enough to warrant a change. I was also pretty selective with the entities I was tracking with my “last_updated” sensors, to make sure there were one’s I knew would move on. kWH consumption is a good one, as even devices in standby should increase this over time.

I haven’t included anything with security at all - am I missing a trick there? I figured that as I don’t have any locks or similar, (and burglars probably came to steal a laptop, not bring one to hack Z-Wave) that I didn’t need it.

Thanks again.

[Edwin_D]

I don’t think you’re missing any tricks, on the contrary. I do not know what integration you use, zwave2mqtt lets you choose, but Homey where I came from picked the security automatically, which was S0 in my case. Then I had similar problems with Homey, until I found a way to force unsecure inclusion. So I stuck with what worked, and it works for me with Home Assistant even better.

Did you check the routes for the tricky devices to see if they use a router that may be less than stellar? I don’t own 700 series stuff yet but I got the impression form the forum that these proved less reliable up until now.

[RickKramer (Rick Kramer - Jansen)]

Personally I’d go with a USB2 non powered hub/extension cord, with a max of 6m (preferably shorter). No experience with powered extension cords, maybe it’s some holy grail When the devices stop reporting, do other sensors still work and report changes? Any logging that might be of interest? If you do go with a Pi, I’d use ZWaveJS2MQTT but just the Websocket part (not MQTT) to connect with HA, unless you’re really needing the MQTT stuff

[jamespreedy]

@RickKramer - Thanks. Didn’t know you could go 6m without power on USB2. It might be a bit short, but I’ll have a poke around in the loft this week and see what I can do. Assume the power draw of the Z-Stick isn’t that much then if other folks are managing that far.

Yes, when the devices stop reporting, there other sensors continue to work and report changes - the Multisensor 6’s in particular never miss a beat. There’s nothing in the logging at all - no errors, no timeouts, nothing at all - it’s almost like these models of devices decide they are going to have a little nap and stop phoning home. Thanks for the websocket advice - never uncoupled it before so will do some homework before giving it a shot. Have MQTT already, dont need Z-Wave to use it.

Have to say, this idea of getting HA to give the nodes a poke seems to be working well (based on an hour or so’s usage this morning). I don’t like it, but it seems to be a reasonably inoffensive way of giving things a gentle nudge. I guess the downside is that if something stops phoning home on its on, this will run every 5 mins indefinitely - giving it a nudge doesn’t restart it or fix the issue …

- id: '2AB1A6AC-75D6-4429-A589-533E3446DD9F'
  alias: "Z-Wave Stale Node / Heavy Switch / Car Charger 1 / Refresh"
  mode: single

  trigger:
  - platform: numeric_state
    entity_id:
      - sensor.zwave_car_charger_1_last_changed
    above: 300

  action:
    - service: zwave_js.refresh_value
      data:
        entity_id: sensor.car_charger_1_electric_consumption_kwh
        refresh_all_values: true

[PeteRage]

@jamespreedy

Thanks. Didn’t know you could go 6m without power on USB2

Keep your eye on the zwavejs logs in debug mode. Bad stick placement results in garbled messages being seen which are now logged. It seems the message integrity checking is better in zwavejs than OZW and I’m seeing a lot fewer crazy temperature reading (1025.5 F). That said, longer may not be better. I tried a max length USB 2.0 cable and it made things worse. Tried a 4 footer and that seems to work well. Of course, could be a cable issue.

Some occasional message garbling may be expected as if two zwave devices send at the same time. But the protocol should retry. (At least this is my understanding, happy to be corrected)

[Domoticon]

@jamespreedy

Then, for whatever reason, they go completely off the rails, and wont report any changes at all. In addition, there’s nothing being reported in the Z-Wave add-on logs that seems out of the ordinary - the devices aren’t throwing errors, they just aren’t reporting any changes.

Looks something similar what I experienced a while ago, although using a complete other setup. FYI, check my topic about it, maybe there is some info you can use…

[Z-Wave JS to MQTT, Synology DSM 7] Aeotec Z--Stick 7 disconnects from HA Virtual Machine

Used Config: Home Assitant 2022.6.0, running on a Virtual Machine. Hosted by: Synology RS820+ (DSM 7.1-42661 Update 2) Zwave Controller: Aeotec Z–Stick 7 (FW & SDK: v7.17.2, marked as Silicon Labs, 700 Series-based Controller, ZST10-700) Home Assistant Z-Wave Add-on: Z-Wave JS to MQTT (0.42.0) Home Assistant Z-Wave Integration: Z-Wave JS Driver version: 9.3.0 Server version: 1.17.0 Server-URL: ws://a0d7b954-zwavejs2mqtt:3000 Issue: The Z-Wave Controller connection is broken after a …

IMHO, first of all you have exclude as much as you can:
1 - Use a separate (dedicated) USB2-port with a proper USB2 shielded extension cable (no HUB, no cables with electronics inside, no lengths > 50cm).
2 - When having troubles, check if the Z-Wave USB-key is still connected/reachable. I noticed some HA CLI messages when there were troubles.
3 - Maybe the Portainer add-on can be of help accessing the HA system, checking the USB config Linux commands lsusb -t or dmesg -w (realtime system check, useful for disconnect/attach USB devices messages). Beware! Before accessing make a complete HA backup, just to be sure!

I hope you can narrow down the root problem…

[finity]

@jamespreedy

How would you approach the 2x sticks approach?

If you still want/need to try it or for anyone else…

you can run two zwavejs2mqtt instances on the same machine in docker with each using the different sticks. Or if it helps for range of course you can also run them on different machines too.

you then create another zwavejs integration in HA pointing to the other zwavejs instance.

There is no reason to use MQTT with this configuration at all. Just set them both up using websockets and as long as they are both accessible to HA over your network (and they would be by default if they are both running on the same host as HA) then you should be good to go.

I did that for a while when first testing zwavejs using my Aeotec and HUSZBZ-1 sticks.

[Domoticon]

@finity

There is no reason to use MQTT with this configuration at all

You’re absolutely right, it is strongly advised not to use it. Check the Z-Wave JS to MQTT add-on documentation page:

Known issues and limitations

• Z-Wave JS to MQTT supports Home Assistant Discovery over MQTT. It is STRONGLY recommended NOT to use that option. Use the Z-Wave JS integration as documented above instead.

[Domoticon]

@jamespreedy

There’s nothing in the logging at all - no errors, no timeouts, nothing at all - it’s almost like these models of devices decide they are going to have a little nap and stop phoning home

As I mentioned before, it looks similar to my earlier experiences. My explanation, not being an expert, to this behaviour looks to me that the USB ZWave-key is still powered (because it is the master controller, therefore communicates with it’s zwave slave devices) but loses communication to the OS (HA). Does this make any sense?

[PeteRage]

Don’t know if this has been done, try a full hardware power cycle including unplugging power, unplugging any hubs, sticks, …

[mterry63 (Martin)]

You can run the z-wave USB controller on a RPi with ser2net over a network connection. Z-waveJS2MQTT supports that out of the box by accepting a TCP connection string instead of a local device path. HomeSeer (Commercial HA software) sells a pre-configured device called a Z-Net which is just a RPi with a Z-wave HAT plugged into it, but a USB controller would be effectively the same. 81 devices with a lot of power reporting is likely reaching a limit for traffic on one network ID. It was reported on that forum that early 7 series controller chips were locking up under similar conditions (high power reporting) but last I saw that was sorted by Silicon Labs in newer firmware.

[jamespreedy]

Hey folks. Thanks all for your advice and help - really appreciate it, and good to hear some of your experiences too. I’ll be trying some for sure - I’ve ordered a non-active USB 2.0 for a start, just to rule it out. I’m going to struggle with the length - I need a couple of meters realistically to get out the top of the server rack, into the loft and away from big metal things, but I’ll trial and error it.

This leads me onto a bit of a n00b question about logs within the Z-WaveJS2MQTT addon. There are settings in both the “General” section of the addon settings pane, and also in the Z-Wave section.

General has “Log enabled”, “Log level”, and log to file.
Z-Wave has the same three options, plus log nodes. Which should I be using to try and squeeze as much debug info as possible out of this scenario? Log nodes seems like it would be helpful.

Aside from these options within the addon, are there any HA logs elsewhere I should be looking for? All I’ve really been doing for now is using the “debug” pane in the addon - which to my untrained eye seems very similar to looking at the addon logs in HA => Settings => System => Logs => Z-WaveJS2MQTT.

Point taken about MQTT - I’d not really done any homework there and was just spitballing when I suggested me doing that.

Again, as always, thanks folks.

[CeesGe (Cees)]

@jamespreedy

They seem fine for a period of time, reporting on change or 300 seconds correctly, whichever comes first. Then, for whatever reason, they go completely off the rails, and wont report any changes at all. In addition, there’s nothing being reported in the Z-Wave add-on logs that seems out of the ordinary - the devices aren’t throwing errors, they just aren’t reporting any changes.

They don’t go dead, and will ping immediately. Refreshing values immediately updates the data values, and we see those changes reflected in my “last changed sensors”. They heal without issue, and also re-interview without issue.

This behaviour is exactly the same compared to what I discover in combination of the smart meter with Domoticz. For that reason I suspect the problem is in the qubino smart meter and not in Home Assistant with zwave js UI nor in Domoticz. I have 3 of those smart meters in my configuration. Strangely enough the frequency of the problem differs per SmartMeter.

Anyway the reason to react is that I am considering moving a large part of my home automation to Home Assistant and in my research stumbled against this problem on the forum.

Good thing is that an automatic remediation is available to give the nodes a poke when the did not respond for a predefined time. That will improve my automation. I am happy to have seen this, at least I assume that this is still the bypass.

[4romany (Romany)]

I used Zwave devices for a couple of years - probably around 2016 or so starting out - and I did not have a positive experience. When they worked they seemed pretty stable - but I found devices expensive (like starting at $45). I had one zwave relay device - similar to a shelly1 in function - that always cost at least $45. They seem to last for about 6 months - and become intermittant or just fail completely. I purchased a GE switch (master and slave for a “traveler wire” setup - within just a few weeks it would forget its configuration - I would default it - reattach it to the hub - and it might stay up for a week - if that long. The dual GE switch combination was like $70. I started using WIFI connected devices instead (tasmota converted) - and almost 6 years later I have yet to have a single tasmota device fail - and I have probably 20 in my network). I’m a IT network guy - I know how to troubleshoot IP devices - I never felt that way with zwave problems. Others folks seem to have a positive experience with zwave – and perhaps my experience is the exception.