Help: Z-Wave JS UI Addon breaking my system

othermore · August 11, 2024, 10:25pm

Hey,

I’ve been using Z-wave JS UI addon for a couple of years. In the last few months I did not instal the updates and after that, things got bad. I’ve tried to document it here, but the thread is long and messy and I thought maybe I can get help if I start a new one.

I have basically two issues:

The Z-Wave JS UI addon is in a bad state. I’m running a somewhat fresh version, 3.9.2. Any attempt to reboot or edit the config of the addon (internal addon config through its UI, not the external addon config in the system settings) makes the whole system to freeze, and I need to hard-reboot the virtual machine.
- As a side effect of the one above, any attempt to reboot my Home Assistant also fails, as it tries to stop this addon
- Similarly, I can’t install updates, of the addon because it tries to stop the addon first
- The logs do not show anything too significant before the freeze, just the expected action taken before it. I’ve checked both the JSUI Addon logs and the journalctl system logs
If I force the update, the addon will no longer start. I can update the addon by setting it not to start at boot, then hard-rebooting, and then installing the update without the addon being running. But after that, any attempt to start the addon results in the following error

s6-rc-compile: fatal: invalid /etc/s6-overlay/s6-rc.d/discovery/type: must be oneshot, longrun, or bundle
s6-rc: fatal: unable to take locks: No such file or directory
s6-linux-init-shutdownd: warning: /run/s6/basedir/scripts/rc.shutdown exited 111

Finally, I’ve tried several times to fully remove and re-install the addon. Without installing a backup, the addon works normally, but as soon as I restore the settings (settings.json), it starts failing again.

I also did a fresh install and manually set all the settings up through the UI, but I ended up in the same place.

On top of that, fresh installs are a bad idea, because even if I restore a backup, my devices are gone (the nodes show up in JS UI, but then they are no longer “provided” by the JS UI addon and Home Assistant can’t use them anymore). I even did a full fresh install and then re-set all the devices in my system, which was quite painful, but that did not work either.

Any help? Any idea of what can be failing? Maybe it’s just my ZWave Controller not getting along well with the latest ZWave JS ?

othermore · August 11, 2024, 10:43pm

BTW in case it helps, I’m running a AEOTEC Z‐Stick Gen5 USB Controller, and this is my settings.json

{
  "mqtt": {
    "name": "Mosquitto",
    "host": "core-mosquitto",
    "port": 1883,
    "qos": 1,
    "prefix": "zwave",
    "reconnectPeriod": 3000,
    "retain": true,
    "clean": true,
    "auth": false,
    "username": "",
    "password": "",
    "disabled": true,
    "_ca": "",
    "ca": "",
    "_cert": "",
    "cert": "",
    "_key": "",
    "key": ""
  },
  "gateway": {
    "type": 0,
    "plugins": [],
    "authEnabled": false,
    "payloadType": 0,
    "nodeNames": true,
    "hassDiscovery": false,
    "discoveryPrefix": "homeassistant",
    "logEnabled": true,
    "logLevel": "warn",
    "logToFile": true,
    "values": [],
    "jobs": [],
    "disableChangelog": false,
    "notifyNewVersions": false,
    "versions": {
      "app": "9.16.2",
      "driver": "13.0.2",
      "server": "1.37.0"
    }
  },
  "zwave": {
    "enabled": true,
    "port": "/dev/ttyACM0",
    "allowBootloaderOnly": false,
    "commandsTimeout": 30,
    "logLevel": "warn",
    "rf": {
      "txPower": {}
    },
    "securityKeys": {
      "S2_Unauthenticated": "[HIDDEN]",
      "S2_Authenticated": "[HIDDEN]",
      "S2_AccessControl": "[HIDDEN]",
      "S0_Legacy": "[HIDDEN]"
    },
    "securityKeysLongRange": {
      "S2_Authenticated": "[HIDDEN]",
      "S2_AccessControl": "[HIDDEN]"
    },
    "deviceConfigPriorityDir": "/data/store/config",
    "logEnabled": true,
    "logToFile": false,
    "maxFiles": 7,
    "serverEnabled": true,
    "serverServiceDiscoveryDisabled": true,
    "enableSoftReset": true,
    "enableStatistics": true,
    "serverPort": 3000,
    "maxNodeEventsQueueSize": 100,
    "higherReportsTimeout": false,
    "disableControllerRecovery": false,
    "networkKey": "",
    "disclaimerVersion": 1
  },
  "backup": {
    "storeBackup": true,
    "storeCron": "23 0 * * 1,3,5",
    "storeKeep": 60,
    "nvmBackup": true,
    "nvmBackupOnEvent": true,
    "nvmCron": "37 1 * * 5",
    "nvmKeep": 7
  },
  "zniffer": {
    "enabled": false,
    "port": "",
    "logEnabled": true,
    "logToFile": true,
    "maxFiles": 7,
    "securityKeys": {
      "S2_Unauthenticated": "",
      "S2_Authenticated": "",
      "S2_AccessControl": "",
      "S0_Legacy": ""
    },
    "securityKeysLongRange": {
      "S2_Authenticated": "",
      "S2_AccessControl": ""
    },
    "convertRSSI": false
  },
  "ui": {
    "darkMode": true,
    "navTabs": true,
    "compactMode": false
  }
}

freshcoast · August 11, 2024, 11:42pm

Sounds like whatever configuration changes you are making seem to trigger something. I’d try making these configuration changes:

Set driver log level to debug
Set driver log to file to true
Never use a raw device path like /dev/ttyACM0, always use /dev/serial/by-id/...
Disable soft-reset because you’re using a VM
Disable the automatic nvm backups
Disable MQTT if you are using the Z-Wave integration

I’d be most suspicious of the soft-reset and NVM settings. Note that I believe you can’t make NVM backups once soft-reset is disabled. 500-series + VMs + soft reset are not a good combo.

Make sure you don’t have the core add-on installed and running at the same time.

Post the driver debug logs if you continue to have problems.

If staring from scratch, don’t make too many changes at once and see if anything in particular breaks it.

Other suggestions:

Try a brand new VM, maybe something is corrupted
Try Z-Wave JS UI outside of a VM

mundschenk-at · August 12, 2024, 6:07am

These are good debugging instructions, but the setup itself should be fine - I’ve had it running for over a year without problems. You might also want to check how you are passing the USB device to the VM (though that should not matter regarding ZWJS updates).

othermore · August 12, 2024, 8:06am

Thanks for your help!!

It actually breaks even if I don’t make any change (just “save”).

I think I have it disabled. I have this parameter set:

  "mqtt": {
    "disabled": true,

The thing is that I can’t use the ZWave JS UI to edit the config, I need to make the changes to the file, save it, and then reboot for it to load. I’ve made the other changes you suggested.

That’s “hassio/addon/core_zwave_js/info” right? This is not installed

Not sure how to do that without loosing all my setup. I have 100+ devices at home (50+ comming from ZWave) and I just don’t want to loose all my automations, settings, etc. Can I really refresh the VM without loosing everything?

I have it passed on as an USB 2.0 (OHCI + EHCI) controller, with a device filter as follows:

Name: Sigma Designs, Inc.
Vendor ID: 0658
Product ID: 0200
Revision: 0000
Port: 0003

othermore · August 12, 2024, 8:25am

So I’ve made all the configuration changes. Nothing changed… I clicked the component’s restart button at 10:23 and it made the system crash again.

Attaching the “dump” and the log here.

The one thing I’ve noticed after these changes is that ZWave JS UI seems to be taking longer to connect to the driver, and I see the following logs while it’s connecting:

2024-08-12 10:25:01.247 INFO Z-WAVE: Setting user callbacks
2024-08-12 10:25:01.261 DEBUG SOCKET: Event INITED emitted to IqrjeI3E5RunIpx1AAAn
2024-08-12 10:25:02.544 DEBUG SOCKET: New connection MQyXPh0ewmAnoRsVAAAp
2024-08-12 10:25:02.544 INFO Z-WAVE: Setting user callbacks
2024-08-12 10:25:02.561 DEBUG SOCKET: Event INITED emitted to MQyXPh0ewmAnoRsVAAAp
2024-08-12 10:25:05.994 INFO APP: GET /health/zwave 301 0.776 ms - 191

These are the logs around the time of reboot (I clicked reboot exactly at 10:23, so you can see there’s nothing around that time, i.e. the system is just normally behaving and then the next thing logged is after the reboot. The freeze and whatever happens around it is not logged, but there’s some trash stored into the file when the crash happens.

2024-08-12 10:22:46.339 DEBUG SOCKET: User disconnected from 7CRWjf8L3gf6a8DaAAAF: transport close
2024-08-12 10:22:46.345 INFO Z-WAVE: Removing user callbacks
[TRASH HERE -- It's just a bunch of null bytes that I can't paste here]
2024-08-12 10:23:37.406 INFO APP: Listening on port 44920 protocol HTTP
2024-08-12 10:23:37.410 INFO ZNIFFERMANAGER: Zniffer is DISABLED

Furthermore, the whole system logs (read with journalctl) are showhing the same thing:

Aug 12 10:22:53 homeassistant addon_a0d7b954_zwavejs2mqtt[422]: 2024-08-12 10:22:53.318 INFO Z-WAVE: [Node 006] Value updated: 128-0-isLow true => true
Aug 12 10:22:53 homeassistant addon_a0d7b954_zwavejs2mqtt[422]: 2024-08-12 10:22:53.366 INFO Z-WAVE: [Node 006] Value updated: 67-0-setpoint-1 22 => 22
Aug 12 10:22:53 homeassistant addon_a0d7b954_zwavejs2mqtt[422]: 2024-08-12 10:22:53.699 INFO Z-WAVE: [Node 006] Is now asleep
Aug 12 10:22:55 homeassistant addon_a0d7b954_zwavejs2mqtt[422]: 2024-08-12 10:22:55.090 INFO APP: GET /health/zwave 301 0.474 ms - 191
-- Boot d6face4539ba478aa53742eec4e6e432 --
Aug 12 10:23:31 homeassistant kernel: Linux version 6.6.33-haos (builder@bfa75d3d66a7) (x86_64-buildroot-linux-gnu-gcc.br_real (Buildroot -gd03dda4f) 12.3.0, GNU ld (GNU Binutils)>
Aug 12 10:23:31 homeassistant kernel: Command line: BOOT_IMAGE=(hd0,gpt4)/bzImage root=PARTUUID=a3ec664e-32ce-4665-95ea-7ae90ce9aa20 rootwait zram.enabled=1 zram.num_devices=3 sys>
Aug 12 10:23:31 homeassistant kernel: BIOS-provided physical RAM map:
Aug 12 10:23:31 homeassistant kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable

Attached files:

Thanks all!

freshcoast · August 12, 2024, 3:07pm

The point of these suggestions is to troubleshoot whether your problem is related to the VM, HAOS, the add-on, etc. At this point it sounds like you don’t have a working setup, and no one else seems to be having the same problem, so you’re going to have to try different things to start eliminating possibilities. I have personally never seen these add-on errors you’ve reported on this forum or elsewhere.

You can create a new VM and not remove your old one. Then install the add-on and see if it’s working or not. If it works, than that could indicate something is wrong the the original VM itself.

You can install Z-Wave JS UI standalone outside of the VM without deleting your current VM. See if it’s working outside of the VM. If so, it provides some clue that something is wrong the the VM or the add-on.

You should also still provide driver debug logs, the UI logs seem to be useless. I would also upload them to a pastebin to make it as easy as possible, or share the log file, no one wants to download and extract a zip file…

othermore · August 12, 2024, 3:29pm

Thanks!

I thought the files I shared are the driver logs. Those are the files written within the store directory in the JS UI. (store/logs/) and I also attached the nodes dump because I saw it in the same page I was sent to about driver logs. Which ones are needed otherwise??

This is the pastebin version of what I think are the driver logs after it was set to debug level. I couldn’t include all logs there, so I focused on the time around the reboot. You can see all the trash in the file when the system crashes, just after 2024-08-12 10:22:46.345 (probably because the file is incorrectly closed due to the crash?)

I have also bought a brand new AEOTEC ZStick gen5+ (I have gen5) and a gen 7 one, and I intend to migrate to gen7 and see if that works (I first need to update the firmware, for which I want to have a backup in case I brick the stick, hence the gen5+ one that I also got).

I have no experience in installing JS ZWave UI outside of Home ASsistant, that’s going to be an experience. But I’ll try these two things as well.

Cheers,

freshcoast · August 12, 2024, 3:34pm

No, it’s not a driver log. Driver logs are named zwavejs_<date>.log.

Going to that link says “This page is no longer available. It has either expired, been removed by its creator, or removed by one of the Pastebin staff.”.

You just download the binary from their GitHub and run it.

othermore · August 12, 2024, 4:33pm

Thanks so much once again!! I feel so lost.

Re the logs, sorry for the confusion. I have to set things up based on the config file rather than the UI, which makes things harder… Here I come again:

Full Driver log file (in google drive)
Gist of Driver logs, around the date of a crash (Tried to reboot the addon at 18:26:00, last log line before that is 2024-08-12T16:25:55.023Z)
Gist of the prior UI logs again, not sure why it disappeared. Crash in this one happens after 2024-08-12 10:22:46.345 – The null bytes are not displayed here

Unfortunately, the driver log (in silly mode) does not contain anything between the moment I push the restart on the addon (which triggers the crash) and the moment the addon starts again.

I also made the test of setting up a brand new VM with the last version of HAOS available for VirtualBox. Then set everything up, installed the ZWave JS UI Addon and set the integration to get all the devices… and I was even able to load a backup and control my house from there. I was really thinking it was going to fix the issue. But then I tried to reboot the Addon and everything crashed just as in my old VM, so the issue persists.

I went back to a fresh install that was working well (able to restart), and manually set the minimal things from the UI: the keys and the device. After saving that, the addon went unstable again, so I’m pretty sure it’s an issue between my stick and the Zwave JS UI.

The one thing I noticed that was different after the update that started breaking everything up was the new “long range” keys. I wonder if my device is incompatible with those and that is creating the problem.

mundschenk-at · August 12, 2024, 5:13pm

The Long Range keys are not used with a 500 series controller.

mundschenk-at · August 12, 2024, 5:15pm

When you say “reboot the addon”, you mean restart it from HA’s UI, right?

PecosKidd · August 12, 2024, 5:18pm

Let’s double check the use of Add-Ons. Go to Settings → Add-ons. There should be “Z-Wave JS UI”. Is there also “Z-Wave JS”? If not, that’s good. But if it’s listed, click on it and verify it is not running (the dot in the upper right should be red not green.)

othermore · August 12, 2024, 5:22pm

Thanks all for your help and support!!

Yes. There’s the option to start and the option to restart (sorry, I have it in Spanish and don’t know how it shows up in english)

Nope, only Z-Wave JS UI:

PecosKidd · August 12, 2024, 5:27pm

Good. Thanks for ruling out that relatively common problem.

Tomahawk · August 12, 2024, 5:29pm

I have used Z-Wave UI for some weeks and then it started to act up after some Add-on updates.

Restarted the Add-on and HA several times without help.

When I rebooted the system all my Z-Wave problems went away.
(Settings-> Right corner … → Restart HA->Advances option->Reboot System)

othermore · August 12, 2024, 5:38pm

Thanks! Right now I can’t really do that, because the system crashes when I try a reboot. My guess is that one step of rebooting is stopping all the addons and when it tries to stop the ZWave JS UI addon, it crashes.

I have hard rebooted the system many many times, and also correctly rebooted it a lot of times (when I set the addon not to start at boot, the system is not running in the next boot and then I can properly reboot the system).

edwin-2023 · August 12, 2024, 5:42pm

You write that you are running inside virtual box. What if the host has a problem. Did you perform a full memory and disk test?
What is de host system?

othermore · August 12, 2024, 6:02pm

That’s a good one. The host system is a Linux Debian system. I have created yet another Home Assistant virtual machine, and I found the same exact issue, so I doubt a HD or Ram Memory issue would be the problem here (I guess it would not affect always the same things, and even if it did, it would not affect the two VMs equally, because they ran on different Hard Drive sectors and memory segments).

But I may have some issue with how the host is passing the USB controller to the VM. Not sure how to debug that though… I checked the sys log and there are no relevant logs around the crash either (all I see is that the network interface entering or leaving promiscuous mode, which is expected).

mundschenk-at · August 12, 2024, 6:19pm

Have you disabled USB power saving on the host? That is also a common cause of problems. I am running Proxmox and don’t see any such issues. Just to be sure, when you say “crashes the system” you mean the VM running HAOS, not the host?