Migrating to Z-Wave JS broke my whole Z-Wave network and DB (Discussion and Suggestions)

Configuration Z-Wave
,
9

I’ve been on ZWaveJS since my first Z-wave device and while I currently have everything working, it hasn’t been simple or easy. After a while I realized I’d have been better off with ZWaveJSMQTT because of its enhance UI and its ability to do firmware updates. But after reading posts from people who attempted THAT migration, I’ve decided it’s not for me. It seems possible, but if you make a mistake, you’re in deep weeds and it’s a long road back.

So all I’m offering here is empathy. I also wish I’d never heard of Z-wave and just stuck with wi-fi - but I’d never say that out loud.

My hope now is that ZWaveJS is moving forward and will eventually include diagnostic and maintenance capabilities comparable to ZWaveJSMQTT. Does anyone know if that’s likely?

10

Considering that Z-wave js is only about 18 months old and they wrote the first version in less than a month, then I would say they are just getting started. They have been adding features consistently at each new release. This is just speculation on my part but I feel confident.

1 Like
11

Hereby is my journey to Z-Wave JS. (Make note of my latest additional information at the end of this post)

Last Thursday I made the change. It was all but a smooth transition. Everybody is talking about the better speed and responsiveness of the Z-Wave JS integration in relation to the legacy Z-Wave integration. But I can’t agree with that.

After starting the migration wizard it told me that it wasn’t able to migrate almost all of my devices. I really don’t see a lot of value in this migration wizard in that part. It helps you install Z-Wave JS and in the end remove the legacy Z-Wave integration but that’s about it.

My Z-wave network was working fine with the legacy Z-Wave integration. I’ve moved over from my Fibaro HC2 controller to HC2 somewhere half 2021 and I found out that the Fibaro was much quicker and more reliable than the Z-Wave integration of HA. But I wanted to consolidate my solutions so I worked around the ‘minor’ issues hoping that Z-Wave JS would solve that in the future. But actually, it didn’t.

Z-Wave JS still is missing some basic features like ‘associations’ (yes, I know… it’s on the roadmap) or just a setting for a global polling cycle (My Fibaro controller had this feature and did calculate the value based on the number of devices you had. In my case it was a global polling cycle of 300sec, this global value could be used at the device level, or you could just add a special interval for a specific device).

Another thing is that I now have some devices that don’t report the status back when operated at the device level, those devices worked fine with the legacy integration. So, I’m not sure what causes that problem, have to use a script now to update the status with “zwave_js.refresh_value”. Those devices are from Fibaro (most of my devices are). It is an older device, a dimmer (FGD211 with firmware version 1.7). I have another one that’s working fine (newer firmware, version 2.2)

I know that Greenwave devices need to be polled to get the power (and other) readings. That was needed also within my Fibaro HC2 controller, so I’ve also used the “zwave_js.refresh_value” service call in an automation rule for that.

I really hope that the (IMHO) basic functionality will be fixed soon.

I’m running HA as VM in Proxmox with the following versions (all latest versions usable at the time of this migration):

  • Host Operating System: Home Assistant OS 7.4
  • Supervisor Version: supervisor-2022.01.1
  • Home Assistant Core version: core-2022.2.8
  • Z-Wave JS integration: 0.1.54
    • Z-Wave JS to 8.11.6
    • Z-Wave JS Server to 1.15.0
      • With a Z‐Stick Gen5 USB Controller (AEON Labs ZW090, Firmware: 1.2)

Edit: March 23, 2022.
The problem about devices not reporting back was due to a faulty unit that flooded the network.
That was also the reason why my network responded so very slow.
All is fixed now, and everything responses lightning fast again. Happy user again.

1 Like
12

Guess it’s time, whether we’re ready or not.

This release of Home Assistant Core 2022.3 will be the final release that provides these integrations. Both the old zwave and ozw integrations are pending removal for Home Assistant Core 2022.4.

13

Yup, decided to make the jump because this is the last release with the old integration.

All my devices were moved over, but the biggest pain was that the all the device ID’s were changed, so I’m currently spending the time to setup my automatons & Lovelace cards again with the right names and right device name.

I would be lying if I didn’t say this was high on the PITA list.

14

Yeah, I’ve been following this topic for a while, I have two houses both running the deprecated zwave, with the only issue being newer devices come up as unknown. This isn’t a big deal since I’ve gone to scripts anyways to configure the zwave parameters in the devices - to make it repeatable. Anyways, it’s been very reliable when running and had zero problems for a year plus. So,

“if it ain’t broke don’t fix it”

While I know I can run the old code as a custom component, I would like to experiment with migrating, but I am concerned about not be able to roll-back. I’m highly concerned with the opening statement here that says the zwave network would not start and roll back failed. Is there any insight into why that happened? Does zwave-js change the config on the stick or network Config on the nodes?

Thoughts on this being a backup strategy. Using HA container, MariaDB and Aeon Labs Gen 5 stick. This assumes I’ve already updated to latest HA.

  1. Backup the gen5 stick.
  2. Shutdown HA
  3. Duplicate the HA config folder
  4. Create a new container
  5. In new Config disable MariaDB (usually do this by editing connection string to be invalid)
  6. Startup JWAVEJS-MQTT container, get it setup with the stick and network key
  7. Wait for the interview process to complete / go around and wake up nodes
  8. Start up new HA and try migration

If it fails, I can go back to the old container and if zwave fails to start I can restore stick from backup,

Any other advice?

15

Attempted the migration tonight to get to 2022.4 and just wanted to add in another tally for the migration breaking a rather large implementation. 99 Z-wave nodes, 300+ Z-wave entities and 0 devices or entities came across. Additionally, every node has only a Ping and Health entity. Re-interview does nothing.

On top of this, restoring from a full backup done about an hour before the migration attempt does not restore to prior state. Z-wave JS integration and add-on remained preventing Z-wave from starting and Z-wave contained a note that the migration was already completed. This makes me completely distrust the built-in backup\restore function.

Luckily, I do hypervisor level backups and was able to do a full VM restore until I can dedicate probably a full day to try again and exclude\include + rename everything back.

Very frustrating indeed. IMO there should be very obvious disclaimer in the migration documention to not rely on Home Assistant backups and that you may be rebuilding Z-wave from scratch.

16

I thought I had it bad with my 53 Z-wave devices. 99…yikes! I finally got around to trying the migration today (via ZwaveJS2MQTT) and with a lot of trial and error and a bit of blind luck, I was able to get the devices I wanted switched over (I had several dead nodes that I was happy to leave behind). However, I lost all of my entities and am left with only the Ping and Health entities for each node, so I have no idea where to go from here except maybe to remove each node and manually add it back again. What a pain. I love Home Assistant (came over from Smart Things a few years ago) and don’t see myself ever going back, but I’m disappointed in this implementation right now.

17

Keeping an eye in this thread, I dont dare to migrate after reading this. Has anyone done the migration smoothly?

As I feel now I have to plan a weekend for doing this with my danalock and all other deviced. It doesnt feel like the will be migrated smoothly atm

18

Ok, after 4 migration attempts (and full VM restores) I was sort of successful on my fifth attempt. This appears to be mostly just poor documention online and in the wizard. Piecing together snippets of information from various posts I had several failure points.

First attempt: I used an old S0 key that was incorrect, my fault. Found my correct key in /.storage/core.config_entries.
Second attempt: The old Z-wave key was in the 0x12 0x34 etc format and the new integration and migration wizard wants that key in the 1234 format. I didn’t see the required format listed in the documentation or wizard and found it in some thread on here.
Third attempt: Something went wrong here and I’m not sure what. When I clicked stop network, HA rebooted. Tried again and many nodes were unavailable. I believe this attempt failed because I attempted the migration before the old Z-wave had finished starting and interviewing devices.
Fourth attempt: This succeeded but I ended up with no entities and devices were named by Node #. At this point I fiddled around with the Z-waveJS2MQTT control panel and realized after a while entities were starting pop back up (not named correctly) and devices were finally being detected correctly (eg GE In-wall Dimmer, etc).

In my last attempt before throwing in the towel and just redoing everything I tried just being patient.

  1. Start the migration
  2. Wait for any line powered devices to show as ready via the old integration
    a. Pretty much all of my battered powered devices said not ready and would never become ready despite waking them up (and successfully performing actions with them)
  3. Stop network
  4. Install the new + supervised add-on via migration
  5. Wait at the screen that says 5000 devices wont be migrated. DON’T CLICK MIGRATE. I just let it sit at this screen for about an hour. Nothing changed on this menu but when I finally clicked migrate, I suddenly had almost all of my nodes correct and entities were correct.

I had to exclude and re-include all of my battery powered stuff but luckily that is only 4 minimotes, a wallmote, and a few motion sensors. I had 1 of 30+ GE switches come across but lost naming, easy enough to fix. All other GE switches came through fine. Also, all of my Inovelli Red switches came across with lost names, easy enough to fix.

So, I am officially migrated now without having to tediously redo everything but documention both in the migration guide and migration wizard should be greatly improved to reduce frustration.

Side note: For renaming things that fail, a good resource is the migration JSON that gets generated at /.storage/zwave.legacy_zwave_migration. It lists entity to node associations to get them back to what automations expect.

1 Like
19

I would also recommend that the HACS integration WATCHMAN be installed. This will check all automations to make sure you don’t have misnamed entities and devices. It really is handy especially as your system grows.

20

Great info! How did you convert the S0 key in the end? My key is written like: 0xXX, 0xXX, and so on 16 times. I guess you remove the commas?

Another thought, is it possible to use the deprecated zwave and the new Zwave JS integration at the same time? In that case I could just screw the migration tool and move my devices one by one as I want myself without having to worry about wrecking the whole thing.

21

remove the commas in addition to the ‘0x’ from the beginning.

so “0x34,0x4A,0x7C…”

becomes “344A7C…”

No. Both servers can’t access the same controller device at the same time.

1 Like
22

ah thanks! and, sounds logical about the controller

23

Same issue here, I waited this long to migrate thinking all of the bugs would be worked out by now, but the migration wizard said it could not migrate any of my 60+ nodes. Has anyone figured out a solution? Do bugs need to be filed? My feeling is that manual renaming of an entire network of nodes and entities is not an acceptable migration path. There’s got to be a better way…

1 Like
24

I too waited this long to migrate hoping the bugs would have been worked out. It seems that is not the case.

I did the upgrade and am actually surprised there was no warning message considering the consequences are pretty well known. Maybe something like, “there are 25 scripts and 50 automations totaling 175 different entities that will be affected by this upgrade” would have been helpful.

And then like many others, the majority of the devices did not migrate. The worst part is going to be the renaming of all of the entities to work with all of the old automations and scripts.

I don’t have a day+ to devote to that in the near future, so I have rolled back to the prior backup.

1 Like
25

Ok, so I finally solved a way to make a backup of ha on my unraid server and tested it out yester day. After that it was time to make a try with the migration. And I have to say that it went very good despite the circumstances. I only have 17 devices and after filling out the legacy s0 key the wizard migrated everything. All my entitynames were lost as I suspected. What took the longest time for me was actually figuring out which device was which. But as soon as I found my system it went pretty smooth. The most important thing was that I never had to re-import any of my devices, even the danalock worked instantly as soon as I found the entity.

Luckily I had a wattage-card that sorted every device from most to least in a list and I knew which most of those devices were. I could rename them from there and afterwards go to the entities page through zwave js integration and find the device and rename its other entities. Also another tip is to just go each physical plug/device and put them online manually (if you have something like the tellstick slim plug that has a button on it), then you can easily find the one that turned on in the entities list and rename it. Or even easier, just turn on the device behind the plug and look for the change in power. And make a copy of all the text on the developement tools page (entities) before you start so that you know all the old entitynames

My 17 devices took me 3-4hours to fix but now its working

PROS:
-The network seems way faster, the danalock reacts way faster now, could be due to removel of dead devices mabye
-I can finally update ha again
-I could remove old devices that I could not get rid of before
-less entities (no zwave. and no “previous_reading”) and a more modern gui with its icons and design in the entities area

CONS:
-It takes TIME to migrate, be sure to have a tested backup
-Cant do much from the integration, its very basic and I’d love to see a separate area for all the zwave nodes/entities and not have to filter the “zwave js” in entities tab
-cant see average round trip time anymore, I had a card that sorted the longest zwave RTT device, nice to have for troubleshooting and knowing where to place your network nodes

26

one weird thing afther the update of ha is that zwave thinks that my entity sensor.z_stick_gen5_usb_controller_node_status is dead. The network seems to be working though.

27

Yes, the prior set of node stats was very helpful.

28

I just completed migrating a house. I’d been using a parallel system for testing for last 4 weeks during my own time (and swapping the zwave stick to it) - to work out the bugs, automation issue and gaps in entity ids and functionally. I highly recommend this approach. Did the cutover in about 1 hour following a 35 step migration process…

Anyways I started creating a tutorial but haven’t seem a lot of pickup so it remains unfinished, glad to work on it if someone wants it,