Aeotec Z-Stick Gen5+ - backups, NVM, etc

[Swallowtail (Simon)]

There are some older topics here on backups, but as a new Z-Stick user there are some changes and I have some follow-on questions.

General backups:
I backup HA daily to Google Drive using GitHub - sabeechen/hassio-google-drive-backup: Automatically create and sync Home Assistant backups into Google Drive, and do a monthly full snapshot of the HA VM backed up to AWS.

Z-Wave backups:
I can see in ZWave JS to MQTT control panel that I can backup nodes.js, and it now also allows to backup the controller NVM. I’m assuming that neither of those (ZWave nodes details or NVM) are captured in my General HA backup routine, and that I need to be backing them up separately.

My questions:

1. Is HA general backup combined with backing up nodes.js and controller NVM within ZWave JS to MQTT sufficient to recover in the event of catastrophic failure (e.g. HA or Z-Stick failure)?
2. Is there a way to automate the Z-Wave backups, or is it just a case for now of “Remember to do it when you change anything in ZWave”?
3. Is there a way to have a second Z-stick kept in sync, other than manually restoring NVM, to keep a backup ZWave network available?

I want to minimise the impact of / improve speed to recover from any failure.

[Swallowtail (Simon)]

No-one has any ideas?

[mwav3 (Tim)]

@Swallowtail

Is HA general backup combined with backing up nodes.js and controller NVM within ZWave JS to MQTT sufficient to recover in the event of catastrophic failure (e.g. HA or Z-Stick failure)

A HA general backup will backup the addon data, which should include the nodes.json and settings.json data. It will not include the NVM data, as that resides on the zstick itself. My thoughts on NVM though- if something goes wrong with it, your zwave stick is likely shot, and you’ll need a new one anyway. Restoring the nvm to the new zstick, if it’s the identical model and firmware, “should” restore the zwave network without reincluding everything. However, if it was me, I would just exclude and reinclude everything at that point. I only have about 22 zwave devices though. If you had 100 or so, might be a different story. The warning in the zwavejs documentation about potentially bricking the device with an nvm restore is enough to steer me away from trying it though.

Z-Wave JS - Z-Wave driver written entirely in JavaScript/TypeScript

@Swallowtail

Is there a way to automate the Z-Wave backups, or is it just a case for now of “Remember to do it when you change anything in ZWave”?

As noted above, it should backup your essential settings, config, and most importantly device names, through the automated backup/snapshot process. However, an old IT saying is you never really have a backup unless you’ve tried to restore it.

@Swallowtail

Is there a way to have a second Z-stick kept in sync, other than manually restoring NVM, to keep a backup ZWave network available?

No. The zwave specification only allows one primary controller. Additional controllers can be added but will be secondary. It will act as a slave to the primary and will not have the whole network config or security keys to communicatewith s0 and s2 devices. If the primary controller dies, the secondary one will be useless and lost without it.

[Swallowtail (Simon)]

Thanks - useful. Given the criticality of Zwave in our smart home with about 35 devices / 500 entities I am keen to minimise the risk of downtime in event of failure. Excluding / reincluding the devices, and dealing with the resultant entity changes, automation changes, template changes, ZWave device parameter changes, etc. would be an extensive amount of work.

@mwav3

A HA general backup will backup the addon data, which should include the nodes.json and settings.json data.

Not sure about this though. I cannot ‘find’ nodes.json or settings.json on the HA filesystem, and from ZWaveJS2MQTT’s “Store” listing it looks to me like that resides on the stick?

It would be good to have more confidence in what gets backed up than it “should” be included.

I feel like this is a useful discussion to be having, and not just for me

[mwav3 (Tim)]

I run zwavejs2mqtt in docker, and my config directory for it is mapped to a persistent folder on the host I can browse to, and have configured to backup to Google drive weekly, so I know mine is backing up. I see the nodes.json and settings.json files on my computer in the zwavejs2mqtt store directory. They are created by zwavejs2mqtt and are not stored in the zstick.

However, since I don’t use the addon version, I preface it with “should” be part of your snapshot. A supervised home assistant install seems to throw files allover the place, so locating them can be difficult, and not everyone of them is backed up.

These files have to be somewhere in the system though, zwavejs2mqtt can’t work without them. If you’re taking a backup of the entire VM you’re capturing everything that way, except the NVM, which is on the zstick. And since the NVM is not as volatile as an ssd, spinning hard disk, or sd card which are constantly being read and written to, I’m not sure you should worry about backing it up too much. But I can understand wanting too of course, but you’ll have to do it manually and periodically

This post gives a directory where zwavejs2mqtt stores files. I would try and find them there.

ZWave JS suddenly losing devices and entities names . System renaming them by itself! #2018

If you have access to the host OS, the files for zwavejs2mqtt are located in: /usr/share/hassio/addons/data/a0d7b954_zwavejs2mqtt/store you’ll have 3 files (they may have different names): <8_digit_hex_value>.json <8_digit_hex_value>.metadata.jsonl <8_digit_hex_value>.values.jsonl Just copy and paste them to the zwave_js’s container’s location. You’ll have to locate those cache files. The path should be similar. Here’s a guy made about HassOS users making the switch:

Also see this note in the documentation for zwavejs2mqtt about backing up

https://zwave-js.github.io/zwavejs2mqtt/#/usage/setup?id=backing-up-settings

[ddwdiot]

I’m new to Home Assistant… and not very IT savvy either… having moved from OpenHab quite recently (and very happy to have done so). But I’ve checked my Z Wave backup routine since the move across and it still works.

I back up the stick occasionally as described here Z-Stick Gen5 Backup Software : Aeotec Help Desk

As I say above, I tested for future disaster by copying the backup to a second (identical) Z-Stick and starting home assistant up with it… all working well.

Not sure if it’s specific to my controller, which is Aeon Labs Z-Stick Gen 5 Plus. And I realise that it would be better to have something that did this automatically. So will watch for more expert answers with interest.

[Swallowtail (Simon)]

@mwav3

Also see this note in the documentation for zwavejs2mqtt about backing up

https://zwave-js.github.io/zwavejs2mqtt/#/usage/setup?id=backing-up-settings

The page says:
## Backing Up Settings

Settings, scenes and the Z-Wave configuration are stored in JSON files under the project store folder. It is a good idea to backup those files, which can be done by backing up the store or using the import/export buttons.

The only access I have to “Store” is through the Control Panel, from which yes I can backup (manually) the whole store by selecting the store and then clicking on the gearwheel icon then downloading it. There is no way on that page to reverse the process from a clean (empty) start.

I see no “Import/Export” buttons for the Store.

In the main control panel section there is “Backup” under Advanced, which just exports nodes.json which as far as I can see would be useless for recovery… e.g.:

  "1": {
    "name": "Z-Stick",
    "loc": "Study"
  },
  "5": {
    "name": "Study downlights",
    "loc": "Study"
  },

@mwav3

I run zwavejs2mqtt in docker, and my config directory for it is mapped to a persistent folder on the host I can browse to, and have configured to backup to Google drive weekly, so I know mine is backing up. I see the nodes.json and settings.json files on my computer in the zwavejs2mqtt store directory. They are created by zwavejs2mqtt and are not stored in the zstick.

I have no visibility of the store as mentioned above, as I suspect is the case for most users.

This is an amazing set of add-ons, very well written and obviously under active development… I’m surprised that clear backup/restore process is not more than just a footnote. And this is an AUTOMATION platform lol… yet the only way to simply backup the ZWave store in a HA install is by going into the control panel and downloading it? What am I missing here?

With 25y+ in IT I know from experience that the only way backups get usefully done is if they are automatic, hands-off, and part of a process. It also needs to be tested and validated; I’ve validated that I can restore my pre-ZWave HA install by recovering the VM “file”, but at this stage I am unclear on how I would recover to fully operational in the event of ZWave corruption, failure or mis-configuration.

@ddwdiot

I back up the stick occasionally as described here Z-Stick Gen5 Backup Software : Aeotec Help Desk

As I say above, I tested for future disaster by copying the backup to a second (identical) Z-Stick and starting home assistant up with it… all working well.

To do that means pulling it out, moving it to a Windows PC, running the backup, powering down the VM, replugging, and restarting. I get that it works… and that’s great - at least it’s a way to backup… but it’s not conducive to regular and testable backups.

Ideally the add-on would have a service call (so it’s schedulable) which allows Store backup/restore to/from the HA filesystem, where it can be included in the many great backup add-ons available.

Anyway, I’ll get off my soapbox. And keep manually downloading the Store / triggering NVM backups whenever something changes in my network… and hope for the best as I have no way of testing that either of those processes will help if something breaks.

[mwav3 (Tim)]

I definitely can understand the frustrations, as the backup should be continuous and automatic. One of the reasons I don’t use the addons and use Home Assistant container, with installing the addon’s equivalent docker image instead, is I have much more control over the system and know exactly where all the files are, and can automate backups and know the files are getting saved. The issues you’re running into with locating the files are one of the reasons I don’t install Home Assistant this way. Sure, the addon/supervised option is easier to setup, but with easy, you lose control over the system. With your IT background, this is an install method you may want to look into some day

Linux

The backup documentation does say the addons folder is included with the backup, but then has a caveat that says “only manually installed or created add-ons, not those installed from the store”, which is unclear to me what exactly that includes. There could be a different addon that you can use to locate and backup these files, I’m just not familiar with all of them. It would be nice to hear confirmation that someone using zwavejs2mqtt has actually restored the backup and recovered, and what their experience was. Hopefully another user can chime in on that.

Common tasks - Supervised

@Swallowtail

just exports nodes.json which as far as I can see would be useless for recovery

I think you may not have all the pieces of the puzzle about how zwavejs works, and what is stored in the program vs what is stored on the zstick. The majority of the zwave network’s configuration, including nodes, node ID’s, devices types, and routing, are stored in the NVM of the zstick. The configured parameters are stored in the individual zwave devices themselves. This means if your entire system attached to the zstick crashed, and you put the zstick on another machine with a clean install of zwavejs2mqtt, all devices would remain included, and you would have the same node ID numbers. The devices would preserve whatever config parameters were programmed to them.

What you would not have with moving the zstick to a clean install is the information stored in zwavejs2mqtt’s nodes.json and settings.json files. You would not have the device names, security keys, and any other settings you customized from the default in zwavejs2mqtt. So instead of “Study downlights”, it would just be “Node 5”. Any securely included device, like a lock, would not function without the security key, and if you didn’t know what the key was you would have to set a new one and then re-include the secure device with the new key. Without those files, it would make recovering from a failure difficult as you’d have to go in and rename everything, and re-include secure devices. This is why the nodes.json file is actually very important to backup, as it will make recovering from a failure much quicker.

As far as automating NVM backup of the zstick, I have never run into any zwave hub/vendor that does that automatically. To backup the NVM, the zwave network needs to be temporarily stopped, so that could be one reason why its generally not done. Although the windows solution using the Gen5 backup software is less then ideal, since its the backup software provided by the vendor that makes the device, I would tend to think it would be a more reliable backup if you ever have to use it vs the NVM backup created in zwavejs2mqtt.

[freshcoast (Z-Wave AI Bot)]

@Swallowtail

This is an amazing set of add-ons, very well written and obviously under active development… I’m surprised that clear backup/restore process is not more than just a footnote. And this is an AUTOMATION platform lol… yet the only way to simply backup the ZWave store in a HA install is by going into the control panel and downloading it? What am I missing here?

You are missing two things:

1. zwavejs2mqtt is third party software. HA developers have no influence over which features it provides, that’s up to an entirely independent developer.
2. The Supervisor backup and restore process is documented (see link already provided). I think that documentation is more than a footnote.

Yes, the supervisor backups include all of the addon data, at least for z2m. You can even do partial backups of individual add-ons, and you can automate those backups with service calls in HA. Did you try it?

I don’t use Supervisor or addons, but I keep a VM around for cases like this. I fired up the VM, installed the z2m addon, got my z-wave network running (with a test network). Then I did a partial backup of the addon, then downloaded the backup file. Opening the tar file, I see all of the store files, including the network cache. So it seems there’s already an adequate solution to address at least some your concerns?

That leaves the NVM backup. Well, admittedly that’s a more challenging problem, but a solution does exist, for those using zwavejs2mqtt. Please note that I said “a solution does exist”, I did not say it’s the best possible solution.

First, I would agree with @mwav3 . Personally, I don’t think backing up the NVM is something you want to do all the time, and I would want to do it manually just in case something goes wrong. I don’t think it’s as risky as doing a restore, but as mentioned, it disables the radio until the process is finished. This could take several seconds to minutes. I would probably only do it after including a node, then at least the composition of the network is preserved. Some routing information may be out of date after a restore, but at least all the nodes are there.

Therefore the easiest solution is to just use the z2m UI to download the NVM backup, and you can keep that somewhere safe. If instead, you do want to automate the backup, and/or you would like the file to be saved in the store directory, then there’s advanced functionality to do that as well. Luckily, there’s an example in the z2m documentation.

const fs = this.require("fs-extra")
const { storeDir } = this.require('../config/app')
const data = await driver.controller.backupNVMRaw()
await fs.writeFile(storeDir + '/NVM.bin', data, 'binary')

It will initiate the NVM backup and then store the resulting binary file in the store directory (you can monitor the process in the debug logs page). If you take this approach, then your supervisor backups will include everything you need to recover from a disaster: the z2m configuration, the z-wave network cache files, and the controller NVM.

If you want to automate the process, you’ll have to enable MQTT and use the driverFunction command with the above code. HA doesn’t provide the functionality.

If you want to run the driver code manually, you can copy and paste it into the Driver Function dialog that is available in the Advance control panel controls.

If you think z2m should save the NVM into the store directory in a simpler way, you should submit a feature request to their Github project.

I would test out the backup and restore process using these series of steps. Note that I’ve never tried it, but this is how it would work in my head:

1. Buy a backup Gen5+ stick. At this point in time zwave-js does not support NVM backup and restore to different types of sticks. It will support this in the future though, so you would theoretically be able to backup from a Gen5 and restore to a 700 series.
2. Make a full backup so you can revert anything that goes wrong below.
3. Make an NVM backup using the code above.
4. Make a z2m partial backup using the supervisor.
5. Download the partial backup, and extract the NVM.bin file.
6. Delete the z2m addon, which wipes the configuration
7. Remove the backup USB stick
8. Insert the new USB stick
9. Install the z2m addon again
10. Start the addon and navigate to the z2m UI. Do minimal configuration to (USB path) to get the driver started. Restore the NVM backup from the UI by uploading the manually in the advanced controls UI.
11. Stop z2m, and restore the partial backup, it should restore all the addon files.
12. Start z2m again
13. Profit???

[freshcoast (Z-Wave AI Bot)]

@mwav3

Although the windows solution using the Gen5 backup software is less then ideal, since its the backup software provided by the vendor that makes the device, I would tend to think it would be a more reliable backup if you ever have to use it vs the NVM backup created in zwavejs2mqtt.

Benefits to using z2m for backup and restore over Silicon Labs PC Controller software:

• Less downtime
  • No need to relocate the stick to another PC
  • Backup can be performed while the network is up and running
• No extra software to install
  • I think the Aeotec software is specific to only Aeotec Gen5/Gen5+ controllers
  • The 700-series requires creating a Silicon Labs account and downloading the Simplicity Studio software, which is a giant 500MB installer.
  • Both of those require Windows, z2m works in Linux
• You can’t backup any of the Pi-hat controllers (Razberry, Z-Pi 7), due to the Windows requirement [1]
• The node-zwave-js author has written a tool to parse its version of NVM backups, which lets you read the data to discover information about the network. It’s an interesting exercise. More importantly, this work enables a future feature:
• You will be able to backup and restore from any controller (as long as it has a minimum version of the z-wave SDK, which means upgrading a Gen5 to 1.02) to any other controller. The implication is that you would be able to upgrade from a 500-series to a 700-series simply via a backup and restore (or go back to a 500, because 700 controllers fail in some networks).
• No reports of the backup and restore in z2m not working, so I wouldn’t say it’s less reliable than the PC Controller software. They use the same serial API commands AFAIK. If course, Silicon Labs knows their hardware the best, so maybe.

[1] Technically, you can backup and restore NVM in Linux using the Silicon Labs zw_programmer software distributed as part of the Z/IP Gateway SDK, but this is out of the reach of most users.

[mwav3 (Tim)]

Thanks @freshcoast . Based on that I would be much more confident using z2m for the NVM backup now. Also good to know the home assistant backup does include the zwavejs2mqtt addon store directory.

@freshcoast

Profit???

This is my favorite step

[Swallowtail (Simon)]

@freshcoast

I fired up the VM, installed the z2m addon, got my z-wave network running (with a test network). Then I did a partial backup of the addon, then downloaded the backup file.

For clarity, how did you “do a partial backup of the addon”? Because when I click on the button that says “Backup” / “Export” I get a copy of node.json. Nothing else.

So already, there is either a UI or a UX issue.

[Swallowtail (Simon)]

@freshcoast

Therefore the easiest solution is to just use the z2m UI to download the NVM backup, and you can keep that somewhere safe. If instead, you do want to automate the backup, and/or you would like the file to be saved in the store directory, then there’s advanced functionality to do that as well. Luckily, there’s an example in the z2m documentation .

const fs = this.require("fs-extra")
const { storeDir } = this.require('../config/app')
const data = await driver.controller.backupNVMRaw()
await fs.writeFile(storeDir + '/NVM.bin', data, 'binary')

Useful - yes. Documented under API calls, not in regard to backups. I searched the documentation for ‘backup’ before originally posting, and none of the resulting documentation provided me with what was required.
Thanks - I’ll keep digging.

My overall position remains that as a competent technical person with many years experience who has learned smart home, ZWave and HA in a short time, it was not apparent (and is still not fully) how to achieve a resilient backed up and restorable ZWave setup. That part should be easy.

@freshcoast

this is how it would work in my head:

1. Buy a backup Gen5+ stick. At this point in time zwave-js does not support NVM backup and restore to different types of sticks. It will support this in the future though, so you would theoretically be able to backup from a Gen5 and restore to a 700 series.
2. Make a full backup so you can revert anything that goes wrong below.
3. Make an NVM backup using the code above.
4. Make a z2m partial backup using the supervisor.
5. Download the partial backup, and extract the NVM.bin file.
6. Delete the z2m addon, which wipes the configuration
7. Remove the backup USB stick
8. Insert the new USB stick
9. Install the z2m addon again
10. Start the addon and navigate to the z2m UI. Do minimal configuration to (USB path) to get the driver started. Restore the NVM backup from the UI by uploading the manually in the advanced controls UI.
11. Stop z2m, and restore the partial backup, it should restore all the addon files.
12. Start z2m again
13. Profit???

This is great, thank you (I’ll test it) and is the sort of thing that should be in the doco. I’ve posted a discussion which I will separately follow up with on github.

@freshcoast

You can even do partial backups of individual add-ons, and you can automate those backups with service calls in HA . Did you try it?

I was not aware of that functionality at all, it has not come to light in anything I have done so far in HA. I will investigate…

BTW folks, this discussion is exactly what I was looking for - really useful advice that is not obviously apparent to new-ish HA learners. Thank you.

[freshcoast (Z-Wave AI Bot)]

I followed the link to the HA documentation that @mwav3 provided. There is literally a section that has step-by-step instructions on making a backup from the UI. https://www.home-assistant.io/common-tasks/supervised/#making-a-backup-from-the-ui

You are mixing up the functionality provided by the z2m control panel, and functionality provided by the HA supervisor. Again, z2m is a standalone product. The add-on version is just a wrapper that allows you to install it in the HA UI. The functionality in the z2m web UI is completely separate from the add-on services provided by HA. Once you are in the z2m web UI, you are outside the domain of HA.

[Swallowtail (Simon)]

@freshcoast

There is literally a section that has step-by-step instructions on making a backup from the UI.

Yep, and as I said just above in my last reply I was not aware that could be applied to add-ons:

@Swallowtail

I was not aware of that functionality at all, it has not come to light in anything I have done so far in HA. I will investigate…

BTW folks, this discussion is exactly what I was looking for - really useful advice that is not obviously apparent to new-ish HA learners. Thank you.

After the comments in here and more reading and learning, now I know that yes you can backup (automatically) zwavejs2mqtt’s store from the HA supervisor.

@freshcoast

You are mixing up the functionality provided by the z2m control panel, and functionality provided by the HA supervisor.

If I am it is because it is not clear from a UX or UI point of view, and/or it is not clearly documented. Forget from your point of view who knows how it works, take it from a new user who does not. Those are the ones who need to be able to easily work it out… the zwavejs2mqtt Control Panel literally has a button that says “Backup”. But it only does ONE FILE, and it is not enough for a recovery.

I’m going to wrap this one up here and continue comments on Github, as from a HA point of view the situation is now clearer, thank you for commenting.

1. You can back up the zwavejs2mqtt add-on (including its Store) from HA backups, this can be automated and restored, and is documented
2. You cannot automate NVM backups at this point
3. The add-on’s Backup button only does nodes.json, which is not enough for an effective backup
4. A restorable position should be achievable by automated add-on backups from step 1 and NVM backup from within the add-on when changes are made to the ZWave network

[blhoward2]

So, a few things as a member of the zwavejs project. We can’t access the HA config directory, at all. The addon runs in a docker container that is purposely segregated. We have no control over this.

As far as I know, the Supervisor backup can’t access the HA store. I don’t know, though, as that’s an HA decision.

Zwavejs2mqtt is broader than the addon. A percentage of users don’t use HA at all. In fact, I’m the only zwavejs driver dev that uses HA. We can’t do things too HA centric. And even for docker users running HA, that process can’t access the HA config directory.

Backing up the nvm requires a soft reset and some other things that don’t play well on some systems. It’ll never be automated from our perspective. It’s also unnecessary. HA could chose to do this but it would be a waste and would risk bricking the stick every backup.

The nvm only changes when you include or exclude a device. It’s only necessary to backup then, not weekly, etc.

[jasonshearer]

Great thread @Swallowtail

I stumbled on it through the Git post. I have added to my backup process when I add/change a z-wave node. Trigger a NVM backup, backup the entire store zip, put it on my NAS that gets backed up to Backblaze.

Thanks!
Jason

[dzmiller]

@mwav3

As far as automating NVM backup of the zstick, I have never run into any zwave hub/vendor that does that automatically.

Homeseer does it automatically. It’s done when the NVM is changed. NVM doesn’t need to be updated on a schedule, just when changed.

[dzmiller]

Having worked with NVM backups for awhile now, my opinion is the best practice for large z-wave networks is to buy two identical Aeotec (or possibly zooz) USB z-wave sticks. Keep one for use in hardware failure.

I include zooz because they have documented NVM conversion procedures and do provide tech support. But the safest practice for large zwave networks IMO is to stay to the center of the mainstream, which is still probably the aeotec stick.

In testing I’ve been unable to backup and restore to the common HUSBZB. Combining z-wave and zigbee hardware is a bad idea beyond little systems anyways.

I’m backing up NVM weekly, and ideally at every inclusion. As I said above best pratice would be for automatic backup when zwave is already down for inclusion/exclusion.