ZHA to Zigbee2MQTTT migration using backup file

NOTE: I searched through past topics and couldn’t get to a straight answer so starting a new thread especially now that HA Core 2022.10.3 is out.

I have a 70+ device network established with ZHA but its limitations are making me want to switch (again) to Zigbee2MQTT (Z2M for brevity).

Now here it gets interesting: I was originally on Z2M, my server crashed (including the backups) so I had to rebuild my instance quickly and used ZHA pairing my existing coordinator to it. Much to my surprise, it rebuilt my network INSTANTLY finding all my previous devices.

With a smile on my face I thought “ok this is gonna be pretty easy…”. After running into a few issues with the actual behavior of devices (ZHA recognizes fewer triggers than Z2M, some devices straight up don’t work…) so I thought I could switch over the coordinator and just get my network back on Z2M.

So I

  • removed the ZHA integration (all devices are gone at this point)
  • install Z2M, MQTT, pair the coordinator with Z2M, start everything
  • nothing shows up and nothing wants to pair with it anymore

I know ZHA can produce a “backup” of the network configuration, has anyone found a way to import that backup into Zigbee2MQTT and restore the network in that direction (ZHA to Z2M)?

Thanks

Suggest you ask the same question here → https://github.com/Koenkk/zigbee2mqtt/discussions

(while if for vice versa migration to ZHA instead ask here → https://github.com/zigpy/zigpy/discussions).

Think I once read that a former Zigbee2MQTT user got help from a ZHA/zigpy developer to migrate from Zigbee2MQTT to ZHA by extracting Zigbee network data from a backup file, but I have not read about someone doing a reverse migration going to ZHA to Zigbee2MQTT.

Migration between different applications is not supported out-of-the-box, as currently, meaning that you can only restore backups in ZHA that were made from ZHA, just as you can only restore backups in Zigbee2MQTT that were made from Zigbee2MQTT. So even though ZHA and Zigbee2MQTT are partially using the same backup format for the Zigbee network layer part (i.e. “Open ZigBee Coordinator Backup Format”) but unfortunatly neither of those applications today have built-in support to automatically extract that part in order migrate a Zigbee network backup made in another application, so a user can currently not perform a backup in Zigbee2MQTT and then restore that backup image in ZHA or vice versa as such backup images from each application not only include the Zigbee network layer backup but also includes the application layer backup. Regardless I doubt that backup images will ever become fully compatible at the application layer (which includes device configuration) between two totally different applications such as Zigbee2MQTT and ZHA, so you should probably only ever expect to at best be able to restore just the Zigbee network backup in a migration which only includes paired/joined devices and network security information so that you at least would not have to manually re-pair/re-join any devices. Suggest that you post a new discussion in zigbee2mqtt for Z2M or zigpy repositories for ZHA to start discussions if you have a specific use case → Koenkk/zigbee2mqtt · Discussions · GitHub and zigpy/zigpy · Discussions · GitHub

from @Hedda Migration between different applications is not supported out-of-the-box, as currently, meaning that you can only restore backups in ZHA that were made from ZHA, just as you can only restore backups in Zigbee2MQTT that were made from Zigbee2MQTT.

Ok I get that maybe backups are never going to be fully compatible, but why did ZHA recognize my entire network instantly while Z2M doesn’t?

If I could answer that question (and documentation on both products is incredibly inadequate to do it short of reading the code) then I would be able to solve my problem pretty easily.

Edit: I also asked the same question on zigbee2mqtt

no answer so far but I’ll gladly take an upvote on that question to raise awareness :slight_smile:

2 Likes

If requesting “Zigbee2MQTT to ZHA Zigbee network migration ability via backup image restore compatibility” you could post a new feature request here in Home Assistant forum in order to try raise awareness since the ZHA component is integrated in Home Assistant core → https://community.home-assistant.io/c/feature-requests/13

However if asking for “ZHA to Zigbee2MQTT Zigbee network migration ability via backup image restore compatibility” you should really instead post feature request(s) to Zigbee2MQTT (and zigbee-herdsman) to raise awareness in those projects since Z2M is a third-party application, see → https://github.com/Koenkk/zigbee2mqtt/issues/new/choose and also https://github.com/Koenkk/zigbee-herdsman/issues/new

Might as well also post requerst here too → https://github.com/zigpy/open-coordinator-backup/issues

Anyway, I believe that both ZHA/zigpy and Zigbee2MQTT/Z2M developers had some kind of cross-application migration compatibility support in mind when they together created that “Open ZigBee Coordinator Backup Format” → https://github.com/zigpy/open-coordinator-backup

That is, the concept of that project was conceived as a first step in standardizing Zigbee backup format.

https://github.com/zigpy/open-coordinator-backup/blob/943c93a90b439830afde2ce885a9f7e5606b8b34/README.md

Now that both ZHA/zigpy and Zigbee2MQTT are using that format it should be easier to build on it.

Did you end up find a way to migrate from z2m to zha?
All I seem to be able to with the new update is to import coordinator info. But I’d like to import all the zigbee devices list as well so I don’t have to re-pair them all

FYI, it is the Zigbee Coordinator that contains the Zigbee network info which includes (paired) devices, so no need to re-pair devicices. What is not included is application configuration and naming of devices:

https://github.com/zigpy/open-coordinator-backup#readme

https://github.com/zigpy/open-coordinator-backup/issues/3

1 Like

Correct me if I misunderstood you, but I added the coordinator to zha integration and none of my devices were shown?

Do you mean that you did a backup to file inside Zigbee2MQTT and then want to do a restore from that backup file inside ZHA after converting that backup file to the format that ZHA expect? Because that is what would be is needed. It will not be as simple as physically moving the adapter from Zigbee2MQTT to ZHA, as then ZHA will just initialize the adapter to factory default and create a new Zigbee network on it. Again suggest reading above and post question about migration to https://github.com/zigpy/open-coordinator-backup and/or https://github.com/zigpy/zigpy/discussions

1 Like

The issue for migration is not the coordinator config. That can be manually transferred from ZHA to Z2m, takes a few minutes but all the config is in cleartext for both systems.

The issue is to convert the knowledge of the actual paired devices in the network.
Both ZHA and Z2M use a database file SQLite3 zigbee.db in ZHA and a plaintext database.db in Z2M.
What one would need is a converter from one DB to the other. Ie a script that would read the ZHA SQLite3 zigbee.db and write a database.db plaintext file for Z2M. Or vice versa :wink:

However it would also appear that the Z2M database contains mapping from the interview that you would not find in ZHA (ZHA may have different names, converters, quirks, …).

I don’t know the inner workings of Z2M, maybe the solution would be to do some migration wizard that would read the ZHA database and magically interviews the device via IEEE address (and any other data needed) and adds it. Some finetuning may then still be required, but at least the basic heavylifting is done.

6 Likes

Can you please write a litte guide what does that mean which steps are required?