My experience in converting from Deconz to Zigbee2mqtt

Hi folks. I recently converted my 50+ node setup from deconz to zigbeetomqtt, and I thought I would do a brief writeup with the hope that others doing a zigbee conversation would find it useful.

I have been running deconz for controlling my zigbee network before I was running Home Assistant. It was one of the addons for homeseer (and much better than their organic one they had), and so when I went to Home Assistant, I just kept all the devices and conbee hardware that ran on a separate raspberry Pi and used the deconz integration to bring them all into HA. Things generally worked very reliably, with the exception of some aqara sensors that seemed to drop off often, especially in some areas.

But the thing that caused me to finally make the move was this zigbee Tuya temp/humidity sensor: Tuya Smart Temperature And Humidity Sensor WiFi APP Remote Monitor For Smart Home var SmartLife WorkWith Alexa Google Assistant| | - AliExpress, which uses AAA batteries instead of a coin battery. That was important for my use case, which was sticking these in a couple freezer compartments to keep tabs on the temps and if the drawer was left open by the kids (sometimes it’s hard to tell if the freezer drawer is completely closed). The coin batteries never lasted long at low temps, but you can get lithium cells that work fine in the cold. The sensor is cheap, speaks zigbee 3.0 and a nice form factor with long lived easy to replace batteries. I like it a lot.

Deconz didn’t pair with these, and along with the desire for better diagnostics (zigbee2mqtt is really quite good!), I made the decision to switch. I did look at ZHA, but I run HA in a virtual machine (so much easier to backup and restore with a VM snapshot!), and the server is in the basement, so I really like the ability to run it remotely on a Pi in a better physical location.

@Hedda did reply in a thread that ZHA supports ser2net (it’s not in the documentation) so that would have worked as a workaround to a direct USB connection, but I really like having the Zigbee function running separately from HA. Having to update to the newest version of HA to get new device support was really unappealing to me, as I usually run a month or two behind the most current HA release to make sure the bugs are worked out (the early monthly releases are really betas), and my family really depends on HA running for a number of functions. This sort of issue was one reason why Chrome destroyed IE as a browser - IE updates came with windows updates, which weren’t always safe to do, whereas Chrome updates happened independently of OS updates. That allowed Chrome to innovate and iterate much faster than IE in the corporate world.

Anyways, I got one of the new Sonoff CC2652P sticks as a coordinator: SONOFF Zigbee 3.0 Dongle P ZB Dongle Plus Wireless Zigbee Gateway Support SONOFF ZBMINI ZHA Zigbee2MQTT USB Capture with Antenna| | - AliExpress, and updated the firmware, and the installed the zigbee2mqtt docker on the Pi. That Pi also runs zwave2mqtt, so all the docker setup was already there. I also used a 6 ft USB extension to keep it away from any interference from the Pi, that also has it’s Wifi and BT shut off. It’s a Pi 3, so it doesn’t have the usually noise from from it’s USB3 ports like a Pi 4 does, but it was also important to keep it separated from the conbee stick that was still running deconz. I actually kept the channel number set to 25 for both of them, as that was the best channel for my network (because of all the WiFi AP’s around), and I didn’t want to try to change channels after the conversions were all done.

Then I started converting the powered devices, and then renaming each one as I did it. I was shocked at how much better zigbee2mqtt’s OTA support was - it was trivial to upgrade my Tradfri bulbs and repeaters. I did all the powered devices first and then the battery devices after, since many of the sensors have trouble switching to a different router. Things were much easier to pair than in deconz, though my securifi peanut plugs needed some manual tweaks to be set up properly, and after a firmware upgrade were greater routers. Some of the sensors had issues staying online. I discovered some of my old OSRAM bulbs were not good routers and causing problems, but a combination of replacing bulbs and using zigbee2mqtt’s permit to join a specific device fixed those. Zigbee2mqtt’s diagnostics were great - so much better than the deconz map, and made me wonder why I put up with phoscon for so long.

After I converted everything, I stopped deconz and removed the deconz from the Pi. It served it’s purpose well over the years, but better options are now available.

My new Tuya sensors paired easily and everything went well, though re-pairing everything took awhile. Zigbee2mqtt also opened up new levels of functionality compared with deconz. In deconz, my peanut plugs were switches, but in zigbee2mqtt, they also could tell me power consumption. So many devices had more visibility on battery levels or other options compared to deconz.

I think most vendors do a lot of tweaking to the zigbee standard so you will be motivated to use their bridge and service platform (using cloud based) to make more profits than just the hardware alone. So zigbee configs will always need a lot of hand tweaking - a lot of that has been invested in zigbee2mqtt and it shows.

Anyways, I am very pleased, and I should have made the leap a long time ago. The diagnostics that helped me debug where issues were was very handy, esp where deconz wouldn’t show where a sensor was connected to, and it’s as fast to execute commands as it was before - much better than my zwave devices, even though the network is much smaller.

Anyway, I hope this is useful for folks who are thinking about platform choices or on deconz like I was.

17 Likes

Thanks for the info - I’ve been considering making the jump myself. Were you able to pair multiple lights in one go? I have a number of lights that are all on the same power, so having to plug one in at a time is going to be tedious, particularly for the GU10s.

I’m also on a VM, but moved deCONZ to the HA add-on last year. I still run MQTT in a separate VM, along with other services like Node Red. I’m guessing I would have to delete the add-on first, or I won’t be able to rename to the same names - after a VM snapshot of course for when I screw it up a couple of times.

Can you also confirm that you had no problems with all devices getting the same names as before? I have some Aqara power plugs that show up as lights, and therefore get the ‘light’ domain’ with deCONZ, but perhaps ‘switch’ with z2m. Any other wisdom you can impart would be appreciated.

EDIT: Looking into it further, I will also have to pass the Conbee through to a different VM, as I think I’m more likely to do the docker install of z2m rather than the HA add-on. This is definitely an exercise for when I’m the only one in the house!

I had also made the jump from DeConz to Zigbee2MQTT and never looked backed.

Besides that the Conbee II stick is an outdated piece of hardware, so no reason to stick with DeConz because of the hardware.

CC2652P + Z2M simply is a great working combination.

What aqara power plugs are switches? Can you share a diagnostics report on it?

Like this one. Although the price has doubled since I bought them. How do I get a diagnostics report?

1 Like

On the … Drop down Button on the deConz integration

Really interesting information. How long did the actual migration of the zigbee devices take as I assume you cannot have the old Deconz network running at the same time as the new one.

This?

Here it is showing up as a light in Phoscon:

Having said that, after checking, it comes across to HA as switch.xiaomi_1, so all good there.

1 Like

In the home assistant interface. Great that it works as expected!

You can run both at the same time, having the Deconz using the Conbee and Z2M with some other coordinator, like a Sonoff 3.0 Dongle. They can even share the same zigbee channel, you just need to make sure they have different PanID. Of course easier (to avoid errors) to have them on different channels.

I migrated over several weeks from ZHA to Z2M.

I recently went from decent to ZHA, why did you opt for zigbee2mqtt instead of ZHA? I use mqtt a lot, but didn’t see any benefits in this case. Is the device support better?

Suggest also recommend these tips and best patches regardless of which Zigbee solution you set up:

https://github.com/home-assistant/home-assistant.io/pull/18864

and

https://www.home-assistant.io/integrations/zha#best-practices-to-avoid-pairingconnection-difficulties

I think is important to understand Zigbee signals are weak and have poor penitration of common building materials so users have to rely on a strong Zigbee network mesh (meaning many Zigbee Router devices) and also keep in mind that Zigbee devices are very sensitive to RMF/EMI/RMI interference.

When in doubt, add more mains-powered Zigbee Router devices that act as good signal repeaters. Personally, I recommend “IKEA Trådfri Signal Repeater” as they are cheap and better than average.

It makes it much easier to troubleshoot and find the real root cause if have already optimized your setup and environment with the aim to work around interference and bad signal reception of a poor mesh.

For that specific dongle there are also more tips on updating firmware and the physical connection, etc.:

https://community.home-assistant.io/t/iteads-sonoff-zigbee-3-0-usb-dongle-plus-based-on-texas-instruments-cc2652p-20dbm-radio-mcu-now-sold-for-14-99/340705/

1 Like

As mentioned above, you can have deCONZ/Phoscon, Zigbee2MQTT, and the ZHA integration all running at the same time as long have they each have their own dedicated Zigbee Coordinator adapter.

That type of environment with separate Zigbee networks that are not aware of each other is no different from having Zigbee devices connected to several different proprietary Zigbee hubs/gateways/bridges like the Philips Hue Bridge, IKEA Trådfri Gateway, Samsung SmartThings Hub, Xiaomi Mi Smart Home Hub, Tuya Smart Life Zigbee Gateway, etc…

As long as they have different Zigbee PAN ID then they should not be aware of each over and not interact in any way, though as mentioned their signal can possibly interfere if using same radio channel.

Yes, much better device support although ZHA is getting better as well.
Personally I also like the Z2M dashboard, visualization and user experience much better than ZHA but that’s personal taste.

I kept both networks up on the same channel - 25. The coordinators were set apart by 3 ft or so, but I didn’t want to use an alternate channel like 15 for z2m, and then have to move it back for fear of having to repair a bunch of the sensors. It worked fine in my case. I did move the powered devices from the outside in (in terms of distance from the coordinators) after moving a couple repeaters at first. Seemed to go well over a few hours (I had family and work obligations to attend to, so could not do it all in one block).

Just wondering if this methode would also work: Backup and Migrate Your Home Assistant Zigbee Network! - YouTube

Moving it from ZHA to z2m

ZHA currently doesn’t support remote operation - you have to use ser2net to remote a stick, which isn’t actually documented. Also, that Tuya sensor I wanted to use isn’t currently supported by ZHA, at least as far as I the April release.

But the big issue is that I didn’t want to have to upgrade HA to get better device support in a new version of ZHA. I like having these functions be independently upgradeable. As I said in the note, new HA releases are often problematic, and I didn’t want to be forced into upgrades to get better device support.

Zigbee is and always will be a heavily human driven process in terms of getting specific devices with full functionality enabled because of the way the industry incentives drive proprietary behavior. Being fully zigbee spec compliant doesn’t seem to help that much. Z2M has has had years of tweaking that really enables great support. ZHA may get there, but it will take time, and given I needed remote support anyways, and wanted the release decoupling, Z2M was really the best choice for my complex network.

I’m sure ZHA will get better over time, but just as a user, since I was comfortable with the complexity of running with a remote Pi and running MQTT (I use zwave2mqtt as well), it was sort of the obvious choice for me.

Michael, one of the nice things about z2m is if you set permitjoin to true in the config, it comes up in inclusion mode without a time out. That enables you to add bunches of devices without having to turn it on for 5 minutes at a time. If you disable that through the webUI and turn it back on, it does make it temporary.

You can set the entity name explicitly by renaming the device and using the option to reset the home assistant entity name. Because the original deconz names I had were things like sensor_68, I chose to rename the devices, but you could change it to pretty much anything you like. You’d have to delete the old device with that name first though (I think).

Z2M does set the device type. With the new renaming function in HA, you can use it turn a switch into a light with a different entity name, and leave the original device as a hidden entity. That seems like a better way to solve your problem than depending on whatever device irregularity in the zigbee driver exists.

Rather than running z2m in another VM, I would encourage you to think about running it in a raspberry pi that is in the best location to talk to devices. That avoids all the USB passthrough issues, and gets you more optimum device routing. I use an old Rpi 3, and it’s plenty good enough for this function.

I use the builtin MQTT addon, but that’s not required by Z2M. That’s one of the nice things about the MQTT architecture.

+1 here. Z2M diagnostics and status info is VERY good. I really appreciate that the developers gave us so much information to help debug bad device behavior but also very explicit network routing information. I may be more sensitive to this because of the issues I have with the overtly wrong info the zwave2mqtt map displays, but for complex networks, you really need the diagnostics.

Is there not a problem of once you move any mains powered routers (bulbs etc) over to the new zigbee network that you destroy the mesh on the old zigbee neiwork?