Guide to deCONZ high level debugging

Configuration Zigbee
1

Here is Kenneth’s guide to deCONZ high level debugging

I see many questions and answers related to deCONZ issues where it is clear that is may not be so easy to understand what is happening where.

There are many interfaces to keep track of and it is important to know them when you want to debug a problem.

In Text graphics the chain looks like this

Devices <-> Conbee USB stick <-> deCONZ software <-> deCONZ Home Assistant Integration <-> Home Assistant “Engine”

And then there is Phoscon which is a Javascript app you load to your browser and it talks directly to the deCONZ software

Instead of the Conbee you may have a Raspbee hat but it works the same except for the USB stuff and the USB device name will be different.

The Device to Conbee interface is a radio interface running on 2.4 GHz in the same spectrum as Wifi and Bluetooth.

Problems with the radio interface will manifest themselves as

  • Unstable and occasionally slow connection between devices and Home Assistant
  • One or few devices showing unavailable at times

Problems with radio interface will normally NOT BE

  • All devices unavailable
  • Conbee stick not available

The solutions for radio problems are:

  • Computers create a lot of electromagnetic noise around them and this can make it difficult for the Conbee to receive the more weak signals from battery devices and devices that are far away. The recommendation is to connect the Conbee to the computer via a 1 meter long USB cable.
  • You can improve the range of the Conbee by placing it so radio waves meet less obstacles from furniture etc. Place the computer in a central position
  • Make sure you have several mains powered devices that work as routers (bulbs, smartsockets, or zigbee repeaters)
  • Make sure all Zigbee mains devices are powered 24/7. If you turn off a device on the wall socket or a lamp switch, it will take time for the Zigbee network to heal and you will experience bad connection. Especially battery powered device can take time to work again if their previous partner is suddenly not there

Wifi and Zigbee share the spectrum. The channel numbers are very different from each other. If you hard code your Wifi router and access points to channel 1 or channel 6 and set Zigbee to channel 25 then there is no overlap at all. This can help with interference problems. At least from your own Wifi network. If it is your neighbors Wifi that gives interference then try and move the computer with the Conbee to a different place in the house away from the interferer.

If a device is permanently not available then it may be the pairing that has gone wrong, maybe because of a bug in the device. Then the cure is to try to re-pair the device. First try without resetting the device. If not reset the device and pair it. If it still does not work then the device is probably broken. You did check the battery, right?

The Conbee to Computer interface is via a USB port.

If deCONZ cannot see the USB stick then there are two possibilities

  • The stick is broken. It is a rare fault but it will happen
  • You run Raspberry Pi 4 with a USB disk connected to a USB3 port – this is known to not work. Google for more info on this special case. Moving disk to USB2 normally fixes it (and slows down disk access)
  • The device name is wrong - this is much more likely to be the issue

The Conbee Stick will always be discovered as a /dev/ttyACM# device where # is usually 0. If you have more than one device that registers as /dev/ttyACM… then you will experience that after a boot of the computer ttyACM0 and ttyACM1 will have swapped places. The typical situation is that you have both a Conbee II stick and an Z-wave stick like the Aeotec stick.

In modern distributions the device manager (udev) will create symbolic links to the devices that are predictable. Something like /dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE1961234-if00

There is a problem however. The current version of the deCONZ Addon does not work reliably with this symbolic link. On my installation it works 8 out of 10 times you start the Addon.

There has been one attempt to fix this but it did not cure the problem. The Addon is as I write this still unstable in this respect.

My advice for the device name problem is

  • If you only have one device that is registered as /dev/ttyACM0 then use that and you are good.
  • If you have two /dev/ttyACM devices then you have these choices
  1. Use the /dev/ttyACM device that is most likely to be chosen at boots and use this in the Addon settings. Reboot again when it goes wrong
  2. Use the /dev/serial/by-id/… link and be prepared to restart the Addon several times manually until you succeed.
  3. If you are on a computer with a full Ubuntu/Debian based operating system (not HassOS) you can add the Dresden Elektronik repository to your apt environment and install deCONZ via apt. That is what I ended up doing. I got tired with the problems with the Addon. deCONZ running on bare metal auto discovers the Conbee device and does not care if the device changes name after a reboot

Note that seen from Home Assistant a missing connection between deCONZ and the Conbee will appear as all devices not available. This is the same symptom as deCONZ not running, or broken link between the Home Assistant Integration and deCONZ.

To check, open Phoscon - and verify that it can connect to the Conbee stick.

If you run the Addon check that it is running.

If you run deCONZ installed from apt - check that the deCONZ daemon is running. It is called deCONZ

OK we assume that you have deCONZ running and it can see the USB Stick.

Next interface is between the Home Assistant Integration and deCONZ

It is important to understand the difference between the deCONZ Integration and the deCONZ Addon.

deCONZ is the software written by Dresden Elektronik. The API portion of it is open source. Home Assistant developers have put this in an Addon (a docker image that the Home Assistant supervisor starts). The deCONZ Integration is written by the Home Assistant community and part of Home Assistant.

You can install deCONZ on a different computer on your network. It does not have to be the Home Assistant Addon. No matter if you run deCONZ as the Addon, or on same computer in the host OS or on a different remote computer, the connection type is the same. The Home Assistant Deconz integration connects via TCP/IP network (physical or virtual) to the deCONZ software. If you can see and confirm that deCONZ is running, and it can see the USB stick in Phoscon, then the problem must be that the Integration does not connect to deCONZ.

Note that part of the initial pairing of the Integration with deCONZ you need to create an API key. In Phoscon you go to the Gateway settings and find the button Authenticate App. Then you have 60 seconds to trigger the Integration to connect. If for some reason you get something reset that you should not have then this can be a thing to check

If the integration suddenly stops connecting to deCONZ after a restart or an upgrade can be more tricky to debug. It should show up in your Home Assistant log if this link is broken. Many people have reported problems with recent updates to the Addon but I have not see any clear single root cause and I guess most people just remove the integration and add it again and clean up the mess it makes when device names are going back to the defaults

If you want to go the route of trying to re-install the Integration then the best advice is to walk through all deCONZ sourced devices in the Home Assistant Integration and check if their names match the names in Phoscon. If not - rename the device to match in Phoscon. This way when you remove the integration and add it again, most names should become the same. If you renamed entities instead, then consider using this for the naming in Phoscon. After all it is the entity names you want preserved as they are used in your automations and in the Lovelace cards

Long post – but I hope it can be a reference to be used when people have one of the common issues in the all too long official deCONZ thread.

30 Likes
2

Fantastic guide!!!

3

Hi Kenneth, thanks very much for this guide. Very helpful indeed.
I recently read this post: https://community.home-assistant.io/t/zha-and-conbee-ii/118779/38 that includes a discussion about the ZLL profile, keeping bulbs and sensors on different meshes, and avoid using bulbs as Zigbee routers. Do you think this is only a potential issue with ZHA or have you made similar experiences?
Thanks

4

Well. I am not a Zigbee expert. But my experience is that a mix Philips Hue and IKEA devices work great and the more devices I have added to my house, the less problems I see. The Philips 4 button remotes worked poorly on a the Philips Hue hub. After I moved everything to deCONZ they are super reliable.
In a normal home you have bulbs/sockets mixed with remotes and motion sensors. Typically from Philips and IKEA as they are more readily available world wide.
I have poor experience with Osram sockets and have removed then all in faviour of Philips.
And then the more advanced users supplement with door/window sensors. I have good experience with Aqara sensors. It all plays well together.

For a short while I had two parallel Zigbee nets. The Philips Hue hub with all lights and switches with Philips own 4 button switches of which at least one would be flashing red and not working daily.
And I had the Aqara sensors and a couple of remotes on Zigbee2MQTT with a USB generic stick. And I had to add repeaters to get the most remote sensors to work.

After I put everything under Conbee II/deCONZ I have a much more stable Zigbee performance

I have had a couple of incidents where a bulb could not be controlled individually but only via group and that was fixed by a mix of reboot and power cycling an IKEA bulb. I since replaced this by a newer one and problem has not shown up again. There are firmware bugs in individual devices that can cause issues. I still have a Philips Hue hub ready to plug to be able to upgrade Philips devices. I used that recently to upgrade all the 4-button remotes
That is time consuming as you have to reset them and pair them with Philips hub before upgrade. And re-pair them back on deCONZ after. But it improved their stability quite a lot
IKEA stuff can be upgraded in deCONZ

I still think my simple advice will work for most and is more based on my professionel experience with radio equipment

5

Kenneth, thanks for the detailed reply. Really appreciated.
Rob

6

Kenneth - do you know of any guides on how to set up deconz on a Pi (not running HA) and connect it to HA? I’m not sure I understand when you say the connection type is the same."

I currently have my HA instance running on a Win 10 VM in Virtual Box - and periodically it drops the zigbee connection, throwing a “failed to set dtr/rts” error in the VM log. So I am exploring a more stable install outside the VM - and I have a spare Pi 3B+ laying around. Any help would be appreciated, as the Wife Approval Ratings are plummeting ! :slight_smile:

7

Hi

Simply follow the installation from the official Phoscon page

https://phoscon.de/en/conbee/install

For Raspberry Pi you can choose between an image for SD card or installation under Raspbian (Raspberry Pi OS)

The documentation is a bit behind. The repository has been updated to also work under Ubuntu 20.04. Just follow the installation exactly as described.

I recommend the beta instead of stable so you have support for the latest devices and so far the betas have been very stable in practical

I recommend changing the computer to having fixed IP address as this lowers the risk of trouble when Home Assistant need to rediscover the deconz on another IP address.
You can do this on the Raspberry/Ubuntu machine or in your router (personal preference)

Once you have deconz running on the Raspberry or a Ubuntu machine you simply add the deconz integration in Home Assistant and as long as the two are on same subnet it discovers the Deconz machine.

To keep the raspberry or Ubuntu up to date just run normal sudo apt update and sudo apt upgrade and this keeps both OS and deconz up to date.

One important detail. To run the deconz with the GUI the Linux must have a desktop environment. It can be any.

You only need the GUI when you need to upgrade firmware on your IKEA and Osram devices or you want to see the mesh.

For normal adding and removing devices you just need the Web application Phoscon.

When you run deconz this way both Phoscon and the rest api runs on port 80.

The documentation misses the last bit of info

To run deconz as a daemon you enbale it with

sudo systemctl enable deconz
sudo systemctl start deconz

If you want to run the GUI for a while you simply log in on the machine (VNC or directly). Then you open a terminal and stop deconz with

sudo systemctl stop deconz

And then you start the GUI deconz from the menu system. I think it ends in programming for some odd reason.

Deconz works like normal with Home Assistant in this mode also

When you are done, you close down the gui deconz and again from terminal

sudo systemctl start deconz

And then you can logout from the desktop again.

The documentation also describes how to run deconz in Docker but I see no advantage doing that on a machine dedicated to Deconz only.

Just for information. The HA addon is Deconz in the GUI version running in a docker with a minimal vnc desktop and it is most of the time running latest beta.

2 Likes
8

I just moved my ConBee II from a NUC running inside a VM alongside Home Assistant to a separate Pi for ZigBee and Z-Wave. I documented my steps here. Just make a backup from Phoscon before you migrate and restore the backup after you migrated.

9

Thanks so much for the detailed response - especially the bits that are behind in the documentation. Oddly, after futzing with this thing for a few hours…it decided to connect as soon as I reached out to you. Go figure. I still think I’ll move it over to the Pi.

1 Like
10

Love that page - I wish I had done a better job documenting everything I’ve done over the last 2 years - it’s been quite a learning experience. Appreciate the link!

11

I wish so as well. That’s why I started from scratch with a second system while keeping production running. This was also my motivation for moving the ConBee to a separate Pi to be able to use it with two HA instances at the same time. Also the zigbee network keeps running when I mess around/restart HA → less downtime → more WAF :grinning:

12

Yeah - this VM is really starting to bother me - to the point I’ve been thinking of setting up a parallel system for better WAF. I assume I can just use a snapshot from my current instance, and be up and running on a Pi pretty quickly? For the Pi version, I think I’ll turn off all history/recorder stuff to avoid the inevitable SD card corruption.

13

@Burningstone I’ve followed line by line your excellent instructions, but running into a snag here. Anything I can check to see where I’ve gone wrong?

  • 192.168.1.223:5900 --> Via VNC, this should take me to ZigBee mesh. I set the password in the docker-compose.yml (with double quotes around the password) but VNC also asks for a username. If I use the pi username “pi” it just logs me into the Rasbian pi interface instead of the mesh - the same as if I didn’t have the port appended to the IP.

  • http://192.168.1.223:8080/pwa --> should take me to the Phoscon Web UI but I get a site cannot be reached error.

14

Add network_mode: host to let docker use the host network. I dit it as well in my setup but forgot to commit the change

15

Add it to the docker-compose.yml file? Just anywhere will do?

16

Anywhere inside the deconz: part will work. Just needs to have the same indentation as container_name etc. E.g.

version: "3"
services:
  deconz:
    container_name: deconz
    devices:
      - /dev/zigbee
    environment:
      - DECONZ_WEB_PORT=8080
      - DECONZ_WS_PORT=443
      - DECONZ_VNC_MODE=1
      - DECONZ_VNC_PORT=5900
      - DECONZ_VNC_PASSWORD=yoursupersecretpassword
      - DECONZ_DEVICE=/dev/zigbee
      - TZ=Europe/Zurich
    image: marthoc/deconz
    network_mode: host
    restart: unless-stopped
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./deconz:/root/.local/share/dresden-elektronik/deCONZ

You can then also remove the “ports” part, as this is not needed when running in host network mode.

1 Like
17

@KennethLavrsen
Thank you for the information

My problems started few weeks ago.
1 of the 10 restarts will result in a working zigbee. Hope HA team will find a solution for this problem.

I have also problems in displaying the gateway page. This problem is also there for long time.

Only method to acces the page is remove “2” in url behind the gateway.

/settings-gateway2.html

18

Did you open an issue on github? Are you sure it is due to the HA implementation and not due to the ConBee/Raspbee or DeCONZ?
Everything works fine for me.

19

Not 100% sure.
But all works find inside phoscon.
Maybe a better option to do what you have done bring phoscon outside HA

I can control all my zigbee devices in phoscon and after few restart it wil also work via HA.
Sometimes there will be auto fix and it will work after sometime without doing any thing.

HA is running in VM. (Proxmox) Conbee is connected to the host.

20

I can definitely recommend that path. I can remember that I had to remove ad re-add the integration in the past when I was still running it on the same Proxmox VM that HA is running in and I restarted the Proxmox host. As far as I can remwmber this issues resolved themselves later, but didn’t check again.