Help Please the wife is losing faith! ZHA/Zigbee router devices becoming Unknown

PS I take it there was no signs of anything else in the logs, do I need to provide more?

Errors like this that are showing up

Logger: homeassistant.components.zha.core.channels.base
Source: components/zha/core/channels/base.py:298
Integration: Zigbee Home Automation (documentation, issues)
First occurred: 19:45:10 (14 occurrences)
Last logged: 22:32:24

[0x8666:11:0x1000]: ā€˜async_configureā€™ stage failed: not enough values to unpack (expected 3, got 2)
[0xA389:13:0x1000]: Couldnā€™t get list of groups:
[0xAD24:1:0x1000]: Couldnā€™t get list of groups:
[0x8666:11:0x1000]: Couldnā€™t get list of groups:

Unfortunately, yes. There really isnā€™t much else that you can do. You could do it in stages, but you will probably end up with the same results.

So that error that you are seeing is a zigpy error and one that usually indicates a bad database (zigbee.db). Letā€™s get some basic info here. What version of HA are you running? Supervised? Core?

This is a bit more info on the async_configure error: ZHA Does Not Remember New Paired Device Across Core Restarts Ā· Issue #45094 Ā· home-assistant/core Ā· GitHub

So that error that you are seeing is a zigpy error and one that usually indicates a bad database (zigbee.db). Letā€™s get some basic info here. What version of HA are you running? Supervised? Core?

This is a bit more info on the async_configure error: ZHA Does Not Remember New Paired Device Across Core Restarts Ā· Issue #45094 Ā· home-assistant/core Ā· GitHub
[/quote]

I have read though that issue and it appears to be fixed and I am not running Xbee. It does seem the errors are the main cause here and a corrupt Zigbee database would tend to make some sense. I imagine that is a complete rebuild through eh? If we find that is the case I would need to know how to delete the old database.

Core-2021.4.4
Supervisor-2021.04.0
Conbee II running ZHA

I thought about that, which is why I was asking about your version and such. However, even when an issue is closed, it doesnā€™t mean it canā€™t resurface. :wink: The kicker for me was the Tuya USB extenders as Iā€™ve never heard of them, but I do know that both Xiaomi and Tuya both like to use Xbee-ish boards in some of their products, so I wasnā€™t 100% certain that wasnā€™t the issue.

Iā€™m actually hopeful that it is just a corrupt DB. There are some methods to take the zigbee.db file, copy it to your local machine and try to repair it using any Sqlite tool (thereā€™s TONS of them available for nearly every operating system). That could work.

Otherwise, if you have backups of your /config directory (or snapshots), you might be able to pull an older version of your zigbee.db and replace the one you have with an older version when things were working properly.

Worst case, stopping HA, deleting (or renaming without the .db extension) zigbee.db and then restarting HA will take you back to scratch and youā€™ll have to repair everything.

Thanks again :grinning: So start with the repair or the zigbee database do you think? Find a way to copy of to my local machine and replace it back. Is it as simple as that or do I need to start/stop services etc. Sorry for the dumb questions, unlike you I donā€™t write code :slight_smile:

PS where in the world are you if you donā€™t mind me asking?

There is no such thing as a dumb question. Ask away! :slight_smile:

Yeah, Iā€™d start with a DB repair first. This would be a good starting point: Fix corrupted / malformed SQLite database (home-assistant_v2.db) (while it mentions home-assistant-v2.db, zigbee.db is also a Sqlite3 database).

You can do this in a number of ways; SSH, rsync, samba, etc. Whatever you use to get to your .yaml config files, thatā€™s the mechanism that you would use.

Youā€™ll want to shut down HA before fiddling with the database. You can leave it up when you copy the db to your local machine, but youā€™ll want to shut it down before replacing/deleting the zigbee.db file on your HA instance.

North Carolina, US :smiley:

Thanks again. I have a developer that works for me he knows databases backwards so I have engaged him to help a little here. He is going to compare a database from a while back. In the meantime do you think these look corrupt or so you think itā€™s just weird characters. He Fragma integrity test and it ran OK.

Let me check against an old copy that I have. Give me a few.

Which table is that?

1 Like

Its the zigbee.db

Yeah, but what table within zigbee.db? For instance this is the endpoints table. Iā€™ve never seen weird characters like that in any of the tables in the DB.

Itā€™s the attributes table. I can share stuff with you privately if that helps and happy to pay for someoneā€™s time as itā€™s better than grief from the better half :slight_smile:

Ok. The attributes table shouldnā€™t have weird characters in it like thatā€¦ BUTā€¦ to be fair, I donā€™t know what the Tuya/Xiaomi devices throw in there, but your Philips and Gledopo devices shouldnā€™t be throwing foreign characters in there like that.

Iā€™m going to go out on a limb and say that there either was some corruption that happened at some point or currently.

So, if you have the time, what you could do is keep that file backed up on your system. Shutdown HA, kill the zigbee.db file there and then restart HA. Pair a couple of devices (Iā€™d go with devices that are close to you that are easy to pair) and then grab a copy of that generated zigbee.db and examine it in the same way. If there are no wonky characters, then the database is absolutely the culprit and thatā€™s going to really suck, because repairing 150+ devices is just craptastic. :frowning:

No payment needed at all! Iā€™m happy to help as much as I can.

As a reference, this is what Philips devices look like in my attributes table:

Thanks mate. You are restoring my faith in forums :slight_smile:
I am at work at preset so will go home tonight and try. My other thought was to search for one of the mac addresses and see if itā€™s one of the failing devices. If there a way of doing that? Obviously the attributes table in the zigbee.db is not good as it shows weird characters.

1 Like

Yup. If you are using something like DB Browser for SQLite, you can use the following SQL:

SELECT * FROM attributes where ieee = '[insert mac here]'

You can also select against the devices table as well.

Also, you might want to hop on the HA Discord and jump in the #zigbee channel. Thereā€™s tons of Zigbee gods there that might be able to help out further as well.

1 Like

Thanks. We had another look at the zigbee.db. All the weird characters are appearing in the ā€˜default light link groupā€™. None appear to be in the groups that I know are the issue.

I will try the discord forum, just never had any luck there as I am not technical enough!

What I find the weirdest of all here is that although in HA the lights that are issues you cannot individually turn them on or off you can use the zha zigbee group and they work normally!!?

Yeah, thatā€™s another thing that leads me to think db corruption as groups of devices shouldnā€™t work if routing was failing, AFAIK.

I donā€™t usually like pinging people, but talk to @dmulcahey. Heā€™s a literal Zigbee god amongst men. :smiley: (and just an all around great person).

1 Like

Last question then eh! Then you can go and eat your dinner I imagine! What would you do first?

A: Do you think I should try deleting individual values from the attributes table, or should I try deleting all attribute values related to individual MAC addresses (that have weird characters in the value column).

B: Contact Zigbee god man :slight_smile:

Donā€™t edit the tables directly. Bad things will happen. I would start with a clean db on your HA instance, pair a couple of devices, check the newly created DB and see if there are weird characters in there again. That should pinpoint the DB as being the culprit here.

Iā€™d reach out to him (Adminiuga is the other person I would recommend) at the same time as this might be something they have seen before.

Also, @Hedda (also a REALLY good Zigbee person) might chime in as well (hint, hint) :wink:

1 Like

Alright I will have a play tonight and hope one of the gods chimes in. Is it usually OK to go to discord and point to thread here or re-post the whole lot do you think?

Perfectly acceptable. It gives context. :slight_smile: