Zigbee2MQTT how to get Connector/Router a device is connected to?

Stiltjack · November 13, 2023, 9:21pm

You might try setting up entities cards with the LQI of all your devices - you should be able to display them on a phone.

The numbers should be taken with a pinch of salt as different manufacturers calculate them differently, and in any case they change from minute to minute, but over time you should be able to identify devices where the quality of the link is consistently poor.

Edit: In the example above the bathroom light has a very poor link, but normally it is quite good. The lights which are consistently poor are the kitchen ceiling lights - I think because they have metal shades which block the signal. (These devices are all routers, by the way.)

FrankRy · November 13, 2023, 9:39pm

hi @Stiltjack ,

i still have

now it would just be nice if you could see where they are connected?

what im a little bit concerned. firmware of the newly flashed routers is shown in zigbeetmqtt in hassio as still the old one (20220125) ? but i have flashed definitely the new one (20221102)?
connector show new fw, as flashed?

UG
PS C:\InstallTemp\Dev\SonoffZigbeeUSB3.0Dongle Stick\phyton\cc2538-bsl-master> python cc2538-bsl.py -p COM3 -evw --bootloader-sonoff-usb CC1352P2_CC2652P_launchpad_router_20221102.hex
sonoff
Opening port COM3, baud 500000
Reading data from CC1352P2_CC2652P_launchpad_router_20221102.hex
Your firmware looks like an Intel Hex file
Connecting to target...
CC1350 PG2.0 (7x7mm): 352KB Flash, 20KB SRAM, CCFG.BL_CONFIG at 0x00057FD8
Primary IEEE Address: 00:12:4B:00:26:B6:A0:20
    Performing mass erase
Erasing all main bank flash sectors
    Erase done
Writing 360448 bytes starting at address 0x00000000
Write 104 bytes at 0x00057F988
    Write done
Verifying by comparing CRC32 calculations.
    Verified (match: 0x9877ee56)


EG
PS C:\InstallTemp\Dev\SonoffZigbeeUSB3.0Dongle Stick\phyton\cc2538-bsl-master> python cc2538-bsl.py -p COM3 -evw --bootloader-sonoff-usb CC1352P2_CC2652P_launchpad_router_20221102.hex
sonoff
Opening port COM3, baud 500000
Reading data from CC1352P2_CC2652P_launchpad_router_20221102.hex
Your firmware looks like an Intel Hex file
Connecting to target...
CC1350 PG2.0 (7x7mm): 352KB Flash, 20KB SRAM, CCFG.BL_CONFIG at 0x00057FD8
Primary IEEE Address: 00:12:4B:00:2A:1A:FE:52
    Performing mass erase
Erasing all main bank flash sectors
    Erase done
Writing 360448 bytes starting at address 0x00000000
Write 104 bytes at 0x00057F988
    Write done
Verifying by comparing CRC32 calculations.
    Verified (match: 0x9877ee56)

COORDINATOR

PS C:\InstallTemp\Dev\SonoffZigbeeUSB3.0Dongle Stick\phyton\cc2538-bsl-master> python cc2538-bsl.py -p COM3 -evw --bootloader-sonoff-usb CC1352P2_CC2652P_launchpad_coordinator_20221226.hex
sonoff
Opening port COM3, baud 500000
Reading data from CC1352P2_CC2652P_launchpad_coordinator_20221226.hex
Your firmware looks like an Intel Hex file
Connecting to target...
CC1350 PG2.0 (7x7mm): 352KB Flash, 20KB SRAM, CCFG.BL_CONFIG at 0x00057FD8
Primary IEEE Address: 00:12:4B:00:26:B6:8E:AC
    Performing mass erase
Erasing all main bank flash sectors
    Erase done
Writing 360448 bytes starting at address 0x00000000
Write 104 bytes at 0x00057F980
    Write done
Verifying by comparing CRC32 calculations.
    Verified (match: 0xa9dc145d)
PS C:\InstallTemp\Dev\SonoffZigbeeUSB3.0Dongle Stick\phyton\cc2538-bsl-master>

FrankRy · November 13, 2023, 10:21pm

linkquality is much more wilder than before flashing. seems there is a lot of reorganizing?

map show a lot of lost devices, but devise overview does not?

i´ll give it a try till tomorrow…

Stiltjack · November 13, 2023, 11:57pm

I don’t think they’re “lost”. Battery-powered end devices spend most of their time sleeping and only check in with a parent router periodically. At the time of your snapshot, that’s what they were doing. Wave at a motion sensor and generate another map, and it will be connected.

khvej8 · November 14, 2023, 4:35pm

Did the devices come back over time?

With this huge number of end-devices compared to routers, have you considered implementing more routers? They do not need to be additional device just for routing. Like plugs or bulbs being part of automating your home. I think this will make it more stable in the long run.

FrankRy · November 14, 2023, 10:38pm

except of 5, they all came “back”.
5 of the devices i had to reconnect manually. (one is left, its the most worse accessable device i have, the rainsensor dyi made from here rain gauge)

but i recognized a very strange behavior of zigbee2mqtt MAP, what did not get better until yesterday… (the map is rebuilt badly every time)

many of the devices are “linked” to coord or routers (dashed line or solid lines), but some of them, do not “glide” and “reorientate” on the map automatically as usual. i recognized all of the “glued to the map”, do show a “0” on the connections lines, what means, linkquality = 0? But device page shows me real link qualities for all of them, and hassio recorder history stat do show me changing lqi stats regularly, but NOT “0”. so only the map in z2m does not work correct any longer?
i dont know if it´s due to the last update of z2m, or of the newly added routers (2 more since 2 weeks), or the 10 new devices since approx 1 week… or the fw-flashing of the 3 existing routers … if i move one of the router-bubbles all of the other bubbles do wobble around and glide to new positions, except the red marked… those are frozen…

what i also had recognized (in the past and todays much more): many of the devices do have an unstable linkquality (even i had just 40 devices). it seems only the ones connected to the coordinator are “stable” (see history graph “flatlines”) the others do vary very much… 80 to 7 lqi. especially the raingauge sensor had lost connection since the beginning, what decided me to reposition the raingauge first, but without any success, i added two more routers… but it does not became better… rain sensor regularyl lost connection (could be long term 50-80, then off instantly, could be good weather or raining…) i have an addittional temp (weather) sensor of aqara at the same spot, without any issue since ever? anoying… i cant access rain gauge, when its raining ort had rained…

i plan to use more routers, but honestly i dont now where to place and how to manage power source for them ( i assmue it would not make sence to place two at the same place?)
but i wonder: one router should handle 50 devices as described here?

i have 5 routes + coord? i have “only” 75 devices? shouldn´t be enough? all devices are in range about 10 meters in the house, 3 floor-levels… brick walls

By model
lumi.weather: 56
lumi.airrtc.agl001: 12
lumi.sensor_magnet.aq2: 8
ti.router: 5

another thought: except of the coordinator, all 5 routers are connected to a plugin power usbsource directly to a wall socket. i plan to make extension cables for all of them. Maybe proximity to the electrical wiring does have any influcence?

but what bothers me a lot, the last 3 routers i had flashedrecently, do NOT show the firmware version in hasso i have flashed? could it be data in hassio (z2m) is outdated, and not beeing updated? i have rebooted, and i had shut off the system to unplug coord from rasp… and since then restartet z2m integration twice?

k8gg · November 14, 2023, 11:49pm

Zigbee power plugs might be a good option / good candidate.

Stiltjack · November 14, 2023, 11:52pm

You don’t have enough routers. If you have 75 devices, I’d suggest that as many as half of them need to be routers.

Your LQI numbers are very low. They should be in the 100-150 range at least, ideally higher. To achieve this, you need more routers - lightbulbs, smart plugs, wall sockets, whatever.

The optimum distance between devices is 10 feet, not 10 meters. Yes, the signal will travel much further in good conditions, but that’s not the point. The point is to build a robust mesh that will self-heal when there are problems. You need more routers.

Stop obsessing about the map. It’s a snapshot visualisation and it will be different every time you generate it.

YOU. NEED. MORE. ROUTERS.

Edit: Have you read this?

jerrm · November 15, 2023, 12:54am

He may need more, but a 50% is a crazy high ratio. I have 13 for a net of 90 devices total, and only 7 actually have end devices attached. Similar ratios at other installs, all solid.

Five could be fine for a 1500sf ranch, probably not for a 4500sf 3 story.

Were these upgraded from a previous router fw, or newly flashed? I’ve seen z2m not populate that correctly when updating router fw. Force remove, restart z2m, re-pair fixed it for me. Not sure if all that was really needed.

Hedda · November 16, 2023, 8:12am

The requirement will be unique to each enviroment and specific devices uses. Some buildings have small rooms with dense wall/cealings/floor that kill Zigbee as its signals are low-power and the messages are very short (small frames). Then some Zigbee devices will have much worse antennas and antenna design than others, as well as have weaker Zigbee radios than others. So two similar setups can get very different results depending on which home they are installed and where exactly the devices are located.

But a the general rule can still be said to be; when in doubt, add more Zigbee Router devices. However, buying and adding a few great Zigbee Router devices will have a much greater impact compared to many poor Zigbee Router devices, so that is therefor what I recommend.

Again, strongly recommend follow the tips I collected here → Zigbee networks: how to guide for avoiding interference and optimize for getting better range + coverage

FrankRy · November 16, 2023, 9:27pm

thank you guys for all the great input! i have realized that i have to upgrade my zigbeemesh anyhow…

i was always fixated on sonoff zigbee3.0 usb dongles as routers the whole time, but not on other powered zigbee devices. Only now i do understand some posts before (about powerplugs and bulbs) what i had missunderstood a long time…

to clear up a misunderstanding: ALL my devices (routers and devices) are within a range of 10-15m in the house. So distances between devices is mostly about 2-4 meters… but distance between routers might be more. I was of the opinion that the dongles distributed evenly around the house with their external antenna would be sufficient, but this is probably not the case.

i will try to upgrate my mesh and report the results in the next weeks…

FrankRy · November 16, 2023, 10:47pm

@jerrm
yes upgraded from previous fw…

so i forced remove, restartet z2m, but it does NOT re-pair anymore! now i have just 4 routers .

any chance to get this router back to work in hassio without deinstalling z2m?

jerrm · November 17, 2023, 1:59am

Re-flash it.

Maybe force remove was wrong. Its been a while.

francisp · November 17, 2023, 4:59am

force remove removes it from the database, but not from the network, that is the problem.

FrankRy · November 17, 2023, 10:10pm

hi @francisp ,

aha network? what does it mean? where is the network information stored? in the router hardware or in zig2m integration files, or in hassio db?

before i reflash anything, any chance to remove from network?
edit files, or restore zig2m from backup?

francisp · November 18, 2023, 4:53am

from the documentation

Note that in Zigbee the coordinator can only request a device to remove itself from the network. Which means that in case a device refuses to respond to this request it is not removed from the network.

and

In case all of the above fails, you can force remove a device. Note that a force remove will only remove the device from the database. Until this device is factory reset, it will still hold the network encryption key and thus is still able to communicate over the network!

So maybe restore a backup

FrankRy · November 18, 2023, 10:14pm

luckily i got back my removed router, but the way to get it back was not as as i expected.

partly restoring a backup zigbe2mqtt add-on alone, caused severe problems. mosquito broker dont wanted to start anymore

2023-11-18 11:35:02: Warning: Mosquitto should not be run as root/administrator.
mosquitto: persist_read.c:550: persist__restore_sub: Assertion `client_id' failed.

so it is nessessary to restore both zigbe2m and mosquito add-in. at same time, to get a “running” system…, but the missing router was not in the device list?

in the end i moved back to todays backup and manually copied the following files from backup… from/to folder

homeassistant\zigbee2mqtt\

configuration.yaml
coordinator_backup.json
database.db
state.json

now the missing router is back…

ffries · November 18, 2023, 10:29pm

I realize disconnected objects spend most of their time sleeping and they only reconnect from time to time. For example motion detectors and/or smoke detectors. While they are asleep, the coordinator cannot contact them and there is no route to them. But if you enter a room or rotate the smoke detector, it works right away and reconnects.

When it comes to routers, they should always be available.

So clearly Zigbee2mqtt does not make a difference between them.

In the above example, only the ZBMINIZ2 is faulty. Other devices work very well.

Now for good coverage, I purchased 5 UZG Zigbee coordinators with Ethernet PoE/Wifi. I will probably use 3 indoors and 2 outdoors (with custom casing).

With latest beta firmware, you can monitor the coordinator from HA. Very cool.

Also, when migrating to a new coordinator, it is better IMHO to repair devices.

FrankRy · November 18, 2023, 10:53pm

hi @francisp
thanx for your explanation. but i have a lot more questions, just so i understand: i wonder how to do correctly remove an device via z2m gui?

=> at device, use remove button, but NOT set force-remove, right?
=> should this request the coordinator to remove the device, as you descriped, correct?
how do is see if its removed succesfully? will it be removed from DB as well in this case? if the coordinator fails to remove it from network, “nothing” will happen? (so coordinator keeps it in network and it will stay in hassio/z2m db)?

so force_remove is only for devices to get rid off and not to be used anymore, correct? i wonder, what remove option “block from joining again” does when it is much easier the other way? will this set an extra flag?

another question: by re-pairing the same device with an new NetworkEnryptionKey (e.g. after reflashing it), will this create new entitiy_ids in hassio for the device too, when it was removed before? Or will all kept the same?
i wonder how the z2m “finds” the same router, after flashing, when not beeing removed before? it should have a new NetEncryptKey too? or is this irrelavent for identifaction of a specific device?

with my sonoff USB3 dongle, i would not have the problem to reflash, to get a new nework encryption key and possibly new entity_ids at the end… this special device do not have any…
but if i would have e.g. an powerplug or bulb used as an router, having a lot of entities_ids beeing used in automations etc. then it would be a prob.

so at the end i wonder, how to get z2m populate a device correctly after beeing fw updated?

FrankRy · November 18, 2023, 11:01pm

@ffries interesting! but i do not unserstand the part with Ethernet PoE/Wifi on a zigbee coordinator? you mean you will split the same mesh into 5 pieces, or you will have 5 independent zigbee meshes? is it possible to use more than one coordinator in zigbee2mqtt?