"zha-toolkit" - Toolkit providing low and high level Zigbee commands through ZHA/zigpy

@le_top Would you consider extending “zha custom” with Zigbee dongle firmware flashing operations?

zigpy-znp have tools for flashing the older CC2530 and CC2531 based dongles but not any newer:

https://github.com/zigpy/zigpy-znp/blob/d37ac29ea7f164e43445c0a87b5b85dd0067de0d/TOOLS.md#flash-operations

JelmerT’s cc2538 cc2538-bsl can however very easily flash CC2652/CC1352/CC2538 via Auto-BSL:

https://github.com/JelmerT/cc2538-bsl (requires ITead_Sonoff_Zigbee-delay patch for Sonoff)

Alternatively the fork llama-bsl for additional board auto-detection features (but no Sonoff patch yet):

https://github.com/electrolama/llama-bsl

Most newer Silicon Labs dongles also support Auto-BSL but need Elelabs Firmware Update Utility:

https://github.com/Elelabs/elelabs-zigbee-ezsp-utility

PS: Not sure if worth flashing CC2530/CC2531 since those older chips are no longer recommended.

FYI, zigpy-cli devs plan on soon migrating to a new unified backup commands for all radio libraries, see:

https://github.com/zigpy/zigpy-cli/pull/2

Initial implementation of cross-adapter network backup/restore

change introduces a network backup/restore utility for any adapter implementing the new radio API

I am willing to add new items to zha_custom if I do not need to spend too much time porting them. Time is my main limitation.

For instance, adding the flashing procedures already provided in Zigpy-* is fairly straightforward.

Adding JemerT’s project is too much work:

  • it’s not provided as a library (no pip install AFAICS);
  • If there is a unintegrated patch, it’s likely not part of the pip install (but I think one can pip install from a repository if it supports pip install);
  • Testing. Firmware upgrade is a critical procedure and takes time to test.

Stopping (disabling) and restaring ZHA could probably be added to the service during the upgrade procedure. Data backup (for safety) and restore (if needed) could also be added to the procedure.

I do not nkow to what extend the Python Scripts Integration gives the scripts access to the serial ports/sockets.
That integration could be a first step to porting to zha_custom by testing the firmware upgrades scripts as they are in the HA environment.

I suppose that zha_custom could probably call another python script natively.

The external scripts can possibly also be fetched using the exposed zip deliveries in the repositories.

Ultimately - if the respective licences allow it - the firmware binary could also be preloaded or fetched from an external source (listed in some file in zha_custom so that an upgrade to “latest” is possible).

But it’s mainly my personal “human” bandwidth problem. I can help out with blocking issues and integration, but not do all the development.

1 Like

I’ll update zha_custom when that is available - I already noticed something was in the making.
Backing up was not my initial reason for extending the original zha_custom ! But I did add a daily coordinator backup to my system.

1 Like

I made it a little bit easier to setup a custom command from the UI.

In UI mode, all commands are available in the drop-down and you can select a ZHA Entity from the second drop down.
For quite a few commands that will do the job.
If you need more than just text for ‘command_data’, you can still select the command and the entity in UI mode and then switch to YAML mode to finish setting up the service call.

You can still enter the IEEE address in Yaml mode if you want to.

1 Like

Any tips on getting/retrieving a list of all active and/or inactive devices known to the Zigbee coordinator?

Nice to get lists with ID, IEEE, LQI, RSSI, Last Seen, power mode, etc. so could example sort by LQI.

Kind of as alternative to zha-device-exporter → https://github.com/dmulcahey/zha-device-exporter

Another use-case scenario could be to export data to Prometheus or other monitoring/alerting tools, e.g.

https://community.home-assistant.io/t/zha-prometheus-export-zigbee-device-informations-to-prometheus/360860

GitHub - dmulcahey/zha-network-card: Custom Lovelace card that displays ZHA network and device information does this for the UI.

What would the benefit have to be versus zha-device-exporter (I have the network card, I did not check the exporter) ?

Ok, this is something that will be useful to those that want to avoiding reading zigbee attributes from the database.

The zigbee attributes you read or write (and read back) can now be written directly to home assistant states or state attributes

For instance, I want to read apparent power from my electrical meter and it’s not available through the standard functionnallity, I can now call this service repeatedly:

service: zha_custom.execute
data:
  command: attr_read
  ieee: sensor.my_smartenergy_metering
  cluster: 0xb04
  attribute: 0x50f
  state_id: sensor.test
  state_attr: apparent_power
  allow_create: True

That reads the attribute and on success updates the attribute. Without “state_attr”, it updates the attribute itself (and you’ll get the updated time as well).

I wonder if I should rename this custom_component and call it zha_toolkit.
What do “you” think?

Also, some functions I have been thinking about (I implemented some):

  • Possibility to trigger events when the service completes, fails or succeeds. That could help setup an automation requiring that the ZHA operation succeeded;
  • Possibilty to designate a state to write the result from (some) ZHA commands to - that could help for reading more than one attribute from a cluster - done for attr_read/attr_write/conf_report - reading multiple attributes not done;
  • Consider that the write operation only succeeds if the readback is equal to the written value (and an option to disable that condition) - done (tested for string type);
  • Maximum retry count - available for conf_report;
  • Will not implement - Timeout (combined with retry) to retry for 2 minutes maximum for instance;
  • Possibly: an id for the operation to avoid running the same command twice at the same time.
1 Like

Hello there, let me first say that I am amazed at the level of expertise in this thread. I am one of the users who has a keen interest in home assistant but is by far not advanced enough to do most of the things explained here. I believe some referred to the user in the beginning of this thread :grin: that will be me or one of me :stuck_out_tongue_winking_eye:

I stumbled upon this thread because I was looking for a solution to go from cc2531 to sonoff v3 dongle without having to redo all my entities and with that automations and with that lovelace dashboard. So I’m hoping that at some point in time there will be a one on one or step by step solution to actually get that done. like a Zigbee backup for dummies (no point intended :face_with_hand_over_mouth::grin:)
I did try the smarthomethings solution but eventually got stuck somehow.

Nevertheless the question was about the naming. With Zigbee being such a widely used and broad standard. I would call it something like “a ZIGnificant toolkit” or maybe even “THE ZIGnificant Toolkit”

My two cents and thank you so much for creating things that less IT apt people can use and enjoy :blush:

:smile_cat: Quite funny. I think you meant “no pun intended”.

I did not entirely create this, but I extended, documented and made it easier and more flexible from the UI.

Regarding the name, I think it should not be too long. Having ‘zha’ in the name reminds that it’s to be used with ‘zha’ and not zigbee2mqtt. ‘zig_toolkit’ could be ok though, and to continu the play of works, “MAziGNIFICANT Toolkit” could be fun too.

There’s already a guide above how to backup and restore, but possibly not easy enough.

1 Like

Hi,

Just installed true HACS and added the zha_custom: to configuration.yaml, made a automation from the blueprint but after running i don’t find a nwk_backup.json anywhere in my /config folder?

it should be here

/config/custom_components/zha_custom/local/nwk_backup.json

If I can do it… :rofl: :joy:

Ok found it. Was searching in the /local (/config/www/) folder. :slight_smile:

1 Like

So I too did the back up and found the backup file as advertised. I am missing a few entities and i’m not sure which ones. But ok a few is less than all 31 of them.

What’s the next step?

Remove the ZHA integration? remove the cc2531, restart, add sonoff v3 dongle and restore?

Here is the procedure near the end of feedback by @Abhi and here is the procedure I added in the README of the component.

thanks I’m going to try this now :crossed_fingers:

EDIT:

Did step 1. Made the backup.
Step 2 removed the CC2531 USB coordinator.
Step 3 inserted sonoff in same socket.
Step 4 rebooted HA (so platform reboot iow the PI that runs my instance)

After step 4 HA came back online but zha.custom did not load. so a restore was not possible. did I read a step wrong?

The previous ZHA integration was saying was not loaded. I removed it and reinstalled ZHA. Then sonoff became visible and also saw most if not all of my devices. However that did not help cause all references in Lovelace as well as automations were not functional anymore.

For now I placed back the cc2531 and did a complete restore of HA. Hope that will set everything back.

Meanwhile I’ll wait for further suggestions here :wink:

Maybe your Sonoff key is not on the same USB port, so you need to modify the port for ZHA.
I also suggest to enable logging for ZHA (and zha_custom).

I have this in my configuration:

configuration.yaml:logger: !include logger.yaml

and then in logger.yaml you can set what you want to log for.

  custom_components.zha_toolkit : debug
  custom_components.zha_custom : debug
  homeassistant.components.zha: debug
  #bellows.zigbee.application: debug
  #bellows.ezsp: debug
  #custom_components.zha: debug
  zigpy: debug
  #zigpy_deconz.zigbee.application: debug
  #zigpy_deconz.api: debug
  ##zigpy_xbee.zigbee.application: debug
  ##zigpy_xbee.api: debug
  #zigpy_zigate: debug
  zigpy_znp: debug
  #zhaquirks: debug
  ## Extra, not sure this works:
  zigpy.zdo: debug
  zigpy.device: debug
  zigpy_znp.api: debug
  zigpy_znp.zigbee.zdo_converters: debug
  zigpy_znp.zigbee.application: debug
  zigpy_znp.commands.application: debug
  zigpy_znp.commands.zdo: debug
  zigpy_znp.commands.znp: debug
  zigpy_znp.logger: debug
  zigpy.util: debug

That will add extra output to home_assistant.log .

Thank you for your help.

Sad to report that it didn’t work at my end. Not sure why. The one usb cc2531 is indeed try/ACM0 and the sonoff get tty/usb while they are both in the same socket.

I did manage to get zha_custom to keep running while exchanging the dongles but then on doing a restore it give me an error message.

I’ll have to postpone for now unless there will be a clear cut seamless exchange possibility. Although I realize that is was already quite clear cut :thinking::wink:

Device paths are not “hard” linked to the physical port, but rather to the type of device and the port type linked to the driver, and the order in which the devices are detected.

Probably somebody with more technical savvy will find a way to do it.

1 Like