Migrating from a Philips Hue Bridge to the ZHA (Zigbee Home Automation) integration

[Stiltjack]

Don’t forget that the first post in a community guide is a wiki - feel free to add to it or correct it.

This guide was written and tested these versions but should also work similarly with later versions:

Home Assistant 2023.8.4
Supervisor 2023.09.2
Operating System 10.5
Frontend 20230911.0 - latest

Background

Until recently Philips Hue devices have been controlled locally via a local API through the Philips Hue Bridge using the Philips Hue integration for Home Assistant, but Philips/Signify as a company has now started required users to open a cloud-based Philips Hue account to use your Philips Hue Bridge via their Philips Hue app, which will or at least can ultimately mean that Philips could hold details of all their data activity, (at the time of writing it only contains basic information about the bridge itself).

Philips’s rationale is that this will allow them as a company to enable their users to control the Philips Hue Bridge remotely and securely. This argument falls flat for most Home Assistant users since we already could use our Philips Hue Bridge securely and locally even without the internet, and for many in the Home Assistant community there are obvious privacy concerns with this change in Philips’s policy. There are a few blog posts about it, like for example here and here.

This change of user terms and conditions mid-lifecycle of the Philips Hue Brigde as a product that affects existing owners has prompted some to bite the bullet and fully move/migrate all their Philips Hue products (and other Zigbee devices) to the native ZHA (Zigbee Home Automation) integration.

The ZHA integration is a hardware-independent local Zigbee gateway solution that is built-into Home Assistant which supports most Zigbee devices regardless of manufacturer and brand, so it allows you to consolidate all your Zigbee devices on a single Zigbee network and enable you to get rid of all your proprietary Zigbee gateway/hubs/bridges.

Requirements

This post assumes that:

• You have been using a Philips Hue Bridge V2 (the square one). If you have one of the round V1 Philips Hue Bridges (still supported by Home Assistant’s Philips Hue integration, but no longer supported by Philips) none of this is necessary as they no longer have internet access. It is however still worth considering moving to the ZHA integration as Philips Hue Bridge v1 is no longer receiving any updates.
• Your lights have been set up using the official Hue app from Philips.
• You are currently using the Philips Hue integration in HA
• You have purchased a Zigbee Coordinator USB adapter recommended for ZHA integration.
  • Home Assistant SkyConnect USB dongle is the “official” device from Nabu Casa and is highly recommended since not only do you get a plug-and-play configuration but also easy to update its firmware if using a Home Assistant Operating System installation (and as a bonus the profit from it goes to pay developers working on improving Home Assistant).

Limitations

This is a summary of the migration process, which is straightforward but may be long-winded as you have to move devices one by one (allow a whole evening if you have a lot of lights). You can run Philips Hue Bridge with the Hue integration and the ZHA integration with a Zigbee Coordinator USB adapter side-by-side if you want to take a more leisurely test-as-you-go approach - but a Zigbee device can’t be joined to both Zigbee networks at the same time, (Zigbee devices can only be paired with a single Zigbee gateway/bridge/hub/controller).

What you be aware of before getting started

Zigbee is a low power radio protocol which relies heavily on its mesh networking technology to automatically workaround and overcome the limitations of the weak radios in each device. In hardware terms, it usually consists of a Zigbee Coordinator USB radio adapter plugged directly into the machine running Home Assistant, and a mixture of other devices, (such as example lightbulbs, motion sensors and buttons), where most of your mains-powered products act as “Zigbee Router” devices that will work as signal-repeaters/extenders, while your batter-powered devices are “Zigbee End Devices” and as such do not forward on any communication-messages to other devices.

Important here is to be aware that Zigbee messages and commands are relayed back and forth to Zigbee Coordinator USB radio adapter and the communication is indirect through a chain of intermediate devices (e.i. your Zigbee Router devices). This makes a network mesh with many Zigbee Router devices very resilient since there are many possible routes for the communication messages to take, and although the signal from each radio is weak the communication messages can find their way round walls and other obstructions as long as you have enough Zigbee Router devices. Zigbee communication messages can also cover large areas and longer distances, provided there are enough Zigbee Router devices that can distribute the traffic along the way. If you live in a modern building with thin walls then this is not something you probably do not have to worry about, but if you are living in a home with thick or dense building materials then you have to remember to add enough Zigbee Router devices at not to far distances so that all communication-messages can go through without problems.

If you have Philips Hue lights then yo are already using a Zigbee network, so there’s no reason why a Zigbee Coordinator USB adapter dongle shouldn’t work just as well (although the antenna in the Hue hub is probably better than the one in your new Zigbee Coordinator USB adapter dongle, so some tweaking may be necessary); on the other hand, if you’re already having connection problems with Hue, you’ll probably get them with your new Zigbee Coordinator USB adapter dongle too.

Most problems are caused by EMF (electromagnetic field) interference from other electrical equipment that is placed too close to Zigbee products - you need to look into this before you tackle the move. There’s a post on the subject here. If your Hue network is problem-free it may be a good idea to position the Zigbee Coordinator USB adapter dongle on the end of a long USB extension cable away from the computer running Home Assistant and any other electric appliances/peripherals/cables, (but preferably close to where you kept your old Philips Hue Bridge if you can).

To forward communication messages and relay commands, Zigbee Router devices have to be mains-powered, connected and always on. In the Hue ecosystem lightbulbs do the heavy lifting; battery-powered devices like buttons and motion sensors can only come at the end of the chain. In the image below they are the round outlying nodes connected by a single route. (The ovals are mains-powered devices relaying signals and the co-ordinator is the rectangle in the centre.)

Setup

Note! There are alternative instructions for setting up Home Assistant SkyConnect USB dongle here, and the ZHA ingeration documentation has more details on available configuration options here.

• Using the included or other USB extension cable, connect the Zigbee radio USB dongle to a USB port on your Home Assistant host machine.
• It should be discovered automatically, and you will be invited to configure the ZHA integration.
  • Click “configure” and follow the instructions to set up the co-ordinator. To begin with this will be the only device on the network and you will return to it to set up each additional bulb, sensor or button.
    • If you are using another Zigbee Coordinator USB adapter on the ZHA list of recommended Zigbee adapters but is not not the list of USB discoverable adapters then you will have click on “ADD INTEGRATION” to search for and install the “Zigbee Home Automation” integration then configure the settings for that adapter (choosing the right radio type, etc. for what you purchased instead).

Adding devices (paring/joining a device)

Adding devices to the network is very straightforward. For each device in turn:

• Put it into “pairing mode”
• Wait for it to be discovered by the new Zigbee network
• Adjust its new ZHA configuration to match the old Hue one so that automations etc. keep running

To put a device into pairing mode, simply delete it in the Philips Hue app - there should be a delete button on the device setup page. This will also remove it from the Philips Hue integration in HA. (If for some reason this doesn’t work other ways of switching on pairing mode are discussed below.)

In HA, open the ZHA integration (Settings | Devices & Services | Integrations), then the co-ordinator and click on Add devices via this device.

After 30 seconds or so it will be discovered by HA and added to the ZHA card.

A good strategy is to move from room to room, working outwards from the Zigbee Coordinator USB radio adapter, keeping a balance of mains- and battery-powered devices on both the old and the new network.

So try to move/migration by first first migrating most of your mains-powered devices that you know act as “Zigbee Router” devices, and beginning with the ones that are located closest to the Zigbee Coordinator USB radio adapter and go on from there. That way you will build those devices as a backbone in your new Zigbee network mesh. Preferably only after migrating all or most of your mains-powered devices should you migrate any battery-powered devices or other devices that you know are Zigbee End Devices. FYI, it is well known within the community that Sengled lightbulbs and all non-neatutral/2-wire dimmers/switches, as well as most Zemismart switches are not Zigbee Router devices.

It is however not a good idea to leave all your battery-powered devices until the very last. Once you have deleted all the lightbulbs from the Hue app, any left over movement sensors and buttons will show as “Not accessible” if there is at least not some main-powered devices left in range because the network supporting them has been removed, and you will have to put them in pairing mode using another method.

Tip: If you have problems pairing a device, do not move it closer to the Zigbee Coordinator USB radio adapter. Even though that approach is recommended by some manufacturers, the general best practice if that Zigbee devices should be paired in the location where it’s going to be used.

Other ways to put devices into pairing mode

Most battery powered Hue devices have a pinhole reset button. For motion sensors this is on the back, at the top.

For other devices it is inside the battery compartment. The exception is the outdoor motion sensor, which has to be waterproof. This has a press button on the back (you have to take it off the wall to access it).

In each case, hold down the button for 10 seconds, until the LED turns green then starts to flash orange and green.

For bulbs, you need a Hue dimmer switch. Turn the light on, then hold the switch as close as you can to it. Press and hold the top and bottom buttons on the switch together for about 10 seconds or until the bulb flashes. It’s now in pairing mode.

Naming

Once you have moved a device to the SkyConnect network, you can give it a friendly name by clicking on the edit icon, top right on the device screen.

If you don’t do this all your devices will be named after the Hue model number and it will be hard to tell them apart. When you click “save” a dialogue box will pop up asking if you want to rename the device’s entities too. Unless you have a good reason to do this, click “no”.

If you have already been using your devices via the Hue bridge and the Philips Hue integration, the associated entities will probably be included in a lot of automations and scripts. Take the time now to edit the new entity IDs to make them the same as the old ones - that way your automations should carry on working as before.

If you were unable to delete a device in the Hue app you may find that the old entities still exist in HA, and you are unable to rename the new ones. To deal with this you can rename the old entities temporarily - to something like sensor.bathroom_dimmer_switch_battery_old for example. Old entities, including renamed ones, will be removed when you finally delete Philips Hue integration.

If you have been using device IDs in your automations… Sorry, you’re out of luck - you’re going to have to edit every one to put in the new device ID.

You can now delete the Philips Hue integration in HA (any left over entities will be deleted too) and unplug the Hue bridge.

Groups

Devices in Home Assistant can be grouped together using a helper, but they are still controlled as separate entities - if you watch carefully grouped lights come on one at a time. If you wish you can group zigbee devices so that they are controlled as a single entity.

Open the co-ordinator and click on Add devices via this device as described above, then click on the Groups tab at the top of the page.

Click on Create group, give the group a name, select the devices you want to include and click on Create group at the bottom of the screen.

The new group will be added as an entity on the Zigbee co-ordinator page, where you can edit name, icon and entity id as required. Notice that a new group will not be created unless it contains at least two devices.

A warning about Zigbee groups

Group commands are network-wide broadcasts that are bounced back and forth between all devices to make sure every possible group member has a chance to “hear” the message. It is not a good idea to issue several in quick succession - for example in an automation - as this may swamp the network. Better to address the devices individually, or use HA groups.

Binding

You can bind a dimmer switch or button to a light or group of lights so that commands pass directly between the two. This means that the button or switch will continue functioning even when Home Assistant is down. (Note: the method below has only been tested with the RWL022 dimmer switch - earlier models may not work.)

Open the dimmer switch page in Devices & Services | Devices, click on the three dots next to Reconfigure and select Manage zigbee device from the dropdown list.

Click on the Bindings tab.

To bind with an individual light select one from the Bindable devices dropdown. To bind with a group, select one from the Bindable groups dropdown and tick the kind of binding you want (On/Off at least, but if you’re not sure, you can tick them all).

Important: before you click Bind press the buttons on the dimmer switch a few times to wake it up.

The switch will now control the light, even without Home Assistant.

Updating SkyConnect firmware

The simplest way to do this is with the Silicon Labs Flasher add-on, which defaults to the latest firmware for SkyConnect.

• Do a full backup of HA (just in case)
• Download the flasher from the add-on store and install it
• In Devices & Services | Integrations, disable ZHA
• Make sure the SkyConnect option is selected in the flasher configuration page
• Click Start on the Info page.

The process should take 30 seconds or so (you can follow progress on the log page) and the flasher will stop when it has finished. You can then enable ZHA again.

SML001 and SML002 motion sensors

There is a known issue with older Hue motion sensors - they disconnect repeatedly and can be hard to pair with the network again. This is because they are trying to reconnect as new devices, which the network will not allow. There is a simple fix for this problem here.

Oddities

Entities derived from the ZHA integration will not be exactly the same as the old Hue ones. Here are a few things to watch out for:

• When you pair a Hue motion sensor via ZHA you will see an additional binary sensor entity “Occupancy”. This is the one that you should use in automations. The entity “Motion” doesn’t seem to do anything and can be disabled.

• Using the Philips Hue integration, there were switch entities which allowed you to turn off motion detection. These don’t seem to exist in ZHA.

---

Other Zigbee guides

• Zigbee networks: how to avoid interference and get the best range and coverage
• A to Zig - a Zigbee glossary
• Zigbee buyer’s guide
• Not many people know that - a random collection of Zigbee trivia
• How to set up custom device handler quirks in ZHA
• Is Zigbee right for you?

[poudenes]

Great great post!!! This will help lot of people (like me) to make the step.
Maybe add some information about groups/zones? I know that you can create groups or zones on the ZHA Co-ordinator so selected lights become 1. This will be the same behaviour as the HUE zones or rooms. For me an important aspect because I use lot of zones/rooms in my automations.

Then this integrations gives your the standard scenes of HUE:

GitHub - Hypfer/hass-scene_presets: Hue-like scene presets for lights in home...

Work great as well.

[Hedda]

@poudenes

Maybe add some information about groups/zones? I know that you can create groups or zones on the ZHA Co-ordinator so selected lights become 1. This will be the same behaviour as the HUE zones or rooms. For me an important aspect because I use lot of zones/rooms in my automations.

Note to use Zigbee groups inside ZHA/Z2M (and not Home Assistant’s Group integration), as well important to note that terminology in Philips Hue app is not same as Home Assistant’s terminology so probably need to explain the different meaning and use cases for Zigbee Groups (in ZHA/Z2M), verses Group, Room, Area, and Zone in Home Assistant.

@poudenes

Then this integrations gives your the standard scenes of HUE:

GitHub - Hypfer/hass-scene_presets: Hue-like scene presets for lights in home assistant

Nice! New ZHA/Z2M users ask how to replicate Philips Hue’s “Dynamic Scenes” / “Custom Scenes”.

We have Scenes and the Scenes Editor in Home Assistant, but I think the most notable user experience difference is that Philips Hue app comes with many default preset scenes and it is very easy to add custom scenes as well as lighting transitions when switching between scenes, so many users who are new to ZHA/Z2M therefor keep asking if there is a comparable native built-in integration with default scene presets and lighting transitions or other simple way to replicate/mimic Philips Hue’s “Dynamic Scenes” and “Custom Scenes”.

WTH, why is not possible to use Hue Scenes templates in ZHA

Why is it not possible to transfer the Hue scenes of the Hue Bridge at least to some extent to automatically? It would at least make it easier for me to switch to ZHA with Hue without a bridge, because when you’re a family man, time is a precious commodity.

Migration from Hue to ZHA

Has anyone migrated from Hue to ZHA? Is there any way to migrate existing Hue scenes to HA? I have couple non Hue devices that I’d love to add to my network but I’m a bit worried about the effort to reconfigure everything with ZHA. Also is there a straightforward way to get the switches to rotate between scenes just like the original switches do?

Migrate from Philips Hue and having scenes

Hi guys, Merry Christmas and happy holidays all! Been doing the HA thing for 2 yrs now and loving it. In the past, I’ve tried to go whole hog with HA and not depend on 3rd party things as much as possible. I tried bringing the Hue lights to Z2M from Hue but when I did, when I asked my Google Home mini to turn on ‘bright’, it ended up turning all rooms with the bright scene. How can I stop this and only set the scene in the current room (instead of telling the GH what room to do it in, even tho…

Hue scenes mimicked in Ha (and on Buttons, was Tiles)

HI, Just thought Id share how I created Off scenes in Hue Api and use these in HA: [15] which is just a fun way of representing [22] had this for a long time already: group: hue_light_control: name: Hue Light Control icon: mdi:spotlight control: hidden entities: - input_select.hue_scenes - input_select.hue_groups - script.set_hue_scene with script: set_hue_scene: alias: Set Hue scene sequence: service: hue.hue_activate_scene data_t…

Hue bridge died - options?

So today I realized that all my Hue lights (50+ devices) were not working. After some basic troubleshooting, it seems the bridge is dead. I’ve contacted Philips Hue support via Twitter to see if they have any sensible suggestions for resolving this but I have a feeling the hub is dead. As I have 50+ devices, I’m not looking forward to having to manually reset each and every device and then re-adding them to my system but I’m starting to come to terms with the fact that is what I’ll have to do. …

Scenes and Groups

Hi Everyone, I’m not sure if I’m doing this wrong – I checked the help for ‘Entities’ and ‘Scene’ to make sure I’m doing this right but it appears it’s not working (no errors, even if formatting looks wonky here); here’s my config: configuration.yaml scene: - name: Cool White entities: light.living_room_east_lamp: on light.living_room_west_lamp: state: on rgb_color: [255,207,120] brightness: 255 color_temp: 153 xy_color: [0.4575,0.4099] - name: Relax …

Philips Hue will force users to upload their data to Hue cloud

Today’s story is about Philips Hue by Signify. They will soon start forcing accounts on all users and upload user data to their cloud. For now, Signify says you’ll still be able to control your Hue lights locally as you’re currently used to, but we don’t know if this may change in the future. The privacy policy allows them to store the data and share it with partners. Last week I wrote how toying with the first Philips Hue hub and their local API led me to start Home Assistant back in 2013. Th…

[Mariusthvdb (Marius)]

@Stiltjack

At the time of writing the new Hue contact sensors have not yet been included in the Philips Hue integration. They are available in the ZHA integration. Notice that they behave like buttons - they do not have a state, but instead an event is fired each time the window/door is opened or closed.

Nice post, wont comment on the pros/cons, just a very nice introduction too regardless the Hue aspects of it

on the contacts sensors, they actually are in the current (beta) releases and will be released officially on coming Wednesday:

Note they do have the binary for open/close so you can automate using that using the Hue integration.

just figured you might be interested to know.

cheers!

[Stiltjack]

@Mariusthvdb

Note they do have the binary for open/close so you can automate using that using the Hue integration.

Thanks - guide updated.

[Hedda]

Also need to mention how-to Zigbee OTA (Over-The-Air) firmware upgrade Philips Hue devices.

ZHA integration can perform updates for some Zigbee products out-of-the-box because the company who manufacturer/sell those already publish some or most of their unencrypted Zigbee OTA firmware images publicly, unfortunately, Philips/Signify is not one of those companies. But perhaps the userbase can someday convince them to do that, more information about that here → OTA Information for Manufacturers · zigpy/zigpy Wiki · GitHub

Fortunately, if you still want to upgrade firmware on your Philips Hue Zigbee devices then manual OTA firmware upgrade is at least still possible via the ZHA integration if you can source compatible Philips Hue Zigbee OTAU image files from elsewhere, see → OTA Device Firmware Updates · zigpy/zigpy Wiki · GitHub

There are well as scripted workarounds if you absolutely want to always upgrade firmware, see for example zha-toolkit → zha-toolkit/README.md at main · mdeweerd/zha-toolkit · GitHub (which can enable automated download from unofficial third-party sources like for example https://github.com/Koenkk/zigbee-OTA)

There is a more deatiled dicussion about why out-of-the-box Philips Hue Zigbee OTA firmware upgrading is not yet featured ia the ZHA integration in this thread → ZHA and Philips Hue Firmware

In summery;

You do not need Philips Hue Bridge to upgrade firmware on Philips Hue Zigbee devices via OTA update (Zigbee Over-The-Air updates).

ZHA does support the same standard Zigbee OTA that Philips Hue support, the issue is that you need the official Zigbee OTA firmware image from Philips/Signify for each Philips Hue device that you want to upgrade, and there is the challenge as Philips/Signify as a company still does not publish such official Zigbee OTA firmware images via a public repository or in any other way make them available for third-parties to download.

So the problem is that Philips/Signify does not yet share their Zigbee OTA firmware image files to officially enable Zigbee OTA updates of Philips Hue devices via a third-party Zigbee Gateway, as so far they only allow Zigbee OTA updates of their devices via the official Philips Hue Bridge.

Meaning that Philips on purpose aim for vendor lock-in for that feature → Vendor lock-in - Wikipedia Today that is really considered a bad faith practice as access to OTA firmware update files to Zigbee devices that you bought and paid for should be considered as fair use even if you do not use a companies propriatory Zigbee Gateway → Fair use - Wikipedia

Anyway, Home Assistant’s built-in ZHA (Zigbee Home Automation) integration and the zigpy library/framework it depends on does support Zigbee OTA Upgrade using the standard Zigbee OTAU format, but the problem is with the companies making Zigbee devices as most of them are simply not providing Zigbee OTA images third-parties or just making them publicly available. See → https://github.com/zigpy/zigpy/wiki/OTA-Information-for-Manufacturers

This could be a non-issue for third-party Zigbee gateway solutions if all manufacturers of Zigbee devices simply publicly posted URLs to unencrypted Zigbee OTA firmware images for their devices to allow third parties to download them directly from the official source. Philips/Signify and many if not most manufacturers do unfortunately not do so. However, some companies do, which is why you can perform OTA upgrade of those directly in Home Assistant’s built-in ZHA (Zigbee Home Automation) integration, (specifically; IKEA, LEDVANCE/OSRAM, SALUS/Computime, INOVELLI, Sonoff/ITead, THIRDREALITY, and a handful more are among the few companies that provide official unencrypted Zigbee OTA upgrade image files available via public servers), and a there are also a few other which zigpy is missing the download code for, see zigpy feature requests → https://github.com/zigpy/zigpy/issues?q=is%3Aissue+is%3Aopen+ota+AND+request

The workaround some other Zigbee gateway solutions (such as example Zigbee2MQTT) provide for getting the images in unofficial ways is to host collections with copies of official Zigbee OTA firmware images that have been collected in various unofficial ways (e.g. either dump the embedded EEPROM of an already updated Philips Hue device or use PCAP packet capturing traffic sniffing techniques to eavesdrop on the communication to the Philips Hue Bridge when it is downloading OTA firmware images and then reconstruct an OTA image from a series of such PCAP packet captures, non of which methods have legally been tested in a court of law but since the actual firmware images can only be used by owners of those devices to update the original devices there is probably a strong argument for “fair use”), see example → https://github.com/Koenkk/zigbee-OTA

Blame the companies that manufacture devices but do not publicly offer official Zigbee OTA source providers that third parties can use too.

Yes I am sure most users would think that it would be very nice if some similar unofficial workarounds were natively built-in into Home Assistant’s built-in ZHA (Zigbee Home Automation) integration or the zigpy library/framework it depends on, and technically it would not be difficult for developers to add an unofficial source (like a GitHub repository with copies obtained via various unofficial means), but there are also many arguments why not include such unofficial sources, like for example; who the users should blame if the OTA upgrade image break/brick their device(?), and what if doing so in an unofficial way would get Nabu Casa and/or the Home Assistant project on bad side of companies that might someway otherwise want to partner with Nabu Casa and join the “Works with Home Assistant” program in the future → https://partner.home-assistant.io/ as easy firmware upgrades is part of the idea behind that partnership → https://www.home-assistant.io/blog/2022/07/12/partner-program/

“Responsibilities = "Provide easy firmware updates: Works with Home Assistant partners are expected to provide firmware updates via user-friendly services provided by Home Assistant”

FYI, the developers of Home Assistant’s built-in Z-Wave integration which do have a matching GitHub repository with caches OTA images for Z-Wave devices have written this related statement into their Z-Wave JS Firmware Update Service documentation addresses the question about providing firmware definition files from unofficial sourcs:

“Warning: We will not accept firmware updates hosted by third parties. All updates must come from the respective device manufacturer. We make an exception for firmwares that are publicly hosted by the manufacturer, but those may still require confirmation the manufacturer’s confirmation before merging.”

Again, the best option for all would be if all device companies could be convinced to allow third parties to download official Zigbee OTA image files directly from official sources to be able to simply add download code to zigpy for enabling the ZHA integration to use them (regardless if they choose to partner with Nabu Casa for the “Works with Home Assistant” program or not) → https://github.com/zigpy/zigpy/wiki/OTA-Information-for-Manufacturers

As explained above, until official sources are made available by those companaies, a Do-It-Yourself workaround for Home Assistant’s built-in ZHA (Zigbee Home Automation) integration is to download unofficial Zigbee OTA images manually from example https://github.com/Koenkk/zigbee-OTA (follow https://www.home-assistant.io/integrations/zha#ota-firmware-updates and https://github.com/zigpy/zigpy/wiki/OTA-Device-Firmware-Updates for how to add files manually to ZHA) or download automatically via scripts from unofficial sources. See example zha-toolkit → https://github.com/mdeweerd/zha-toolkit/blob/main/README.md#ota_notify—downloadtrigger-device-fw-update

PS: FYI, there is also a somewhat related feature request here regarding making OTA updates easier to apply per device in the ZHA integration → ZHA integration UI options to apply OTA firmware update (OTAU) per Zigbee device?

[Hedda]

Here is by the way a few good video guides showing how very easy it is these days to start from scratch with setting up and begin using Home Assistant’s built-in ZHA integration as a Zigbee gateway solution:

Set up Zigbee in Home Assistant - Using ZHA to ditch your 3rd party hubs

How to Set Up ZHA in Home Assistant - TUTORIAL

Set up Zigbee with Home Assistant - ZHA Integration guide

Zigbee + Home Assistant: Ultimate Beginner's Guide (2024)

Home Assistant and Zigbee is all you need!

Also check out this showing a scratch setup by a complete new beginner who never used Home Assistant:

The Start of My Smart Home | Home Assistant Setup

Watching these can remove the FUD (Fear, Uncertainty, and Doubt) symdrome from beginners

[jonasgustavsson (Jonas)]

Regarding binding: I have not managed to get it to work with the slightly older RWL021 model. Hence, it may only be RWL022 that supports binding.

Also, for actually using the dimmer switch, I used the blueprint in post #95 in this thread: ZHA - Philips Hue Dimmer Switch (RWL020, RWL021) - #95 by Twiglet

[Hedda]

Philips has now shut down their “Hue Labs” which offered customizable “Hue Lab Formulas”. Guess that is another good reason why more people would want to migrate away from the Philips Hue Bridge:

• https://www.philips-hue.com/en-us/support/article/huelabs/000003

• https://labs.meethue.com/news/test1

Does anyone have tips on replicating different Hue Labs Formulas from Philips Hue Labs?

Philips Hue themselves have posted how users of the official Philips Hue Bridge and Hue app can partially replicate some Hue Labs Formulas on the official Philips Hue Bridge but even their fans are complaining that the new features Philips introduced in their Hue app can not fully replace all the popular “Hue Labs Formulas” in the same way.

• Article Support Page | Philips Hue

Smart Home Point has posted this video and these blog posts that cover some of these wanted effects:

Hue Labs Has Shut! How To 'Fix' Philips Hue App Lighting Effects

• Hue Lab Formula Examples – Color Loop, Living Scenes (More)
• Can Philips Hue Cycle Colors? - A Tutorial - New Video
• Philips Hue Lights Can Change Color Automatically (10 Ways)

It depends if those effects are programmed as attributes in the actual device firmware or if all of the magic for those effects happens in software of a native app on a mobile or the software application running on the hub (or in the cloud).

If all of the magic for those effects happens in the native app or the hub and not onboard the device itself then you would manually have to replicate those effects using “scenes” and “automations” or “scripts” in Home Assistant. Similar to asked and answered here → https://community.home-assistant.io/t/animations-effects-like-hue-labs-for-z2m-lights/554252/2

But (more likley) if those effects are programmed and activated via attributes in the actual device firmware then you would need code for a device handler with custom “quirks” for that specific device model in the ZHA Device Handlers library.

Zigbee devices that do not only use standard configuration and parameters (default ZCL clusters and attributes) but instead also implement custom manufacturer clusters and attributes (also known as “quirks”) will need a custom handler/converter/parser/translator as Python script code in the upstream “ZHA Device Handlers” library repository to extend custom manufacturer device “quirk” support for specific non-standard ZCL clusters and attributes (which Zigbee gateway implementations that depends on zigpy and the ZHA Device Handlers library, like Home Assistant s ZHA integration, then can make use of).

https://github.com/zigpy/zha-device-handlers/#readme

If you can not code that Python script code yourself to write the needed custom handler/converter/parser/translator for adding that specific device to the “ZHA Device Handlers” repository/library then suggest submitting a “device support request” (with device signature and diagnostic information) as new issues → https://github.com/zigpy/zha-device-handlers/issues

See this example just as referece → https://github.com/zigpy/zha-device-handlers/issues/1927

Without a “device support request” as a new issue posted to the “ZHA Device Handlers” repository/library requesting support for an unsupported device or device feature with Device signature and Diagnostic information the ZHA integration developers will not even know about the device unless they by random chance happened to have bought it themselves.

The reason is why non-standard devices need a custom handler/converter/translator is explained in ZHA integration documentation here → https://www.home-assistant.io/integrations/zha#zha-exception-and-deviation-handling

Zigbee devices that use clusters and attributes that are standard in the official ZCL (Zigbee Cluster Library) do not need custom handlers/converters/translators as explained in the ZHA integration documentation here → https://www.home-assistant.io/integrations/zha#knowing-which-devices-are-supported

PS: Off-topic but FYI, this also works kind of similarly for Zigbee2MQTT which also requires a custom handlers/converters/parsers/translators for specific devices → https://www.zigbee2mqtt.io/advanced/support-new-devices/01_support_new_devices.html

[Hedda]

Here is by the way a few good video guides showing how very easy it is these days to start from scratch with setting up and begin using Home Assistant’s built-in ZHA integration as a Zigbee gateway solution:

…

Also check out this showing a scratch setup by a complete new beginner who never used Home Assistant:

…

Watching these can remove the FUD (Fear, Uncertainty, and Doubt) symdrome from beginners

I think the biggest hurdle for most new users is that they do not know how easy it is to setup ZHA today.

Therefore this step-by-step purchase recommendation might help nudge new users to take the plunge.

What to buy and do if you would like avoid some common pitfalls when starting from scratch:

1. Buy a Zigbee Coordinator USB radio dongle that is well supported by the community, like either:
   1a): Get the EFR32MG21-based official Home Assistant SkyConnect USB radio dongle/stick (which was later renamed to “Home Assistant Connect ZBT-1” by Nabu Casa) if you are sure that you will only use Home Assistant’s built-in native ZHA (Zigbee Home Automation) integration, but if you do make sure to first update to the latest EmberZNet (NCP) firmware using the official Web-Flasher on a different computer before getting started with it, that will make it into a dedicated Zigbee Coordinator and disable the multi-PAN/multiprotocol support that is not longer recommended. The downsides to Silicon Labs based radios are not recommended for Zigbee2MQTT and as the SkyConnect model does not have an external antenna it is extra important to start adding enough Zigbee Router devices as Zigbee repeaters/extenders (especially before adding any battery-powered devices). The upsides is that it is very easy to update firmware on it from Home Assistant, and even easy migrate to a different Zigbee Coordinator adapter using the ZHA integration if you ever want to repurpose this SkyConnect dongle hardware for Matter by reflashing it into a dedicated Thread radio adapter using OpenThread firmware instead.
   1b): Alternatively, you could instead get the CC2652P-based “Sonoff Zigbee 3.0 USB Dongle Plus (which was later renamed to “ZBDongle-P” by ITead), and flash latest Z-Stack 3.x.0 community firmware on it using ZigStar Multi Tool on a different computer before getting started with it. The reason for that is that with a CC2652P-based Zigbee Coordinator you can begin by using Home Assistant’s built-in native ZHA (Zigbee Home Automation) integration but it still leaves you with the option to later reuse that same CC2652-based Zigbee Coordinator adapter if you even want to move from Home Assistant’s built-in native ZHA (Zigbee Home Automation) integration to to the stand-alone Zigbee2MQTT (a.k.a. Z2M) Zigbee gateway application (which acts like an external third-party hub/bridge, but while it is known for great Zigbee device compatibility it targeting advanced users that need to tweak more low-level Zigbee attributes). Another bonus with this “ZBDongle-P” dongle is the external antenna which makes it get better reception, (not having an external antenna should only be a problem if you have not add enough Zigbee Router devices as Zigbee repeaters/extenders).
   1c): Another alternative to a USB radio dongle that is not recommended can be to instead get a network-attached Zigbee Coordinator Ethernet radio adapter (which connects using TCP/IP over a wired LAN), but using one of those instead of a USB radio dongle increases the complexity of the setup and makes it more complicated to troubleshoot issues + means you need to replace the whole network-attached Zigbee Coordinator adapter if you ever want to upgrade. Personally I suggest that new users should avoid getting a network-attached Zigbee Coordinator adapter.
2. Since you want to place your Zigbee Coordinator USB radio dongle at least 0.5-meter/2-feet away from the computer running Home Assistant or even longer to avoid it causing EMI interference with the Zigbee Coordinator I recommend buying a long shielded USB extension cable and always use it when connecting the Zigbee Coordinator USB radio dongle to the computer running Home Assistant.
   2a): SkyConnect dongle does ships with a 0.5-meter/2-feet but that is really be minimum. Better if you buy an even longer shielded USB extension cable than that. This cable should preferably be well shielded, and easiest for the next step below is if it is a USB 2.0 extension cable, (however most USB 3.0 extension cable are usually better shielded but then you need to make sure following the next step below about only connecting to a USB 2.0 port or USB 2.0 hub).
   2b): Alternatively, if you want to have the dongle in a different room even from the computer running Home Assistant I suggest buying a “USB Ethernet RJ45 Extender Adapter” converter-kit and and a single long CAT5e/CAT6 shielded Ethernet cable with RJ45 connectors as such a converter will allow you to convert any CAT5e/CAT6 Ethernet cable into a very long dumb USB extension cable (note that 30 meters or 100 feet is the recommended maximum length for USB 2.0 data traffic over such a passive cable). There are many inexpensive variants of such USB Extender Over RJ45 adapters available, see example this → https://www.amazon.com/Male-RJ45-Female-Extension-Adapter/dp/B083W3D65G
3. Connect the Zigbee Coordinator USB radio dongle you bought via the long shielded USB extension cable to a native USB 2.0 port on your computer/server. Do not connect it to a USB 3.0/4.0 port as those are infamously well known to generate serious EMI/RMI/EMF interference to all low-power 2.4GHz devices (including Zigbee, Thread. and Bluetooth).
   3a): If your computer/server does not have a native USB 2.0 port then buy and connect a powered USB 2.0 hub (with an external-supply) and then connect the USB extension cable + Zigbee Coordinator USB radio dongle via that powered USB 2.0 hub, (note that is needs to be power USB 2.0 hub with external-supply and not a USB 3.x hub / USB 4.x hub or no external-supply). See example → https://www.amazon.com/AmazonBasics-Port-USB-Power-Adapter/dp/B00DQFGJR4/
4. Place your Zigbee Coordinator radio adapter in a location where it is not too close to any known sources of EMI/RMI/EMF interference. That includes all types of electronics and appliances or cables/wired with electricity. Seriously, all Zigbee radios are extremely sensitive to electromagnetic interference and electromagnetic fields, to the point of it being ridiculous sometimes.
   4a): Preferably also place your Zigbee Coordinator radio adapter so that it is not too close to a wall, ceiling, floor, etc. as just getting it 25-centimeter/10-inches away from any building surface will make it get better reception.
5. After you are done initial installation/configuration of Home Assistant’s built-in native ZHA (Zigbee Home Automation) integration, then start adding all mains-powered Zigbee Router devices, beginning with those that are closest to the Zigbee Coordinator. The best process is if you finish doing this 24-hours before start adding battery-powered Zigbee devices.
   5a): If you do not have many mains-powered Zigbee Router devices or you do not want to start by moving such devices from their existing Zigbee network(s) then you need to consider buying a few dedicated mains-powered Zigbee Router devices to connect first. Regardless, be aware that adding a few “known good” dedicated mains-powered Zigbee Router devices to your Zigbee network will make it more stable and robust. See → Zigbee networks: how to guide for avoiding interference + optimizing using Zigbee Router devices (repeaters/extenders) to get best possible range and coverage
6. If you have problems pairing a device, do not move it closer to the Zigbee Coordinator USB radio adapter. Even though that approach is recommended by some manufacturers, the general best practice if that Zigbee devices should be paired in the location where it’s going to be used.
   6a): If your paired/joined Zigbee device is not showing all expected entities (attributes) in the ZHA integration then first try re-pairing it a few times, it is likely that the device interview process was interupted for some reason, and some battery-powered devices might have to have their batteries replaced and/or manually be kept alive by activating a change state on the device (like for example triggering a motion sensor with motion).
   6b): if your paired/joined Zigbee device still is not showing all expected entities (attributes) in the ZHA integration after being re-paired then chances are it will need custom ZHA Device Handlers (also known as “zha-quirk”) for that specific device. This is more common if it is a newly released or brand new device that no one added support for previously or an odd device with complex features that almost no one else have bought due to it being a niche product with a very small userbase. See → Zigbee Guide: How-to setup local custom device handler quirks in ZHA integration
7. When in doubt, keep adding more mains-powered Zigbee Router devices!
   7a): Pro-tip is that you can easily convert inextensive Zigbee Coordinator USB radios adapter like the “Sonoff ZBDongle-P” and “Sonoff ZBDongle-E” models into very powerful “known good” dedicated mains-powered Zigbee Router devices by simply re-flashing them with Zigbee Router firmware and then powering them via USB-charger adapters to make the best permanent Zigbee repeaters/extenders that you can currently buy. The alternative is to just buy a few pre-made dedicated Zigbee Router products, like the “IKEA Tradfri Signal Repeater ” and/or the “Aeotec Range Extender Zi ” (which work very well out-of-the-box but are not as good as the above mentioned DIY solution).

Additional note: If you have any Zigbee lightbulbs/lights that are Zigbee Router devices (which most are) then you need to make sure that the power for those are not connected via a dumb switch (like an easily accessible wall-switch) that people that flick ON and OFF. Zigbee Router devices are meant to bve always connected and always powered, and if they constantly become unavailable (unnessesary) on a regular basis then that will mess up your Zigbee network mesh paths as devices will need to keep finding alternative routes/paths in your Zigbee network mesh. And understand that while it might alleviate the situation a little by adding a few “known good” dedicated Zigbee Router devices in strategic locations it is not a permanent alternative to leaving your Zigbee lightbulbs/lights connected to dumb switches if they are Zigbee Router devices.

If you do that then getting started should be more or less plug-and-play as avoided most common pitfalls.

PS: Again, highly recommend you also read and do your best to follow all the best practice tips here → Zigbee networks: how to guide for avoiding interference + optimizing using Zigbee Router devices (repeaters/extenders) to get best possible range and coverage

[Hedda]

@Hedda

Also need to mention how-to Zigbee OTA (Over-The-Air) firmware upgrade Philips Hue devices.

ZHA integration can perform updates for some Zigbee products out-of-the-box because the company who manufacturer/sell those already publish some or most of their unencrypted Zigbee OTA firmware images publicly, unfortunately, Philips/Signify is not one of those companies. But perhaps the userbase can someday convince them to do that, more information about that here → OTA Information for Manufacturers · zigpy/zigpy Wiki · GitHub

Fortunately, if you still want to upgrade firmware on your Philips Hue Zigbee devices then manual OTA firmware upgrade is at least still possible via the ZHA integration if you can source compatible Philips Hue Zigbee OTAU image files from elsewhere, see → OTA Device Firmware Updates · zigpy/zigpy Wiki · GitHub

Update on Zigbee OTA firmware updates for Philips/Signify devices: From an end-user perspective, otau_dir is gone since February this year and has been replaced with z2m_remote_index and z2m_local_index as alternative to get URLs for updates from third-party.

Be aware that while ZHA integration can still not perform Zigbee OTA updates on Philips Hue devices out-of-the-box, but it is now relatively easy to manually enable it via advanced zigpy configuration by editing Home Assistant’s YAML configuration file and point to URL of Koenkk’s Zigbee OTA repository which is an unofficial collection of extracted Zigbee OTA update files that is used in Zigbee2MQTT (also known as “Z2M”) for optional OTAU of Zigbee devices when manufacturers do not host public download for third-parties:

zha:
  zigpy_config:
    ota:

      enabled: true # This should already be enabled by default

      z2m_remote_index: https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master/index.json # z2m_remote_index URL points to Koenkk's zigbee-OTA `index.json` used in Zigbee2MQTT

      z2m_local_index: /path/to/z2m-ota/index.json # z2m_local_index points to a Z2M OTA `index.json` file if stored locally

• Edit configuration.yaml, restart Home Assistant, then Settings → System → Updates → Refresh.

Note! This is experimental and should at this time only be used by developers. The developers of the ZHA integration have said they have a long-term plan to eventually add their own Zigbee OTA repository which will only contain tested and verified Zigbee OTA files from official sources.

OTA v2: provider and metadata rewrite

So for now you have the option to choose to use unofficial sources at your own risk, which could brick the devices since flash from unofficial sources, however also keep in mind that it is generally recommend to not update OTA firmware Zigbee devices unless you are having specific issues with a device and then only update that device, (i.e. “if it ain’t broke, don’t fix it”).

FYI, ZHA docs for end users will not get this info since again should only be used by developers, see:

Update ZHA section about Zigbee OTA

PS: ZHA/zigpy developers are currently working on making it easier to edit these and other OTA config:

Use an OTA provider configuration list instead of per-manufacturer keys

Pass all compatible firmware metadata when receiving an OTA upgrade request