Would this provide a way to commission nanoleaf bulbs to an openthread border router?
I’ve been digging into seeing if I can break down the BLE protocol and possibly use that but I feel a little out of my element in doing so.
I just want a way to control my bulbs from an MQTT wall switch and have gone down so many rabbit holes, at least I finally have a functioning openthread network.
@TheTofu I did find pieces of the Nanoleaf Thread commissioning over BLE. I don’t know where the network details are created (the Elements/… Border Router, the app, …?) before being provisioned onto the bulbs. It may not matter. There may be other, more standard, provisioning paths for other Border Routers, e.g. the HomePod Mini. I don’t have one of those (nor the HAP/Thread specs) so I can’t say for sure.
Of interest to you, perhaps, I did get an OpenThread BR to join the existing Nanoleaf Thread network using a $10 nRF52840 dongle although I didn’t test it at all. I may revisit that at some point as the Elements BR seems to have a reproducible bug for my gear where although bulbs immediately join the Thread network they aren’t reachable for up to 2 hours. The same USB dongles make great Thread sniffers.
Hi @lambdafunction, I have the same dongle but couldn’t figure out how to flash it with the Border Router firmware. Did you set it up as an RCP or NCP? If you followed a guide to configure the device, would you mind sharing the link?
I have a few Nanoleaf devices (Shapes, Light Strip and bulbs), plus a Google Nest WiFi network that, at least in theory, is capable of joining the Nanoleaf thread network. I’m hoping to use these bits and pieces along with the dongle to get decent Thread coverage across my home, and automate my lights via HomeAssistant. I’ve been lurking in this thread for weeks trying to contain my excitement 
@dwurf see this OpenThread doc although I used nRF Connect for Desktop to flash the firmware.
EDIT: to be clear, the RCP firmware is 1/2 of the solution. You’ll need the actual BR software to run the RCP. I followed this doc: OpenThread Border Router Build and Configuration. I grabbed the Debian 11 “nocloud” qemu image, ran it in kvm, and installed the OTBR bits on top.
I currently have 2 nordic dongles and an MDK dongle, but I have no other nanoleaf devices besides the bulbs and an essentials strip, nor do I have any other thread routers, so mainly investigating either BLE or reverse engineering the EUI64 somehow.
Did some digging with wireshark for mDNS and BLE to see if I could figure out what the app is doing but didn’t really have any luck. Currently thinking of trying one of these to see if I can get my bulb to pair, and then hopefully I’ll be able to figure out some way to get them to provision.
@TheTofu If you check the bottom of the nanoleaf-ltpdu README you’ll find the command the Nanoleaf app sends to the bulb to provision Thread network details (0x0801). Not sure what you mean by reverse engineering the EUI64 but there’s also a command to retrieve that value (0x0202).
I think aiohomekit has a CLI you could use to pair. BLE and CoAP are both available in the dev branch.
Ah, sorry was having a bit of a mental rambling and wasn’t sure if BLE would use the same commands, which if it does then that makes life a good bit easier.
As far as aiohomekit I’m just not sure how exactly to use it so I was trying to find something already decently well developed just to get BLE working and then work on the thread/coap side.
One thing I’m unsure of is does BLE use coap? From everything I can find it doesn’t, but not sure if home kit is doing some extra stuff to encapsulate it or something.
Before Thread provisioning, the bulbs do HAP over BLE if you’ve got an Apple device. I don’t know what Android does over BLE yet but there are 2 non-HAP BLE services available.
After Thread provisioning, the bulbs do HAP over CoAP for Apple and a custom Nanoleaf protocol over CoAP for Android (LTPDU, the initial scope of this feature request). The Border Router translates the link-layer protocol from WiFi to Thread and back.
HAP defines the entire protocol including communication. It just so happens that the data frames are almost identical over BLE and Thread (HAP PDUs). The data frames look completely different for (non-Thread) WiFi devices.
Here’s an example of a single request as seen on the Thread network:
Everything from Internet Protocol Version 6 down is more or less copied as-is from my Home Assistant dev instance. The Data bytes are encrypted HAP over CoAP over UDP over IPv6 over Thread (6LoWPAN/802.15.4).
If you had a dump of the BLE communication, it would just be the Data portion on top of BLE signaling.
Finally got my Nanoleaf border router, seemed to take forever to get the latest firmware (seemed to go through many point releases instead of going straight to the latest).
Hope to get back to finishing off the dev branch in aiohomekit, but work is hectic atm so i’ve just been too done in to pick anything up after work.
The HA release today has the bulk of the changes to make the dev branch work with HA. Still a couple of things I need to rework.
After that the remaining work is test coverage for aiohomekit, documentation about what we know about getting a mesh up and running, and testing for regressions,
Some interesting news, apparently Nanoleaf’s current essentials line isn’t going to support matter, which is quite frustrating as that’s half the reason I bought these bulbs :
Hopefully they’ll at least make it easier to add the bulbs onto any thread network, rather than just the few specific devices they support now.
I haven’t had any luck trying to get the damn things to provision over BLE, debating whether to get a Nanoleaf controller off ebay or something just for the border router.
https://www.theverge.com/2022/3/24/22994597/matter-delay-nanoleaf-essentials-eve-wemo
While there may be more certainty in the industry, current smart home owners are still in a quagmire of uncertainty around which products to buy and when. Especially now that Chu has confirmed to The Verge that Nanoleaf’s Thread-enabled A19 smart light bulb and light strip won’t work with the standard. “[They] will require a different chip,” says Chu. “We hope to launch a new Essentials A19 and a new light strip when Matter arrives this year.” This is a blow to owners who bought the products thinking that being Thread-enabled meant they would be upgradable to Matter. The devices will continue to work with HomeKit over Thread, however, and with other platforms either over Bluetooth or over Thread using a cloud-to-cloud integration, says Chu.
I lost motivation to do any more tinkering on the BLE side of things since I don’t really know what I’m doing, and ended up buying a cheap shapes kit on ebay. I was able to get everything configured and working on thread, but then had to spend a bit more time to get everything working in home assistant as well.
I currently have 1 shapes controller, 1 essentials strip and 3 essentials bulbs working well and responding fairly quickly, will update if I find any issues down the line.
Current steps I had to take:
Pair all devices to nanoleaf android app, make sure firmware is up to date, then factory reset everything and remove from app. Pair just the shapes controller and make sure it’s showing up in thread in the app, it may take a few restarts of both the controller and the app to get it to actually work.
Pair any essentials strips or bulbs to the app, and confirm they show up in thread, and then power them off (might not need to power them off but I only paired one at a time to home assistant just in case)
Add the following repository to the addon store in home assistant https://developers.home-assistant.io/ and install the Custom Deps Deployment addon, do not start it yet. Go to the config page and add the following and hit save:
pypi:
- git+https://github.com/Jc2k/aiohomekit.git@dev
- git+https://github.com/roysjosh/aiocoap.git@fix-269
apk: []
Start the addon and look at the log tab to make sure everything was successful. Make sure you’re not using the built in nanoleaf integration, then restart home assistant from settings.
Edit the config_flow.py in homekit_controller and the generated zeroconf.py in HA Core.
You can do this by running the following
find / -name config_flow.py | grep homekit_controller
find / -name zeroconf.py | grep generated
If you’re running supervised then you can search in /var/lib/docker/overlay2, I think only the overlay with merged needs to be edited but I modified any instance I could find just in case.
You need to modify the following:
config_flow.py
await conn.pairing.connection.reconnect_soon()
change to
await conn.pairing.connection.reconnect_soon(updated_ip_port)
zeroconf.py
add under _hap._tcp.local
"_hap._udp.local.": [
{
"domain": "homekit_controller"
}
],
Then restart
Notes:
It took me a while to understand what Custom Deps Deployment actually did, as before that I kept trying to copy homekit_controller from various branches to my custom_components folder and running into issues.
Eventually I thought I solved it by specifying lambda’s fork of aiocoap, which worked to get a single essentials light paired, but kept running into the binding issue when trying to pair others, at which point I realized I needed the specific fix branch, after which everything started working properly.
Device discovery worked at first, but after the first few restarts it stopped notifying me, possibly because I was waiting until I was already at the integrations page before turning on my bulbs to pair.
Hopefully the aiocoap issue gets resolved soon so the everything can be merged into core finally.
Thanks to both @lambdafunction and @Jc2k for their work on this!
Edit: Added a couple extra steps to get homekit_controller to update if the border router assigns a new address.
Bad news, it seems like power cycling the thread border router, in this case a shapes kit (may be nanoleaf specific issue), causes any paired thread devices to timeout on verifying pairing.
The shapes connection still works, I did notice that the device I tested with (an essentials strip) comes up with a different IPv6 address when it reconnects to thread after the shapes are power cycled, not sure if that makes any difference. Added log output below.
Logger: homeassistant
Source: deps/lib/python3.9/site-packages/aiohomekit/controller/coap/pairing.py:73
First occurred: 8:09:11 PM (3 occurrences)
Last logged: 8:09:54 PM
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/local/lib/python3.9/asyncio/tasks.py", line 492, in wait_for
fut.result()
asyncio.exceptions.CancelledError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/connection.py", line 394, in connect
await self.do_pair_verify(pairing_data)
File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/connection.py", line 353, in do_pair_verify
response = await asyncio.wait_for(
File "/usr/local/lib/python3.9/asyncio/tasks.py", line 494, in wait_for
raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/pairing.py", line 71, in _ensure_connected
await self.connection_future
File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/connection.py", line 397, in connect
raise AccessoryDisconnectedError("Pair verify timed out")
aiohomekit.exceptions.AccessoryDisconnectedError: Pair verify timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/pairing.py", line 139, in subscribe
await self._ensure_connected()
File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/pairing.py", line 73, in _ensure_connected
raise AccessoryDisconnectedError("failed to connect")
aiohomekit.exceptions.AccessoryDisconnectedError: failed to connect
Update: I added the lines to my config but was looking at the clickable logs not the full log, it seems like the IPv6 address changing is the issue after all
2022-05-17 20:19:46 DEBUG (MainThread) [homeassistant.components.homekit_controller.connection] Starting HomeKit controller update: AE:2A:98:FF:FF:FF
2022-05-17 20:19:46 DEBUG (MainThread) [aiohomekit.controller.coap.connection] Pair verify uri=coap://[fd03:9201:7cb1:0:e150:20c3:e14e:d375]:5683/2
2022-05-17 20:19:46 DEBUG (MainThread) [aiohomekit.protocol.tlv] sending [
2022-05-17 20:19:54 WARNING (MainThread) [aiohomekit.controller.coap.connection] Pair verify timed out
It’s still trying to connect to the old IPv6 address, I can open a ticket in github as well, would this be aiohomekit or aiocoap?
@TheTofu I’m 99% sure I have a local change that tracks zeroconf address updates, it just isn’t committed or pushed. A restart of HA should get you going in the meantime. IIRC it took a change to both the HA homekit_controller component as well as aiohomekit. I’ll push that code soon.
I’ve tried a few restarts and it didn’t do anything, though I also haven’t had any new integration/device notifications in a while so I wonder if something’s wrong with my zeroconf or it just doesn’t like being in a proxmox VM? Setting the logs to debug it seems like it’s working fine though…
Hey, please try to make this change locally:
diff --git a/homeassistant/components/homekit_controller/config_flow.py b/homeassistant/components/homekit_controller/config_flow.py
index 979d47d9cd..9d9e5557ff 100644
--- a/homeassistant/components/homekit_controller/config_flow.py
+++ b/homeassistant/components/homekit_controller/config_flow.py
@@ -247,7 +248,7 @@ class HomekitControllerFlowHandler(config_entries.ConfigFlow, domain=DOMAIN):
# will do nothing if the device is already connected
# Note that not all backends have this
if hasattr(conn.pairing, "connection"):
- await conn.pairing.connection.reconnect_soon()
+ await conn.pairing.connection.reconnect_soon(updated_ip_port)
if conn.config_num != config_num:
_LOGGER.debug(
"HomeKit info %s: c# incremented, refreshing entities", hkid
You may have to find / -name config_flow.py | grep homekit_controller unless you know where it is on your HA install already.
@TheTofu you could also set these loggers to debug in your configuration.yaml:
logger:
default: info
logs:
homeassistant.components.zeroconf: debug
zeroconf: debug
@lambdafunction I am having similar issues with the bulbs randomly seeming disconnected. A few times this has happened because I rebooted the Nanoleaf shapes unit and other times it just seems random and a reboot fixed it.
I was able to manually edit using the new ipv6 addresses and it starts working again. I tried changing the file you mentioned and it didn’t make a difference. I’ll try turning on the debug functions you mentioned and report back when it happens.
@psumatt in that case please also enable aiohomekit: debug and homeassistant.components.homekit_controller: debug
Ok, so I enabled the debug logs, but there are a ton of them, and not sure what to look for here. That said, it seems your change did work, upon reboot. I had tried it a few times and it wasn’t working, but then I went in yesterday and searched for that file again. It seems there were two copies of the homeassistant folder - I had modified one of them but not the other (I guess I chose wrong?).
So, @lambdafunction - is it expected that it would pick up the new IP address automatically or only after a restart? Nanoleaf as a thread router seems to generate a new IP address for the individual nodes anytime it disconnects from the router or is restsarted.