HomeKit Accessory Protocol (HAP) over CoAP/UDP (was: Nanoleaf Essentials bulb via Thread/CoAP)

@TheTofu Hmm let me make sure I don’t have any other commits locally… Also make sure your edit stuck & didn’t get reverted after an update?

I just unplugged my Elements BR & within ~5 minutes the Essentials were back online:

Device address update from zeroconf: old=[fdc3:5f:3f7f:0:bdc1:27fc:e1f8:4807]:5683 new=[fd71:826a:ca3d:0:4689:4bef:e1c4:a5b3]:5683
Device address update from zeroconf: old=[fdc3:5f:3f7f:0:a6ca:42ef:852d:bbc2]:5683 new=[fd71:826a:ca3d:0:824e:1cf:ae38:e3ba]:5683
Connected to CoAP HAP accessory at [fd71:826a:ca3d:0:824e:1cf:ae38:e3ba]:5683!

Do you see any “Device address update from zeroconf” lines in your logs? You’ll need debug turned on for aiohomekit. Also, what HA version are you on? Thanks!

Yeah, I double checked this morning and the edit stuck (the update thing happened to me on friday lol) , I remember testing before and connection was restored if the controller was unplugged for just a minute or two (I even tested it just now and it seemed to reconnect fine after a few minutes), I might try unplugging for longer (20 minutes+) to see if that does anything different.

If it happens again I’ll see if I can check the logs for the address update, I’m currently on 2022.6.1, have held back on updating so I don’t have to reapply the edits constantly.

So small update, it seems like when it doesn’t reconnect zeroconf for whatever reason isn’t picking up the bulbs, but so far power cycling the shapes is all I need to do to get it working again instead of restarting HA.

Is there any way to send a reboot command to them?

Not that I know of though I haven’t really done anything with the other NL products. Cheap WiFi relay/outlet?

When https://github.com/home-assistant/core/pull/74857 lands, all of my outstanding homekit_controller changes have landed on dev. This will mean the dev and main branches of aiohomekit will work interchangeably with the dev branch of home assistant (what will become the august release). I.e. you won’t need to patch anything to do with home assistant, you’ll just need to update aiohomekit and possibly aiocoap. This is good - the 1.0.0 release of aiohomekit is massive, and being able to roll it back easily makes this all the safer.

I still need to look into Pass along IP updates from zeroconf by roysjosh · Pull Request #49 · Jc2k/home-assistant · GitHub. It doesn’t look like the TCP/IP backend needs this change. So i’d like to make sure of that, and if so make UDP do it the same way. (reconnect_soon is going away completely soon).

I still don’t have much free time but currently trying to finish cleaning up some Zeroconf changes in aiohomekit that are blocking 1.0.0. When those are done i’ll be merging the dev branch, releasing 1.0.0 and rolling it out with homekit_controller. Hopefully this will be in the August release. This means the code for CoAP and BLE will be shipping in a prod release without any git branches of home assistant or aiohomekit. Depending on how well testing goes, one or both of them will be behind feature flags (probably env variables).

Realistically CoAP will be behind a feature flag until the 3 issues with aiocoap referenced in https://github.com/Jc2k/aiohomekit/issues/68 are resolved and in a stable release of aiocoap (or we have been able to work around them in aiohomekit).

BLE is probably going to get the most attention with the little time I have this month - there is a big BLE push right now and exciting stuff happening elsewhere in the project (e.g. https://github.com/home-assistant/core/pull/74653) which i’m hoping to leverage. But it would also be good to verify that I haven’t completely broken CoAP without all the branch faff.

1 Like

1.0.0 is out, and the PR to land it in HA is almost over the line (https://github.com/home-assistant/core/pull/75198).

When that is there, the BLE and CoAP code are all present but disactivated. You can use the AIOHOMEKIT_TRANSPORT_BLE=1 and AIOHOMEKIT_TRANSPORT_COAP=1 env vars to turn them on. When we’ve ironed out all the big wrinkles we’ll get rid of those feature flags.

Right now BLE is hit and miss, it seems to work for the Eve devices I have access to but I can’t get it to work for my Nanoleaf light strip (which i was hoping to get working on thread next).

I haven’t had chance to check CoAP/Thread yet. I know that at the very least still need to look into: Pass along IP updates from zeroconf by roysjosh · Pull Request #49 · Jc2k/home-assistant · GitHub.


Thank you for all the work, excited to see things being merged into HA proper, I’ve been holding off on updating HA until fixes were merged.
Hopefully one day we can get thread commissioning working through ble, Nanoleaf also updated their android app quite a bit recently (I’m assuming ios app got updated as well) so I might revisit sniffing ble packets to see if I can figure out something with thread.

Would be interesting to see them update the desktop app with similar functions, but at least now the android app actually shows the thread network properly now.

@TheTofu the BLE comms are mostly known, including Thread provisioning.

The BLE service 6d2ae1c4-9aea-11ea-bb37-0242ac130002 contains the Android characteristics:

  • secure endpoint 6d2adfda-9aea-11ea-bb37-0242ac130002
  • CoAP endpoint 6d2adda0-9aea-11ea-bb37-0242ac130002

Similarly to /nlsecure you write a 32 byte x25519 public key to the secure endpoint to set up the cipher contexts. After that, you write CoAP frames (header and all!) to the CoAP endpoint. IIRC the entire CoAP frame is encrypted, not just the payload. It’s a little weird.

I haven’t tried it yet but you should be able to send encrypted LTPDU commands (e.g., lb/0/oo) via the CoAP endpoint. Additionally, you should be able to set up Thread with the th/tc command. The format is documented in my git repo.

@Jc2k Thanks again for all your work here (and others as well). I have Thread working with my Nanoleaf bulbs, and now trying to see if I can get BLE working with some devices (have an Eve Door and Eve Energy to try it out with here).

Dumb question - is there anything else needed to get Bluetooth discovery to work?
I have edited the const.py file to set AIOHOMEKIT_TRANSPORT_BLE= True, but I’m not seeing any devices show up. I’m guessing maybe that isn’t the right way to do it? Where should I be setting the AIOHOMEKIT_TRANSPORT_BLE=1 variable?

I tried running bleak_lescan from the command line and it’s definitely finding a bunch of BLE devices.

Thanks again!

We have actually recently removed that flag, in dev (what will become beta later today) you just need to have the bluetooth integration configured. (I believe that integration will be active by default by beta, though i am not 100% sure).

@lambdafunction - did you get anywhere with the iOS provisioning method (https://github.com/Jc2k/aiohomekit/issues/81)

Interesting ok. I’m running dev, and just configured bluetooth and then restarted. Still not seeing any entries to add though for some reason. Should I be looking at logs for the Bluetooth integration? Or Homekit_Controller?

You can only pair a HomeKit device to one controller at a time - any chance they are still paired to iOS?

Okay so help me understand (newbie here) I have three nanoleaf essential light strips and want to build out more nanoleaf stuff…should I wait to integrate until thread and matter come out? will it be like a native integration into home assistant? or should I use this (gonna need some help lol) and get it integrated now…Running Home assistant on a Raspberry pi 3+ Thanks all!

I purchased 12x essential bulbs and 2x strips for my kitchen counter lights because nanolead said that their products would be updated to matter when that was released. They have since reverted their stance on this and will make you buy new bulbs.

I would shy away from expanding on the nanoleaf essentails platform until Matter is released and more platforms can update their current Zigbee items to Matter via firmware update.

Of course, this news was released right after my return window had passed.

1 Like

I tried it with a brand new Eve Door, so I don’t think so. I’m not seeing any Bluetooth devices at all though, so I kind of feel like something else is going on with my setup.

Turn on debug level logs for homeassistant.components.bluetooth first then and lets see what you get.

So I see a bunch of stuff like this, which makes me think it should work:

2022-07-27 18:42:21.020 DEBUG (MainThread) [homeassistant.components.bluetooth] Device detected: xx:xx:xx:xx:xx with advertisement_data: AdvertisementData(local_name=‘Eve Door 15C9’, manufacturer_data={76: b’\x061\x00X4UW\x04\xdc\n\x00\xa4\x0c\x02\x02/\xb1\n\xb9’}) matched domains: {‘homekit_controller’}

Fair point, I guess I meant more that I’m still trying to figure out how to do all that, I’m fairly new to tinkering with BLE, I appreciate the info nonetheless.
I’m a bit too busy lately to dig into it, and thankfully everything works using essentials as a border router, but if I find anything I’ll be sure to post it here.

As far as I understand, Essentials won’t support matter over thread directly, but will still be able to link to matter through a bridge device of some kind, their PR has just been terrible at actually explaining anything properly.

I’m guessing there’s not enough resources on their hardware to support both matter and homekit via thread, along with their own proprietary stuff, my speculation is they originally thought they could add a toggle (similar to the remote control/thread router toggle on the Shapes I have), but along the way realized that wasn’t going to be feasible, and having two different firmwares or hardware versions was going to cause a lot of issues and confusion with customers.

I also wonder if maybe Matter’s implementation of thread started to require more overhead as the project grew, which is part of why you don’t really see anyone doing what nanoleaf did and putting out thread devices promising support for later since the project has been evolving over time as more companies sign on, and nobody wants to develop their own thread communication protocol when homekit exists and matter is on the way.

That said @madisonjar, I still wouldn’t recommend buying more Essentials unless your system is already committed to homekit, or you’re comfortable tinkering. The integration here still needs a compatible homekit over thread bridge, I bought a cheap Shapes kit to do this, you can see their current list here https://helpdesk.nanoleaf.me/en-US/essentials-thread-and-border-routers-15563

Hopefully there’ll eventually be a solution implemented to provision onto thread via BLE, it seems feasible but it just need some work to figure out, assuming nanoleaf hasn’t put some kind of restrictions in place for provisioning.