HomeKit Accessory Protocol (HAP) over CoAP/UDP (was: Nanoleaf Essentials bulb via Thread/CoAP)

Tried to get this working and it seemed to have made no change, but I could see zeroconf actually seeing the strip, after some more digging I realized I never edited the generated zeroconf.py, and sure enough when I checked it was missing

    "_hap._udp.local.": [
        {
            "domain": "homekit_controller"
        }
    ],

Added that under _hap._tcp.local along with your config_flow.py change and that seems to have gotten things working again, and it seems to tolerate address changes without having to restart homeassistant (tested by power cycling the shapes and strip, then confirming it had a different ipv6 address)

I’ll modify my earlier post to add those steps.

@TheTofu @psumatt thanks for testing and the feedback. That change to config_flow.py should allow runtime detection of address changes via zeroconf updates. I’ll work to get it merged into @Jc2k’s branch.

1 Like

How the heck do you get Essentials bulbs to join the Thread network created by the Shapes Controller? I can add the shape to the Android app and it shows up as a border router, but the bulbs just refuse to connect via Thread. I assume they’re getting connected over BT right to the app, since they go offline when I turn off BT.

@Tol First thing I’d try is power cycling the bulbs (actually remove power, not just from the app), sometimes that got them to join thread.

Otherwise I’ve had two different methods work for me, one is to go into the app, open the shapes, then go into settings for the shapes, and switch remote mode on which should turn off the thread router.
Once this is remove power from everything, plug in the shapes, go back into the app and switch thread mode back on, then bring the bulbs on one at a time. This doesn’t bring them in 100% of the time, but usually after a while I’ll power cycle the bulbs and they’ll be in thread mode.

The other method is to just factory reset the shapes, app, and bulbs, remove power from everything, setup the shapes first and confirm it’s showing up as a thread border router, then add the bulbs one at a time.

Thankfully, once they’re actually setup in thread mode they seem pretty reliable, especially with a homekit connection, and they control much faster.

@lambdafunction After some time with everything in place, it seems that I do still need to restart HA Core if the shapes controller ever loses power, but the restart has so far always fixed it, just a hassle that I have to restart since I’ll randomly get my partner calling me saying the lights don’t work if there’s been a power outage at some point lol

Is there anything I can do to make it a little more seamless? At this point I’m considering just having a HA Restart shortcut added to her phone or something.

@TheTofu Hmm let me make sure I don’t have any other commits locally… Also make sure your edit stuck & didn’t get reverted after an update?

I just unplugged my Elements BR & within ~5 minutes the Essentials were back online:

Device address update from zeroconf: old=[fdc3:5f:3f7f:0:bdc1:27fc:e1f8:4807]:5683 new=[fd71:826a:ca3d:0:4689:4bef:e1c4:a5b3]:5683
Device address update from zeroconf: old=[fdc3:5f:3f7f:0:a6ca:42ef:852d:bbc2]:5683 new=[fd71:826a:ca3d:0:824e:1cf:ae38:e3ba]:5683
Connected to CoAP HAP accessory at [fd71:826a:ca3d:0:824e:1cf:ae38:e3ba]:5683!

Do you see any “Device address update from zeroconf” lines in your logs? You’ll need debug turned on for aiohomekit. Also, what HA version are you on? Thanks!

Yeah, I double checked this morning and the edit stuck (the update thing happened to me on friday lol) , I remember testing before and connection was restored if the controller was unplugged for just a minute or two (I even tested it just now and it seemed to reconnect fine after a few minutes), I might try unplugging for longer (20 minutes+) to see if that does anything different.

If it happens again I’ll see if I can check the logs for the address update, I’m currently on 2022.6.1, have held back on updating so I don’t have to reapply the edits constantly.

So small update, it seems like when it doesn’t reconnect zeroconf for whatever reason isn’t picking up the bulbs, but so far power cycling the shapes is all I need to do to get it working again instead of restarting HA.

Is there any way to send a reboot command to them?

Not that I know of though I haven’t really done anything with the other NL products. Cheap WiFi relay/outlet?

When https://github.com/home-assistant/core/pull/74857 lands, all of my outstanding homekit_controller changes have landed on dev. This will mean the dev and main branches of aiohomekit will work interchangeably with the dev branch of home assistant (what will become the august release). I.e. you won’t need to patch anything to do with home assistant, you’ll just need to update aiohomekit and possibly aiocoap. This is good - the 1.0.0 release of aiohomekit is massive, and being able to roll it back easily makes this all the safer.

I still need to look into Pass along IP updates from zeroconf by roysjosh · Pull Request #49 · Jc2k/home-assistant · GitHub. It doesn’t look like the TCP/IP backend needs this change. So i’d like to make sure of that, and if so make UDP do it the same way. (reconnect_soon is going away completely soon).

I still don’t have much free time but currently trying to finish cleaning up some Zeroconf changes in aiohomekit that are blocking 1.0.0. When those are done i’ll be merging the dev branch, releasing 1.0.0 and rolling it out with homekit_controller. Hopefully this will be in the August release. This means the code for CoAP and BLE will be shipping in a prod release without any git branches of home assistant or aiohomekit. Depending on how well testing goes, one or both of them will be behind feature flags (probably env variables).

Realistically CoAP will be behind a feature flag until the 3 issues with aiocoap referenced in https://github.com/Jc2k/aiohomekit/issues/68 are resolved and in a stable release of aiocoap (or we have been able to work around them in aiohomekit).

BLE is probably going to get the most attention with the little time I have this month - there is a big BLE push right now and exciting stuff happening elsewhere in the project (e.g. https://github.com/home-assistant/core/pull/74653) which i’m hoping to leverage. But it would also be good to verify that I haven’t completely broken CoAP without all the branch faff.

1 Like

1.0.0 is out, and the PR to land it in HA is almost over the line (https://github.com/home-assistant/core/pull/75198).

When that is there, the BLE and CoAP code are all present but disactivated. You can use the AIOHOMEKIT_TRANSPORT_BLE=1 and AIOHOMEKIT_TRANSPORT_COAP=1 env vars to turn them on. When we’ve ironed out all the big wrinkles we’ll get rid of those feature flags.

Right now BLE is hit and miss, it seems to work for the Eve devices I have access to but I can’t get it to work for my Nanoleaf light strip (which i was hoping to get working on thread next).

I haven’t had chance to check CoAP/Thread yet. I know that at the very least still need to look into: Pass along IP updates from zeroconf by roysjosh · Pull Request #49 · Jc2k/home-assistant · GitHub.

3 Likes

Thank you for all the work, excited to see things being merged into HA proper, I’ve been holding off on updating HA until fixes were merged.
Hopefully one day we can get thread commissioning working through ble, Nanoleaf also updated their android app quite a bit recently (I’m assuming ios app got updated as well) so I might revisit sniffing ble packets to see if I can figure out something with thread.

Would be interesting to see them update the desktop app with similar functions, but at least now the android app actually shows the thread network properly now.

@TheTofu the BLE comms are mostly known, including Thread provisioning.

The BLE service 6d2ae1c4-9aea-11ea-bb37-0242ac130002 contains the Android characteristics:

  • secure endpoint 6d2adfda-9aea-11ea-bb37-0242ac130002
  • CoAP endpoint 6d2adda0-9aea-11ea-bb37-0242ac130002

Similarly to /nlsecure you write a 32 byte x25519 public key to the secure endpoint to set up the cipher contexts. After that, you write CoAP frames (header and all!) to the CoAP endpoint. IIRC the entire CoAP frame is encrypted, not just the payload. It’s a little weird.

I haven’t tried it yet but you should be able to send encrypted LTPDU commands (e.g., lb/0/oo) via the CoAP endpoint. Additionally, you should be able to set up Thread with the th/tc command. The format is documented in my git repo.

@Jc2k Thanks again for all your work here (and others as well). I have Thread working with my Nanoleaf bulbs, and now trying to see if I can get BLE working with some devices (have an Eve Door and Eve Energy to try it out with here).

Dumb question - is there anything else needed to get Bluetooth discovery to work?
I have edited the const.py file to set AIOHOMEKIT_TRANSPORT_BLE= True, but I’m not seeing any devices show up. I’m guessing maybe that isn’t the right way to do it? Where should I be setting the AIOHOMEKIT_TRANSPORT_BLE=1 variable?

I tried running bleak_lescan from the command line and it’s definitely finding a bunch of BLE devices.

Thanks again!

We have actually recently removed that flag, in dev (what will become beta later today) you just need to have the bluetooth integration configured. (I believe that integration will be active by default by beta, though i am not 100% sure).

@lambdafunction - did you get anywhere with the iOS provisioning method (https://github.com/Jc2k/aiohomekit/issues/81)

Interesting ok. I’m running dev, and just configured bluetooth and then restarted. Still not seeing any entries to add though for some reason. Should I be looking at logs for the Bluetooth integration? Or Homekit_Controller?

You can only pair a HomeKit device to one controller at a time - any chance they are still paired to iOS?

Okay so help me understand (newbie here) I have three nanoleaf essential light strips and want to build out more nanoleaf stuff…should I wait to integrate until thread and matter come out? will it be like a native integration into home assistant? or should I use this (gonna need some help lol) and get it integrated now…Running Home assistant on a Raspberry pi 3+ Thanks all!

I purchased 12x essential bulbs and 2x strips for my kitchen counter lights because nanolead said that their products would be updated to matter when that was released. They have since reverted their stance on this and will make you buy new bulbs.

I would shy away from expanding on the nanoleaf essentails platform until Matter is released and more platforms can update their current Zigbee items to Matter via firmware update.

Of course, this news was released right after my return window had passed.

1 Like

I tried it with a brand new Eve Door, so I don’t think so. I’m not seeing any Bluetooth devices at all though, so I kind of feel like something else is going on with my setup.