HomeKit Accessory Protocol (HAP) over CoAP/UDP (was: Nanoleaf Essentials bulb via Thread/CoAP)

Ah, sorry was having a bit of a mental rambling and wasn’t sure if BLE would use the same commands, which if it does then that makes life a good bit easier.

As far as aiohomekit I’m just not sure how exactly to use it so I was trying to find something already decently well developed just to get BLE working and then work on the thread/coap side.

One thing I’m unsure of is does BLE use coap? From everything I can find it doesn’t, but not sure if home kit is doing some extra stuff to encapsulate it or something.

Before Thread provisioning, the bulbs do HAP over BLE if you’ve got an Apple device. I don’t know what Android does over BLE yet but there are 2 non-HAP BLE services available.

After Thread provisioning, the bulbs do HAP over CoAP for Apple and a custom Nanoleaf protocol over CoAP for Android (LTPDU, the initial scope of this feature request). The Border Router translates the link-layer protocol from WiFi to Thread and back.

HAP defines the entire protocol including communication. It just so happens that the data frames are almost identical over BLE and Thread (HAP PDUs). The data frames look completely different for (non-Thread) WiFi devices.

Here’s an example of a single request as seen on the Thread network:

image

Everything from Internet Protocol Version 6 down is more or less copied as-is from my Home Assistant dev instance. The Data bytes are encrypted HAP over CoAP over UDP over IPv6 over Thread (6LoWPAN/802.15.4).

If you had a dump of the BLE communication, it would just be the Data portion on top of BLE signaling.

Finally got my Nanoleaf border router, seemed to take forever to get the latest firmware (seemed to go through many point releases instead of going straight to the latest).

Hope to get back to finishing off the dev branch in aiohomekit, but work is hectic atm so i’ve just been too done in to pick anything up after work.

The HA release today has the bulk of the changes to make the dev branch work with HA. Still a couple of things I need to rework.

After that the remaining work is test coverage for aiohomekit, documentation about what we know about getting a mesh up and running, and testing for regressions,

Some interesting news, apparently Nanoleaf’s current essentials line isn’t going to support matter, which is quite frustrating as that’s half the reason I bought these bulbs :
Hopefully they’ll at least make it easier to add the bulbs onto any thread network, rather than just the few specific devices they support now.
I haven’t had any luck trying to get the damn things to provision over BLE, debating whether to get a Nanoleaf controller off ebay or something just for the border router.

https://www.theverge.com/2022/3/24/22994597/matter-delay-nanoleaf-essentials-eve-wemo

While there may be more certainty in the industry, current smart home owners are still in a quagmire of uncertainty around which products to buy and when. Especially now that Chu has confirmed to The Verge that Nanoleaf’s Thread-enabled A19 smart light bulb and light strip won’t work with the standard. “[They] will require a different chip,” says Chu. “We hope to launch a new Essentials A19 and a new light strip when Matter arrives this year.” This is a blow to owners who bought the products thinking that being Thread-enabled meant they would be upgradable to Matter. The devices will continue to work with HomeKit over Thread, however, and with other platforms either over Bluetooth or over Thread using a cloud-to-cloud integration, says Chu.

I lost motivation to do any more tinkering on the BLE side of things since I don’t really know what I’m doing, and ended up buying a cheap shapes kit on ebay. I was able to get everything configured and working on thread, but then had to spend a bit more time to get everything working in home assistant as well.
I currently have 1 shapes controller, 1 essentials strip and 3 essentials bulbs working well and responding fairly quickly, will update if I find any issues down the line.

Current steps I had to take:

  1. Pair all devices to nanoleaf android app, make sure firmware is up to date, then factory reset everything and remove from app. Pair just the shapes controller and make sure it’s showing up in thread in the app, it may take a few restarts of both the controller and the app to get it to actually work.

  2. Pair any essentials strips or bulbs to the app, and confirm they show up in thread, and then power them off (might not need to power them off but I only paired one at a time to home assistant just in case)

  3. Add the following repository to the addon store in home assistant https://developers.home-assistant.io/ and install the Custom Deps Deployment addon, do not start it yet. Go to the config page and add the following and hit save:

pypi:
  - git+https://github.com/Jc2k/aiohomekit.git@dev
  - git+https://github.com/roysjosh/aiocoap.git@fix-269
apk: []
  1. Start the addon and look at the log tab to make sure everything was successful. Make sure you’re not using the built in nanoleaf integration, then restart home assistant from settings.

  2. Edit the config_flow.py in homekit_controller and the generated zeroconf.py in HA Core.
    You can do this by running the following

find / -name config_flow.py | grep homekit_controller
find / -name zeroconf.py | grep generated

If you’re running supervised then you can search in /var/lib/docker/overlay2, I think only the overlay with merged needs to be edited but I modified any instance I could find just in case.

You need to modify the following:
config_flow.py
await conn.pairing.connection.reconnect_soon()
change to
await conn.pairing.connection.reconnect_soon(updated_ip_port)

zeroconf.py
add under _hap._tcp.local

    "_hap._udp.local.": [
        {
            "domain": "homekit_controller"
        }
    ],

Then restart

  1. Once everything is back online, go to your integrations, add homekit controller, select your controller, pair as normal, then follow with the essentials devices 1 at a time until all are paired.
    With this hopefully you’ll have everything linked and working.

Notes:
It took me a while to understand what Custom Deps Deployment actually did, as before that I kept trying to copy homekit_controller from various branches to my custom_components folder and running into issues.
Eventually I thought I solved it by specifying lambda’s fork of aiocoap, which worked to get a single essentials light paired, but kept running into the binding issue when trying to pair others, at which point I realized I needed the specific fix branch, after which everything started working properly.
Device discovery worked at first, but after the first few restarts it stopped notifying me, possibly because I was waiting until I was already at the integrations page before turning on my bulbs to pair.

Hopefully the aiocoap issue gets resolved soon so the everything can be merged into core finally.
Thanks to both @lambdafunction and @Jc2k for their work on this!

Edit: Added a couple extra steps to get homekit_controller to update if the border router assigns a new address.

1 Like

Bad news, it seems like power cycling the thread border router, in this case a shapes kit (may be nanoleaf specific issue), causes any paired thread devices to timeout on verifying pairing.
The shapes connection still works, I did notice that the device I tested with (an essentials strip) comes up with a different IPv6 address when it reconnects to thread after the shapes are power cycled, not sure if that makes any difference. Added log output below.

Logger: homeassistant
Source: deps/lib/python3.9/site-packages/aiohomekit/controller/coap/pairing.py:73
First occurred: 8:09:11 PM (3 occurrences)
Last logged: 8:09:54 PM

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 492, in wait_for
    fut.result()
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/connection.py", line 394, in connect
    await self.do_pair_verify(pairing_data)
  File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/connection.py", line 353, in do_pair_verify
    response = await asyncio.wait_for(
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 494, in wait_for
    raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/pairing.py", line 71, in _ensure_connected
    await self.connection_future
  File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/connection.py", line 397, in connect
    raise AccessoryDisconnectedError("Pair verify timed out")
aiohomekit.exceptions.AccessoryDisconnectedError: Pair verify timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/pairing.py", line 139, in subscribe
    await self._ensure_connected()
  File "/config/deps/lib/python3.9/site-packages/aiohomekit/controller/coap/pairing.py", line 73, in _ensure_connected
    raise AccessoryDisconnectedError("failed to connect")
aiohomekit.exceptions.AccessoryDisconnectedError: failed to connect

Update: I added the lines to my config but was looking at the clickable logs not the full log, it seems like the IPv6 address changing is the issue after all

2022-05-17 20:19:46 DEBUG (MainThread) [homeassistant.components.homekit_controller.connection] Starting HomeKit controller update: AE:2A:98:FF:FF:FF
2022-05-17 20:19:46 DEBUG (MainThread) [aiohomekit.controller.coap.connection] Pair verify uri=coap://[fd03:9201:7cb1:0:e150:20c3:e14e:d375]:5683/2
2022-05-17 20:19:46 DEBUG (MainThread) [aiohomekit.protocol.tlv] sending [
2022-05-17 20:19:54 WARNING (MainThread) [aiohomekit.controller.coap.connection] Pair verify timed out

It’s still trying to connect to the old IPv6 address, I can open a ticket in github as well, would this be aiohomekit or aiocoap?

@TheTofu I’m 99% sure I have a local change that tracks zeroconf address updates, it just isn’t committed or pushed. A restart of HA should get you going in the meantime. IIRC it took a change to both the HA homekit_controller component as well as aiohomekit. I’ll push that code soon.

I’ve tried a few restarts and it didn’t do anything, though I also haven’t had any new integration/device notifications in a while so I wonder if something’s wrong with my zeroconf or it just doesn’t like being in a proxmox VM? Setting the logs to debug it seems like it’s working fine though…

Hey, please try to make this change locally:

diff --git a/homeassistant/components/homekit_controller/config_flow.py b/homeassistant/components/homekit_controller/config_flow.py
index 979d47d9cd..9d9e5557ff 100644
--- a/homeassistant/components/homekit_controller/config_flow.py
+++ b/homeassistant/components/homekit_controller/config_flow.py
@@ -247,7 +248,7 @@ class HomekitControllerFlowHandler(config_entries.ConfigFlow, domain=DOMAIN):
             # will do nothing if the device is already connected
             # Note that not all backends have this
             if hasattr(conn.pairing, "connection"):
-                await conn.pairing.connection.reconnect_soon()
+                await conn.pairing.connection.reconnect_soon(updated_ip_port)
             if conn.config_num != config_num:
                 _LOGGER.debug(
                     "HomeKit info %s: c# incremented, refreshing entities", hkid

You may have to find / -name config_flow.py | grep homekit_controller unless you know where it is on your HA install already.

@TheTofu you could also set these loggers to debug in your configuration.yaml:

logger:
  default: info
  logs:
    homeassistant.components.zeroconf: debug
    zeroconf: debug

@lambdafunction I am having similar issues with the bulbs randomly seeming disconnected. A few times this has happened because I rebooted the Nanoleaf shapes unit and other times it just seems random and a reboot fixed it.

I was able to manually edit using the new ipv6 addresses and it starts working again. I tried changing the file you mentioned and it didn’t make a difference. I’ll try turning on the debug functions you mentioned and report back when it happens.

@psumatt in that case please also enable aiohomekit: debug and homeassistant.components.homekit_controller: debug

1 Like

Ok, so I enabled the debug logs, but there are a ton of them, and not sure what to look for here. That said, it seems your change did work, upon reboot. I had tried it a few times and it wasn’t working, but then I went in yesterday and searched for that file again. It seems there were two copies of the homeassistant folder - I had modified one of them but not the other (I guess I chose wrong?).

So, @lambdafunction - is it expected that it would pick up the new IP address automatically or only after a restart? Nanoleaf as a thread router seems to generate a new IP address for the individual nodes anytime it disconnects from the router or is restsarted.

Tried to get this working and it seemed to have made no change, but I could see zeroconf actually seeing the strip, after some more digging I realized I never edited the generated zeroconf.py, and sure enough when I checked it was missing

    "_hap._udp.local.": [
        {
            "domain": "homekit_controller"
        }
    ],

Added that under _hap._tcp.local along with your config_flow.py change and that seems to have gotten things working again, and it seems to tolerate address changes without having to restart homeassistant (tested by power cycling the shapes and strip, then confirming it had a different ipv6 address)

I’ll modify my earlier post to add those steps.

@TheTofu @psumatt thanks for testing and the feedback. That change to config_flow.py should allow runtime detection of address changes via zeroconf updates. I’ll work to get it merged into @Jc2k’s branch.

1 Like

How the heck do you get Essentials bulbs to join the Thread network created by the Shapes Controller? I can add the shape to the Android app and it shows up as a border router, but the bulbs just refuse to connect via Thread. I assume they’re getting connected over BT right to the app, since they go offline when I turn off BT.

@Tol First thing I’d try is power cycling the bulbs (actually remove power, not just from the app), sometimes that got them to join thread.

Otherwise I’ve had two different methods work for me, one is to go into the app, open the shapes, then go into settings for the shapes, and switch remote mode on which should turn off the thread router.
Once this is remove power from everything, plug in the shapes, go back into the app and switch thread mode back on, then bring the bulbs on one at a time. This doesn’t bring them in 100% of the time, but usually after a while I’ll power cycle the bulbs and they’ll be in thread mode.

The other method is to just factory reset the shapes, app, and bulbs, remove power from everything, setup the shapes first and confirm it’s showing up as a thread border router, then add the bulbs one at a time.

Thankfully, once they’re actually setup in thread mode they seem pretty reliable, especially with a homekit connection, and they control much faster.

@lambdafunction After some time with everything in place, it seems that I do still need to restart HA Core if the shapes controller ever loses power, but the restart has so far always fixed it, just a hassle that I have to restart since I’ll randomly get my partner calling me saying the lights don’t work if there’s been a power outage at some point lol

Is there anything I can do to make it a little more seamless? At this point I’m considering just having a HA Restart shortcut added to her phone or something.

@TheTofu Hmm let me make sure I don’t have any other commits locally… Also make sure your edit stuck & didn’t get reverted after an update?

I just unplugged my Elements BR & within ~5 minutes the Essentials were back online:

Device address update from zeroconf: old=[fdc3:5f:3f7f:0:bdc1:27fc:e1f8:4807]:5683 new=[fd71:826a:ca3d:0:4689:4bef:e1c4:a5b3]:5683
Device address update from zeroconf: old=[fdc3:5f:3f7f:0:a6ca:42ef:852d:bbc2]:5683 new=[fd71:826a:ca3d:0:824e:1cf:ae38:e3ba]:5683
Connected to CoAP HAP accessory at [fd71:826a:ca3d:0:824e:1cf:ae38:e3ba]:5683!

Do you see any “Device address update from zeroconf” lines in your logs? You’ll need debug turned on for aiohomekit. Also, what HA version are you on? Thanks!

Yeah, I double checked this morning and the edit stuck (the update thing happened to me on friday lol) , I remember testing before and connection was restored if the controller was unplugged for just a minute or two (I even tested it just now and it seemed to reconnect fine after a few minutes), I might try unplugging for longer (20 minutes+) to see if that does anything different.

If it happens again I’ll see if I can check the logs for the address update, I’m currently on 2022.6.1, have held back on updating so I don’t have to reapply the edits constantly.