Hey. I’m using Elements as border router with 3 Essentials bulbs connected to it over Thread (via Android app). What do I need to do to add it to HA?
@spanky13 you need to find a way to add these environment variables to your HA which will depend on how you are running things:
export AIOHOMEKIT_TRANSPORT_BLE=1
export AIOHOMEKIT_TRANSPORT_COAP=1
After that, you’ll see HomeKit devices that are available over BLE or CoAP. You’ll need the 8 digit code on the bulbs to pair. Make sure to put the code in how HomeKit requires, not how it is printed on the bulb. Let us know how it goes!
Thank you for the quick response. I can’t seem to get these feature flags to stick after restarts. Will continue researching about that.
By the way, the process you described is for connecting the Essentials bulbs (bluetooth only) to HA via bluetooth, is that right? Adding them to HA over Thread is still a work in progress?
@spanky13 how are you running HA?
CoAP is the Thread support. Add those env variables and you’ll be able to find unpaired accessories over BLE or Thread (CoAP).
Think of a 7-layer cake: the Border Router sees a cake coming out of the kitchen, chops off the top 4 layers, puts them on a new 3-layer cake bottom, and finally smooths it over to look good again. That’s what the BR does when it translates (HAP over CoAP over UDP over IPv6 over) WiFi to Thread.
Here’s a screengrab of what the WiFi network traffic and Thread network traffic looks like. You can see they are both doing a CoAP PUT to /nlsecure with the 0101 TLV + 0020 bytes of data:
WiFi:
Thread:
Gotcha! I’d love to give it a shot and report back if I observe anything weird. I’m running HA OS on a Raspberry Pi.
I couldn’t figure out how to pass environment variables on my HA OS so I just hacked the code. If you’re familiar with vi/vim, give this a try.
Enter the homeassistant container from HA OS:
docker exec -it homeassistant sh
And then let’s edit some Python!
vi /usr/local/lib/python3.10/site-packages/aiohomekit/const.py
and edit line 36 (OLD)
if "AIOHOMEKIT_TRANSPORT_COAP" in os.environ:
to read (NEW, spacing and capitalization are important):
if True:
You’ll also have to edit aiocoap to fix an issue they haven’t merged a PR for yet:
vi /usr/local/lib/python3.10/site-packages/aiocoap/transports/ws.py
and edit line 187 (OLD):
else:
to read (NEW):
elif port != 0:
Finally, restart HA from the web interface. You should see Essentials as new devices!
1 Like
@lambdafunction Thanks. I applied the code hacks but somehow the bulbs still don’t show up in HA. I manually looked for them in HomeKit Controller integration too but only the Elements panel is there. Are there any logs that I can share to help diagnose the problem?
Have you already added them to a homekit system, google nest, or ios app? If so that’ll take up the one admin slot the essentials have and you’ll have to factory reset them.
Currently the best way to acquire them is to use the android app, which apparently also has a better thread network viewer.
Edit:
Updated HA finally and made the changes lambda outlined, seems to work fine for me though I already had my essentials acquired to HA previously.
I had them previously added to the Nanoleaf Android app only. Just to be sure, I did a factory reset on the bulbs, added them again to the Nanoleaf app, then looked for them in HA. Still no dice.
Here’s my HA setup:
- HA OS on raspi
- Nanoleaf Elements added to HA using the Nanoleaf integration (is this correct?)
- Nanoleaf Essentials connected to Thread network created by the Elements border router (using Android app)
@spanky13 first, start with an mDNS service browser. On Android I use one called Service Browser by Andriy Druk on the Play Store. It’s OSS and seems to work fine. Look for _hap._udp. (HAP) services; there should be one for each bulb. Additionally, if you tap into a bulb you will see the mDNS properties; ensure that sf is 1 (pairable). On Mac I use dns-sd -Z _hap._udp to find the bulbs and on Linux I use avahi-browse -r _hap._udp.
You can also try to ping the bulbs.
ping6 Nanoleaf-A19-12FA.local
OR
ping -6 Nanoleaf-A19-12FA.local
If that looks good, go call the service “Logger: Set level” from the Developer Tools:
service: logger.set_level
data:
aiohomekit: debug
homeassistant.components.homekit_controller: debug
zeroconf: debug
And let that sit for a bit to see if HA detects any HAP devices. Maybe turn a bulb off, wait a few seconds, then turn it back on with the logs at debug. Bulbs should trigger the BR to broadcast an mDNS service on startup.
1 Like
I need to call a freind who owns an android over to my house haha! All this development has me itching to try this.
You’re killing it lambda!
If sf is not set to 1, what are the next steps? Do I need to delete the bulbs from HomeKit in order for them to start showing up in Logger or at the very least having an sf value of 1?
@Leiesoldat sf=0 means that the accessory is paired to something. You need to either delete it from whatever “Home” it is paired to or factory reset it. Both should work.
@lambdafunction sf was indeed 0 in my bulbs. So I did a factory reset and did not add them to the Nanoleaf app anymore. HA was then able to find the bulb and I am now able to control it from there.
Some early feedback on the integration:
- There’s a bit of a delay (1-2 seconds) the first time you control the bulb (either turn it on or off). Subsequent commands after that gets instant response. Then it’ll be slow again after several minutes of not issuing commands.
- If I turn off the physical switch connected to the bulb, HA does not recognize that as an unreachable device.
At this point, I’m just glad that we now have the ability to control the bulbs in HA. The native Nanoleaf app is just too finicky to be considered useful 
Hats off to everyone who contributed to this @Jc2k @lambdafunction @TheTofu
@spanky13 glad to hear you got it working! It sounds like you may be connected over BLE? You can check from the HA console:
jq '.data.entries[] | select(.domain == "homekit_controller") | select(.data.Connection == "BLE")' /mnt/data/supervisor/homeassistant/.storage/core.config_entries
Since you’ve got Android, you should be able to add the bulbs to the app again to get them on Thread. Or wait for the HA service, but that will be 2022.9 at the earliest.
Hmm, having various issues since I upgraded to HA 2022.8.3, some of which are homekit related.
Seems like occasionally the bulbs will lose connection and I’m not sure why, I may also just have some kind of network DNS issue, not sure if that would cause what I’m seeing:
Logger: homeassistant
Source: components/homekit_controller/config_flow.py:288
First occurred: 7:01:53 AM (148 occurrences)
Last logged: 10:55:12 PM
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/discovery_flow.py", line 74, in _async_process_pending_flows
await gather_with_concurrency(
File "/usr/src/homeassistant/homeassistant/util/async_.py", line 199, in gather_with_concurrency
return await gather(
File "/usr/src/homeassistant/homeassistant/util/async_.py", line 197, in sem_task
return await task
File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 222, in async_init
flow, result = await task
File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 249, in _async_init
result = await self._async_handle_step(flow, flow.init_step, data, init_done)
File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 359, in _async_handle_step
result: FlowResult = await getattr(flow, method)(user_input)
File "/usr/src/homeassistant/homeassistant/components/homekit_controller/config_flow.py", line 288, in async_step_zeroconf
await conn.pairing.reconnect_soon()
File "/usr/local/lib/python3.10/site-packages/aiohomekit/controller/coap/pairing.py", line 228, in reconnect_soon
address = self.pairing_data["AccessoryAddress"]
KeyError: 'AccessoryAddress'
Logger: aiohomekit.controller.coap.pdu
Source: components/homekit_controller/connection.py:629
First occurred: 9:43:42 AM (19 occurrences)
Last logged: 10:48:44 PM
Transaction 0 failed with error 6 (Invalid request)
Logger: aiohomekit.controller.coap.connection
Source: components/homekit_controller/connection.py:629
First occurred: 7:47:41 AM (91 occurrences)
Last logged: 10:55:42 PM
CoAP POST returned unexpected code <aiocoap.Message at 0x7fdf6f1ea350: Type.ACK 4.04 Not Found (MID 39198, token a8d5) remote <UDP6EndpointAddress [fdee:b12:3711:0:a015:26c6:b359:f13c] (locally 2603:8000:f101:2930:2459:9c09:ceb8:59cb%ens18)>>
CoAP POST returned unexpected code <aiocoap.Message at 0x7fdf6e732c50: Type.ACK 4.04 Not Found (MID 25189, token 4fda) remote <UDP6EndpointAddress [fdee:b12:3711:0:c65b:4519:b08e:d483] (locally 2603:8000:f101:2930:2459:9c09:ceb8:59cb%ens18)>>
Decryption failed, desynchronized? Counter=11/14
Decryption failed, desynchronized? Counter=12/14
Decryption failed, desynchronized? Counter=14/16
Slightly off-topic, but if you do have Nanoleaf gear connected via HomeKit BLE, there is a bunch of work in flight that will make things much faster over the coming months.
- Looks like HA OS 9 will switch dbus implementations to https://github.com/bus1/dbus-broker. Early testing shows this is twice as fast at reading. https://github.com/home-assistant/operating-system/pull/2053
- There are more changes here: https://github.com/altdesktop/python-dbus-next/pull/126.
- And here: https://github.com/hbldh/bleak/pull/923.
All of these changes together have a huge impact on HomeKit BLE performance.
1 Like
@TheTofu - at least the first one looks like my bad and probably caused by: https://github.com/Jc2k/aiohomekit/pull/133. Probably stick on .2 for now if you can.
I’m having trouble pairing the Essential bulb.
The environment:
- RPI 4 running on DietPi
- HA container: v2022.8.3
- both code hacks in the python scripts are done
- Essential bulb is paired to the BR (NL Shapes), working in NL android app, and available over Thread
- Essential bulb can be seen in Service Browser (android app)
- Essential bulb is auto-discovered (after turning it off and on after 5-10 secs) by HA which ask to integrate it through Homekit Controller
When trying to integrate it, I have some Timeout error every time (tried many times, turning off/on the bulb, reset etc).
Error log:
2022-08-11 09:34:02.634 WARNING (MainThread) [aiohomekit.controller.coap.connection] Pair setup 1/2 failed!
2022-08-11 09:34:02.642 ERROR (MainThread) [homeassistant.components.homekit_controller.config_flow] Pairing attempt failed with an unhandled exception
Traceback (most recent call last):
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 456, in wait_for
return fut.result()
asyncio.exceptions.CancelledError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/homekit_controller/config_flow.py", line 502, in async_step_pair
self.finish_pairing = await discovery.async_start_pairing(self.hkid)
File "/usr/local/lib/python3.10/site-packages/aiohomekit/controller/coap/discovery.py", line 57, in async_start_pairing
salt, srpB = await self.connection.do_pair_setup(
File "/usr/local/lib/python3.10/site-packages/aiohomekit/controller/coap/connection.py", line 291, in do_pair_setup
response = await asyncio.wait_for(
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 458, in wait_for
raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError
Any idea on this ?
I could try to make it work over BLE only to test, but I dont know how to force this, as it’s over CoAP right now.
@TheTofu the last one is the “normal” series of errors when comms get messed up. Either the bulb loses our session or some traffic is dropped/repeated causing the encryption context to get out of sync. I try to recover in the code but in general I’ve found that if you see the first error (CoAP POST returned … 4.04) that the bulb has lost the session. In the future I hope to make this more seamless to the frontend but for now the bulb toggles to “unavailable” for a minute.