Here’s my entity-map. Running ha supervised with host networking mode, VLAN disabled.
I’ve found how the iOS Nanoleaf app lists and executes scenes. Using the vendor-specific characteristic 0xA28E1902CFA14D37A10F0071CEEEEEBD, the app writes “Nanoleaf” TLVs (big-endian, two byte tags and lengths) wrapped in HAP Value TLV8s. TLV 0x0703 lists the current scene identifiers by returning a list of bytes. TLV 0x0704 gets a scene’s details (???) by passing the identifier byte as a 1-byte argument in the value. TLV 0x0706 executes a scene, again via the 1-byte identifier.
The return from 0x0704 needs more analysis but here’s the raw values (slightly decoded as TLV8) given a certain identifier:
Northern Lights (7 colors)
0xFB: 01.05=fb01140001
02.16=0738f2632db2641f725d0fb26407f26400b2644cf264
Blaze (3 colors)
0xFC: 01.04=fc021400
02.0a=03092e5c072eca077234
Date Night (4 colors)
0xFD: 01.04=fd02180a
02.0d=0402a362035e64002d3602a84a
Starlight (4 colors)
0xFE: 01.05=fe03180f50
02.0d=04382d38359ac30e5e510bea53
I’ve also added the readable name as determined by what the app sent via 0x0706 when I tapped a scene.
Wonderful, thank you @Rambooo.
Good news about the scenes @lambdafunction. Will start thinking about how to support this in homekit_controller. Right now all the infrastructure is around mapping a service or characteristic to an entity.
Ideally we’d create a home assistant scene for each nanoleaf scene (i think?). But that would mean multiple entities for the same characteristic. Right now for a characteristic entity there is a unique id based on serial, aid, service iid, and characteristic iid so you can’t do that with any of the helpers we already have.
We need to figure out when/how the list of scenes is refreshed. Right now all the entities assume they just read from a char (or subscribe to it). Do we get an EV if the scenes change? Or do we need to poll. There isn’t really a hook to do an RPC when creating entities right now, as so far we’ve been able to get everything from the entity map data.
Something for after COAP lands in aiohomekit!
@lambdafunction @Rambooo
Sorry to bother you again, but i cant seem to get this working at all.
I tried putting a debugger on the aiohomekit
2022-02-02 07:57:37 DEBUG (MainThread) [aiohomekit.zeroconf] candidate data {b'id': b'17:B0:27:C3:00:E0', b'md': b'NL42', b'ff': b'1', b'c#': b'3', b's#': b'1', b'ci': b'5', b'pv': b'1.1', b'sh': b'jxHieg==', b'sf': b'0'}
2022-02-02 07:57:37 DEBUG (MainThread) [aiohomekit.controller.controller] Discovered IP devices: [IPDiscovery(host=192.168.1.62, port=6517)]
I’ve added to components from the git
and changed their manifest with a version to all of them:
added the custom deps, installed it and restarted:
The last thing im missing is the generated/zeroconf since i cannot find the right location when using my setup (Raspberry pi with HA OS).
Also i managed to get mDNS to work but no luck:
Nanoleaf recognized my shapes but not my essentials.
I also found the scrapped android phone and i can see the thread connection in the nanoleaf app which also seems to response pretty quick
Do any of you have any good ideas i can try to test out ?
Wearing my “guy that gets the support tickets on GitHub” hat, my advice at this stage is to wait a bit longer. This stuff will be in the main branch as quick as possible. But right now the goal posts are moving a lot, for example the tlv8 dependency isn’t even needed any more and its only been a few days. I’m reluctant to encourage people to run homekit_controller as a custom component at the best of times - its a you break it you keep the pieces situation.
Ideally we can get the changes to zeroconf upstream sooner rather than later and that should help streamline this process - then we’ll only need a patched aiohomekit and homekit_controller.
I’m currently going through @lambdafunction’s branch as quick as I can and we are getting it in piece by piece adding tests as we go. There’s still a lot to do, but we are getting there.
Now with my support hat on, if you can’t see the device in netdisco theres a couple things that could be happening:
-
netdiscois kinda deprecated. Support for udp homekit is new and isn’t via netdisco, so it might not have been added to netdisco. It’s possible netdisco just isn’t looking for your bulb. Up above they were usingavahi-browse -ato look for these bulbs. - Or if it does support udp homekit then the problem is at this stage with your network rather than any of the custom components. We’d need to know what your network looks like. Are there vlans? What make is your wifi? Your router? How do you run home assistant? In a container? Host networking or bridged networking? There are settings on UniFi that are known to break HAP mDNS. Getting this to work with iOS and then looking at the thread status in the Eve app would prove that all the HomeKit protocol is working between your Nanoleaf and your iOS device over your WiFi, but it could still be a network issue on your HA box.
I’d start by finding another mDNS tool and seeing if that can see the bulbs.
1 Like
@TRoager make sure you remove the "NL*" from the Nanoleaf component. And the zeroconf changes are quite important otherwise the bulbs won’t be detected. Something like find /path/to/ha -name zeroconf.py | grep generated should help locate the right file.
Okay, a little more progress on scenes… I created a scene with a single color (RGB #5500FF or Hue/Sat/Brightness 260/100/100). I chose the color to find patterns in the encoded bits (if encoding is RGB, 5 is 0b0101 and F is 0b1111; if encoding is HSB, 260 is 0b1_0000_0100 and 100 is 0b0110_0100). The NL app sent 0x0701 with a payload of 01050001180001020401413264 during scene creation, probably to preview it on the bulb.
01050001180001020401413264 is tlv8 payload
01.05=0001180001 (probably the fade/highlight/random settings)
02.04=01413264 (first byte is number of colors, then 3 bytes/color)
413264 hex -> binary 010000010011001001100100
010000010011001001100100
... no 0b1111 or 0b0101
... but 260 exists and 100 exists twice!
0 1_0000_0100 110_0100 110_0100
bit0=?, 9 bits for hue, 7 for saturation, 7 for brightness!
Let’s double-check with the builtin scenes…
Blaze
03 092e5c 072eca 077234 (3 colors + 3 bytes per color)
092e5c -> 0 000100100 1011100 1011100 (hue:36 sat:92 brightness:92)
072eca -> 0 000011100 1011101 1001010 (hue:28 sat:93 brightness:74)
077234 -> 0 000011101 1100100 0110100 (hue:29 sat:100 brightness:52)
And plugging those into colorizer.org gives:
Looks good!
Note that the NL* change itself should be optional, if its not there is a bug. Unfortunately its policy that a “Native” integration gets dibbs on the discovery, which is why that line exists in the first place (unless the nanoleaf integration is false in saying it handles NL* devices, if there are some NL devices it does support and others it doesnt we need a better matching rule).
There is an exception for this rule in the latest release where the native integration is cloud based to encourage people to use local control. I am also pushing for an exception when the native integration doesn’t support push events like homekit_controller.
The NL* line shouldn’t stop it from being discoverd if you go on “Add integration”, choose “homekit controller” and then look in the devices drop down. If it doesn’t show up there without the NL* change we have a bug.
Hmm we need a way to keep Essentials bulbs and Light Strips from going to the native Nanoleaf component as it won’t be able to talk to them at all. Worst case the models could be listed individually. I believe the bulbs do show up following your instructions as someone in this topic did just that. I’ll double-check later.
@Jc2k Sorry I wasn’t meaning to rush anyone here, I just got so excited for discovering that there were some development and seeing that it actually were doable I wanted to try it out and if anything help with debug or provide feedback.
I will wait on the release for now since I’m in no rush to use the bulb’s specifically.
But what I am interested in is the Thread communication with Eve Motionblinds and I’m guessing that they might share something.
I just pre-ordered these blinds and they are running on thread.
@TRoager It’s all good, I didn’t mean to admonish anyone, i’m excited too. I have quite a few Eve products and I understand that Thread makes them much more responsive so keen to see if we can provision mine onto a Nanoleaf border router some day.
1 Like
Hello everyone…
I found your discussion here, but didnt have read all of the 72 updates 
So you’ve found a way to integrate Thread Devices (Nanoleaf Essentials in my and your case) into HA?
If its possible for me to integrate it to HA, I will use it as main controller, i mean Thread is the future and if HA doesnt support it I dond know what system to use
Thanks
@thepi - We are currently in the process of extending the homekit_controller integration to support HomeKit’s flavour of Thread.
@lambdafunction has done all the hard work of figuring out the protocol and writing the PR’s against aiohomekit/homekit_controller. I’m the maintainer of both of those and we are integrating his work right now. It’s going to take a little while to get it into a HA release, though, but it is going to happen.
What we don’t yet have is a way to provision Thread devices in a generic way. So this thread talks about using the Android Nanoleaf app to set up a mesh of Nanoleaf devices, then homekit_controller can see your Nanoleaf essentials devices using Bonjour/Zeroconf much like it does ordinary HomeKit devices. But we can’t (yet) provision a Eve thread device onto that mesh.
That said, right now unless you like messing around with git checkouts, you should hang on for us to get it into a release.
2 Likes
Lol, I have a PR to do just that waiting to submit. Great minds or something.
Latest official dev core caused some errors and loss of connection on my setup, so I updated the custom components and dependencies to their current versions.
Everything seems to be working well again but I’m getting these two previously unseen warnings on logger:
Logger: aiohomekit.controller.coap.pdu
Source: deps/lib/python3.9/site-packages/aiohomekit/controller/coap/pdu.py:95
First occurred: 1:56:54 PM (3 occurrences)
Last logged: 1:56:57 PM
Transaction 0 failed with error 6
Logger: aiohomekit.controller.coap.connection
Source: deps/lib/python3.9/site-packages/aiohomekit/controller/coap/connection.py:463
First occurred: 1:56:54 PM (3 occurrences)
Last logged: 1:56:57 PM
Failed to read 55.50 iid 37
Is this something expected or do you need more information? Reporting just in case. @lambdafunction @Jc2k
Right now everything is in a bit of flux.
On my side there is:
- https://github.com/Jc2k/aiohomekit/tree/dev (note: dev and not main)
- https://github.com/Jc2k/home-assistant/tree/homekit_controller_coap_ble (should work with dev aiohomekit, but hasnt been rebased in a few merges so might not work as a custom component)
I’m keeping those in sync but right now can only test BLE and the existing IP backend. Errors in those need reporting as they are what will be released into HA core eventually.
In terms of dev, things landing there won’t be compatible with the original branches that you used. Indeed, I am currently cutting up my homekit_controller patches into seperate PR’s and getting them merged “early” - for those to work you need the dev branch of aiohomekit OR the very latest stable aiohomekit. The branch you were using is too out of date now.
The goal is that you won’t need a patched version of HA at all, just a patched version of aiohomekit. Hoping to achieve that in the next week.
And then the CoAP and BLE work we are doing will form aiohomekit 1.0.0.
I have rebased my branches, homekit_controller_coap_ble should work as a custom component with dev (probably not with 2022.02.x though).
The outstanding diff is +53/-46.
I should be able to get that down a bit further today as there are some functions that changed name but behave the same (e.g. find_ip_by_device_id can now find CoAP and BLE devices, so its name changed). That will be a trivial PR but touches a few tests.
The remaining changes further reduce the number of code paths expecting _tcp._hap.local/zeroconf to a minimum. This will be a little more work but shouldn’t be too bad.
@Rambooo those warnings are fine. When we first connect to a device, we read anything marked as readable. This includes both simple values (on/off, brightness) and control endpoints (write commands & read their responses). 55.50 is the list pairings control characteristic. The “transaction failed” warning is just the precursor to the second warning. I’ll think about either making a suppression list or just moving it to a debug level.
To add to what @Jc2k said, I’ll try to keep my hap-over-coap branches functional until we’re live in HA but I’ll mostly be testing the two repos linked above (aiohomekit dev & home-assistant/core homekit_controller_coap_ble). You should either stick with both of mine or both of @Jc2k’s repos.
Rebased again this morning. Some more of my PR’s were merged so now we are +42/-34 away from dev.
homeassistant/components/homekit_controller/config_flow.py | 32 ++++++++++++++++----------------
homeassistant/components/homekit_controller/manifest.json | 2 +-
homeassistant/generated/zeroconf.py | 5 +++++
tests/components/homekit_controller/test_config_flow.py | 37 ++++++++++++++++++++-----------------
4 files changed, 42 insertions(+), 34 deletions(-)
I think I am 3 PR’s away from having no differences between dev and my branch, at which point upgrading aiohomekit will be enough to turn on CoAP and BLE.
2 Likes