Native MQTT push updates from Hue Hub

Configuration
#21

Never expected to see such A+ quality feedback here, you guys are awesome :+1: :+1: :+1:

@rclsilver Impressive approach, maybe slightly beyond my skills atm, but Node-RED has been on my todo list for quite a while now - will get there soon :slight_smile:

@dlitz Thx also from my end for the zlib hint!! Have to play with this

@benwick666 I assume there was no latency reduction in circumventing clipd, but your vision of going as low level as possible there rocks! BTW before having (noticed) MQTT on the hub, I went down a somewhat similar bash route like yours previously. My source was logread though, learned from Rocka84’s great work.

#22

@j0hnd03 Probably not more than a couple msecs of latency reduction but the direct access does get me more useful (less gibberish-y) information to work with, since the clip-publish queue message is processed down from the dimmer’s original message. That information’s worth a lot from the coding side. Also, accessing the zlib-encrypted queue gives me the state of the bulbs themselves, initially as retained messages upon connecting, and then whenever the state changes. So using that, I wrote another script similar to the above, to take the info from a Hue bulb (as reported whenever it changes) to send to the other, non-Hue bulb(s) via Openhab. It’s a bit more complex (a large chunk of the code is just to convert the XY color to HSB color). I can post it if someone is interested. It single-handledly solves the problem of tying non-Hue bulbs to a Hue dimmer switch. (Note, scenes don’t work, but that’s not a feature I’ve used often anyway.)

#23

I’ve set up a git repo with improved versions of my scripts for syncing a non-Hue light with a Hue light, and one for mapping dimmer button events (or other sensors, theoretically) to requests so the dimmer can trigger other automations outside the Hue-niverse (via OpenHab in my case). https://github.com/20goto10/hue-jazz … suggestions and pull requests welcome.

1 Like
#24

Trying to get caught up on all the changes with HA and I am heavily invested in HUE motion sensors. I perused this topic with interest and understand the basics.

Is there now any advantage to doing things this way as opposed to to the new HA CORE )core-2021.10.6) which can apparently use push updates from the hue bridge?

I used to use a custom hue component to poll the bridge faster and then the official integration pulled the sensors at a slow 5 sec polling rate and then I remember the only time I saw Baloob swear in git bug thread was that one LOL.

Anyways I would do this if there was advantage in response time versus the new HA core version

#25

I have not measured but I don’t think there is really a noticable advantage in response times anymore after the changes in 2021.6.
Looks like we picked the perfect time to invest time into this :slight_smile:
Eversince I read the release notes of 2021.6 I have been wondering how the hue push update changes introduced by HA actually worked technically though… could not find anything on it.

#26

Good points. Who knows, HA is becoming more and more popular and perhaps we got an inside track on the API. Still at the mercy of Signify I guess and who knows what they may do. I am pretty sure that its not anybodies first rodeo and we have all seen companies who were open close up and try and ring every last drop of profit for there shareholders.
Thanks for spending the time on this …it may be needed in a pinch :wink:

#27

Hi guys,

Since the latest bridge firmware update (1.48 to 1.49) my hue-app fails to connect to my rooted hue-bridge.
My second bridge (not rooted) works normal after the update.

Does anyone have a similar issue?
I would love to get some advice on how to fix this. I’m a bit lost :wink:

#28

Mine is not offering 1.49 yet, will report back when I cross that bridge :slight_smile:
But if you can still get a shell you could try to grind logread for info or test if reverting the iptables mod helps

#29

I tried to revert all edits, and fiddled with ‘env bootslot 0/1’, but I couldn’t get the bridge to update and run like it should.

So, I did a factory reset, and I can tell you that’s a real pain with over 50 lights and accessories. All my scenes gone in Hue, Homekit, Domoticz, rooms and zones gone… I wish you all for the best! Let me know how it goes for you…

#30

I feel your pain, not looking forward to this upgrade.
Looks like it contains slightly bigger OS changes this time.
My hub still does not offer it but at least we can witness these and a few hue specific changes from “inside” thanks to our elevated privileges :wink:
Well possible that we’re causing some interference though… recently noticed this in logread for the first time (not sure since when this is logged)

Tue Jan 11 01:46:55 2022 daemon.info iot-connectivity[1664]: factory_reset
Tue Jan 11 01:46:55 2022 daemon.info iot-connectivity[1664]: issue while trying to generate the mosquitto bridge configuration

Tue Jan 11 01:48:29 2022 daemon.info croupierd[1161]: Error in reading JWT file [Errno 2] ENOENT
Tue Jan 11 01:48:29 2022 daemon.info croupierd[1161]: Error in reading JWT file [Errno 2] ENOENT

And I’m not sure if I noticed these micropython processes before

 1443     1 root     S     5248  8.7   0  0.1 micropython /usr/bin/updated --storage /home/updated/componentinfo.json --initial 135 --interval 86400
 1664     1 root     S     4808  7.9   0  0.1 micropython /usr/bin/iot-connectivity
 1161     1 root     S     4992  8.2   0  0.0 micropython /usr/bin/croupierd

At least the hub is getting safer, yay :smiley:
grafik

#31

Upgraded to 1.49 (1949107040) as well today, afterwards still had root access via ssh and no connectivity issues with the hue app.
However, diagcd crashed repeatedly for me on the new version, eventually sending the hub into reboot (after 26 crashes).

cat /tmp/exitlog/diagcd.exit.0
{"exitcode":127,"runtime":6,"timestamp":"2021-12-14T14:33:32"}

logread -f
Sat Jan 22 16:43:53 2022 daemon.info procd: Instance diagcd exit with error code 32512 after 0 seconds
Sat Jan 22 16:43:53 2022 daemon.info procd: Instance diagcd::instance1 s in a crash loop 26 crashes, 0 seconds since last crash
Sat Jan 22 16:43:53 2022 daemon.info procd: - shutdown -

For now, I stopped diagcd via init.d (/etc/init.d/diagcd stop) as a quick&dirty fix. No bootloop since.
Will see if I find more on exit codes from diagcd, but for now looks like I could dodge the bullet here :slight_smile:

#32

That sounds like my issue after the update… I wish I visited the log to come to the same fix. But what is diagcd for anyway? Without looking it up, it sounds like a diagnostic tool to compare directories or such.

Even after removing the few customizations, it made it crash though…

#33

Yup, logread is quite useful on the hub, quick check never hurts :slight_smile:

diagcd seems to be as its name indicates - a daemon for sending diagnostics data (reset reasons, crash codes, etc) to http://diagnostics.meethue.com/v2/diagnostics/report
Its configuration lives in /etc/config/diagcd and is referenced by the init script /etc/init.d/diagcd

Manually running diagcd with config - I ASSUME - explains what kept it from doing its job on my device after upgrade to 1.49… customizations regarding (lib)curl:

# /usr/bin/diagcd /etc/config/diagcd
Error relocating /usr/lib/libcurl.so.4: fcntl64: symbol not found
Error relocating /usr/lib/libcurl.so.4: __ctype_b: symbol not found

Fix:

  • Get opkg running again (binary, feed & config in here)
  • Install libc from here using opkg
  • opkg install curl
  • profit
    grafik
#35

Hi Ben
If i want to publish it again to the hue mqtt,
can you help with the code,no luck sofar for me. So publish the translated topics to the Hue mqtt
this way we can pick it up translated via the same hue mqtt server :slight_smile:
Thanks for this code allready and help

is it also possible to publish command to the MQTT on the bridge to turn the lights On and Off?

#36

Hi !
Like bwired, I would like to use this new MQTT communication to turn on/off light instead of the official REST API. Do anyone have success sending messages to hue bridge?
Maybe it’s not very usefull as the REST API is very powerfull, but I think we would have a very low latency which is important when dealing with lights.

#37

After some digging, I found that the topic “cmd/clip/light/update” will trigger the wanted changes :tada:
The payload doesn’t need compression (I didn’t try with) and is in this form:
{"data":{"color":{"xy":{"x":0.6058826495868515,"y":0.339189081823449}},"id":"794079c7-d65d-4339-b1b6-f247a1962c41","on":{"on":true},"type":"light"},"metadata":[{"key":"hue-application-id","value":"a23bb0ff-1a44-4f01-90f6-fe4abff4fce7"},{"key":"origin","value":"clip_v2"}],"request_id":"15ef9b9a-819f-4X56-b7ae-f14e442528cd","response_topic":"response/eb35401e-ea57-4e5d-80b7-d10b852f92a3"}

1 Like
#38

I’m meanwhile thinking about the next escalation we could attempt on the hub.
Recently realized we have direct access to zigbee on the hub (/dev/ttyACM0 /dev/ttyZigbee). cat’ing these devices reveals readable zigbee messages…
Wondering about tying in there with the objective of getting the hub to speak zigbee with non-Philips-supported devices. Might even be possible to run zigbee2mqtt on that thing… wouldnt that be awesome :star_struck:

[email protected]:~# cat /dev/ttyACM0
[Zcl,Req,0,240]0,208]
[Zcl,Req,0,241]
[Network,ClearRoutingEntry,0,1]
[Zcl,Req,0,2,243,208]
[Network,SendMtoRR,0,01000]
1 Like