Philips Wiz (not Hue) Bulbs: how can we advocate for an API

Hardware
55

Heads up for everyone. I used this custom component for a couple of days and then disabled it. When this component was enabled, my HA’s event loop used to get stuck every couple of minutes. Everything used to just get stuck - e.g. if I am opening the file editor, opening an automation, testing a template, checking for state, listening to events, loading history for an entity, anything and everything was affected. The action would fire from the UI and the result would come after 10+ seconds. I have the system monitor plugin and my Pi4 used to have 3.5 GB+ of free memory and the load average and cpu usage used to always be low. Automations used to get stuck. Incoming webhook calls from IFTTT or Google assistant used to time out! I kept scratching my head trying to figure what is wrong here. In the logs it kept giving warnings that every Tuya switch, google home, wiz light and every other switch/light was taking 30+ seconds to update. The wiz warnings kept showing UDP errors with python tracebacks showing UDP send timeouts to the lights. During this time, I would see that many of my chromecast devices, tuya switches, and even device_trackers would become “unavailable” 100s of times a day. Even the time sensor used to get delayed ticking the time!

The moment I disabled this component (commented out all the wiz lights from my config), my HA ( 0.106.6) runs flawlessly - does not get stuck at all, no update failures/timeouts in the logs, and all automations/webhooks are firing smoothly and on time.

I haven’t checked out the code yet, but @Sbidy is there any chance you are not properly using the asyncio on the event loop and that the UDP I/O is blocking hass.io’s core?

Glad to furnish any other details to help fix this.

56

Also do note that in my setup, there are some lights in the far corners of the house which are at a significant latency away from my router (e.g. the wiz app takes 2-3 seconds to turn them on/off), so that can explain how no one else noticed this problem with their HA core. Regardless, even some tuya switches and lights are farther away and they never cause the HA core to get stuck.

57

Hey @angadsingh
Thank you for the feedback - it is possible that the code doesn’t support the async feature wrong.
So I’ll try to reproduce the error. It can be a problem of the signal latency and the to short chosen timout for the UDP connection.
Can you please provide the error message or exception from the wiz_light integration?

Edit: Here the issue in GitHub https://github.com/sbidy/wiz_light/issues/6

1 Like
58

@Sbidy It is exactly the same error as you have put in your github issue. The problem is not that the exception is coming - it is good that you have short timeout in fact. The problem is that UDP I/O is blocking HA core. You need to do it asynchronously. Perhaps have a look at https://docs.python.org/3/howto/sockets.html#non-blocking-sockets

Also I see that in async_update (https://github.com/sbidy/wiz_light/blob/f31f229a395993e884d9e6fa56ba5ca3b75778d8/light.py#L179) you are calling update_state(), update_brightness(), etc. which are in turn calling self.getState() in https://github.com/sbidy/pywizlight/blob/d341b2b0d486d92b77037299b08cc019e251096d/pywizlight/bulb.py multiple times, which is sending the same UDP message getPilot. Why not just call it once? Why flood the lights with the same UDP message.

59

AHA!! Thank you so much for your comment, I could not get mine to work and it was because I had Circadian Rhythm on!

60

I took the liberty to fork’s @Sbidy’s generous effort and fix the issues.: https://github.com/sbidy/wiz_light/pull/9
https://github.com/sbidy/pywizlight/pull/2

Till the above PRs get merged, my component is also available for direct use at: https://github.com/angadsingh/wiz_light (this component does not need a dependency on pywizlight like @sbidy’s component)

Bug fixes:

Features:

  • Now supports switching the light to rhythm mode! (rhythm is defined as a scene for HA)
  • Implements a pattern of sending multiple command UDP datagrams until response is received
  • Consolidates getPilot and setPilot calls using a PilotBuilder and PilotParser. Removes unnecessary UDP calls for each and every attribute (color, temperature, brightness, scene, etc.) and makes a combined getPilot/setPilot call
  • enhanced debug logging for UDP

The problem with rhythm mode was that in @Sbidy’s component, he was making several different UDP calls in the turn_on method for setting brightness, color temperature, scene, etc, each of which were sending their own UDP setPilots, and then a final setPilot which had no params. Turns out, that if there is a rhythm defined in the room, it gets activated on the light by sending a setPilot with no params :slight_smile:. I am using that to allow you to switch any light to a ‘Rhythm’ scene now (and it detects if its in a rhythm scene as well). It all works flawlessly now.

For async I/O I am using https://github.com/jsbronder/asyncio-dgram, which internally uses asyncio DatagramTransport, which allows completely non-blocking UDP transport:
https://github.com/angadsingh/wiz_light/blob/caa90cebd9f8ccb2d588c900e36fbf19277eda9c/wizlight.py#L293

Since UDP can loose packets, and your light might be a long distance away from the router, I am continuously sending the UDP command until there is a response from the light. This has in my tests worked way better than just sending once and just waiting for a timeout.

This is the result:

2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.light] [wizlight 192.168.1.88] updating state
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.wizlight] [wizlight 192.168.1.88, connid 384b0e6912316c] connecting to UDP port
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.wizlight] [wizlight 192.168.1.7, connid 384b0e6911bdc4] listening for response datagram
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.wizlight] [wizlight 192.168.1.7, connid 384b0e6911bdc4] sending command datagram 0: {"method":"getPilot","params":{}}
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.wizlight] [wizlight 192.168.1.7, connid 384b0e6911bdc4] sending command datagram 1: {"method":"getPilot","params":{}}
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.wizlight] [wizlight 192.168.1.7, connid 384b0e6911bdc4] sending command datagram 2: {"method":"getPilot","params":{}}
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.wizlight] [wizlight 192.168.1.88, connid 384b0e6912316c] listening for response datagram
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.wizlight] [wizlight 192.168.1.88, connid 384b0e6912316c] sending command datagram 0: {"method":"getPilot","params":{}}
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.wizlight] [wizlight 192.168.1.7, connid 384b0e6911bdc4] response received: {'method': 'getPilot', 'env': 'pro', 'result': {'mac': 'a8bb50a4f94d', 'rssi': -64, 'src': '', 'state': True, 'sceneId': 0, 'temp': 3154, 'dimming': 58}}
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.light] [wizlight 192.168.1.7] updated state: True
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.wizlight] [wizlight 192.168.1.88, connid 384b0e6912316c] response received: {'method': 'getPilot', 'env': 'pro', 'result': {'mac': 'a8bb50a5806c', 'rssi': -53, 'src': '', 'state': True, 'sceneId': 13, 'speed': 100, 'temp': 6500, 'dimming': 100, 'schdPsetId': 11}}
2020-03-18 11:49:56 DEBUG (MainThread) [custom_components.wiz_light.light] [wizlight 192.168.1.88] updated state: True
3 Likes
61

Thanks to @angadsingh the implementation get more quality and resilience :slight_smile:.
I’ve moved some stuff to the pywizlight packet. Now the integration should be worked with the pypip package to separate the logic form the integration.

The new version with all changes from @angadsingh are available in the testing branch.
Please test this new integration with your bulbs - it works with Philips Wiz Hue and the SLV Play.

For installing the testing branch version use git clone --branch testing https://github.com/sbidy/wiz_light.git in your custom components folder. Please restart the container or home assistant service and check the log file for INFO (SyncWorker_6) [homeassistant.util.package] Attempting install of pywizlight==0.3.0.

If no issues coming up I’ll merge the testing into the master.
Maybe I will add the configFlow feature also to the new master release (mmh … a versioning would be great :smiley: … )

1 Like
62

Thank you all for the hard work on this. I’m new to HA, so this might be a simple answer.

I have one Wiz bulb in HA and would like to add it to the Living Room area. I don’t seem to be able to do it in the gui or the config.

Any idea?

63

Thank you for your hard work! Just to let you know that its working with the WIZ A.E27 lamps as well!
Hope to see a HACS or core component soon.

64

Thx @Sbidy and @angadsingh for the updates. Seems I have issues with the older version of the custom component too due to the blocking call. The async approach seems to be the correct way to go. But after trying to install the asyncio-dgram package my HASS is giving issues. Not sure if it’s related to some version compatibility as I’m still on version 0.95.4 and Python 3.5.4 (I need to move it but have to make sure nothing breaks as I’ve lot of components and integrations/scripts working flawlessly for now and hence not taking the newer releases that require Python upgrade - maybe will create 2nd instance and cutover to it).
But for the interim is there an alternate option to move to asyncio/non-blocking option without going via the asyncio-dgram option. I would like to move to async operations as the core keeps on getting stuck and at that point the current task (as mentioned above by @angadsingh times out or doesn’t return back if from the UI);

Any option to move forward on this is appreciated.

65

what issues are you seeing? its really a small simple single file wrapper over asyncio’s datagram API. HomeAssistant natively and internally uses asyncio. The core runs on it. So this is nothing different. Just a utility wrapper: asyncio-dgram/asyncio_dgram/aio.py at 71e2e2fad8cf210b9551ca3281c95b96c88adc84 · jsbronder/asyncio-dgram · GitHub

66

Thx for your prompt response. I was trying to install the lib using pip3 but it is only compatible with python 3.6+ and my python version is 3.5.3.

Based on your suggestion i’ll try to natively import this lib/aio.py and see if that works… something that can bring this up quickly without going thru lot of changes plus avoiding the lockup problem with the wiz integration…

thx again, any other insights on this welcome…

67

Updated the changes from your repo (not original from @Sbidy) and updated it to make it compatible with Python version < 3.7 also removed the external dependency on the asyncio_dgram module as it couldn’t be installed via pip3 due to older Python version.

This version is only meant for use in case someone has an older version of HASS with Python <=3.6 and cannot move to a new version due some specific integrations/needs. All others should use the updated @Sbidy version of Wiz light component along with asyncio_dgram.

My changes are forked and available at:

thx @angadsingh for your insight and great work on making this async (no more lockups seen) and also thx to @Sbidy for this custom component that allows folks to use relatively inexpensive Wiz lights with HASS (fraction of the cost of other lights with similar or less capabilities and no need for a hub!)

1 Like
68

Good for your @mhpetiwala. Do plan to upgrade your python version soon, so that you can use the updated pywizlight component directly and get new features/fixes being added by @Sbidy

69

Hi, I have a tunable white bulb and it shows up as a bulb able to change all colors. How can I contribute to iron this out?

@sbidy: I’m more than happy to contribute hardware. How can I help?

Do you guys have any plans for auto-detection of lamps in the network?

70

Hey, yes that’s a problem. We try to figure out how the bulbs can be detected as RGB+W or single color. The quick and easy way is to add a additional parameter into the the config.
But the final solution should be a auto-detection.

I’ll try to bring in the configuration parameter into the code on Friday.
Can you post your system config and type of your bulb?

echo '{"method":"getSystemConfig","params":{}}' | nc -u -w 1 <YOU BULB IP> 38899
71

The response of my Philips Wiz Connected Tunable White Wi-Fi LED (A19) is:

{"method":"getSystemConfig","env":"pro","result":{"mac":"a8bb50891fbe","homeId":654213,"roomId":990480,"homeLock":false,"pairingLock":false,"typeId":0,"moduleName":"ESP01_SHTW1C_31","fwVersion":"1.18.0","groupId":0,"drvConf":[20,1]}}

I also have Philips Wiz Full color recessed Downlights:
{"method":"getSystemConfig","env":"pro","result":{"mac":"a8bb50f22e05","homeId":654213,"roomId":1000729,"homeLock":false,"pairingLock":false,"typeId":0,"moduleName":"ESP01_SHRGB1C_31","fwVersion":"1.18.0","groupId":0,"drvConf":[44,1]}}

72

Just loaded your custom component - controlling well! Many thanks for your work so far!

To assist understanding bulb types, I have a Wiz brand A B22 WiZ60 DW

Fixed colour (Warm White), dimmable bulb. I noticed that many of the scenes options in the light controller cause errors (not available) as well as a colour picker (which is also not applicable).

Bulb response is:
{"method":"getSystemConfig","env":"pro","result":{"mac":"zzthemac","homeId":zzhomeid,"roomId":zz,"homeLock":false,"pairingLock":false,"typeId":0,"moduleName":"ESP01_SHDW_01","fwVersion":"1.18.0","groupId":0,"drvConf":[20,2]}}

73

I am having trouble getting @Sbidy’s repo to fully work.

My lights were created fine but I always get the following logs due to UDP connection refusal:

image
image960×644 210 KB

I made sure the IP addresses for my lights are static before getting these logs.

Any help would be greatly appreciated!

74

is the physical switch to the light on and the light connected to your router on specific IP 192.168.1.41 as shown in the error above. Seems it’s not able to reach the bulb’s controller and repeatedly attempting and failing with timeout…

Are you able to reach this bulb and control via the WiZ app? or have you tried any command line options to try and reach it…