Emulated Hue suddenly stopped working with Echo Dot V2

To break this all down into something actionable, there’s two issues with two different causes:

  1. After telling Alexa to turn on/off a device or set its brightness/speed/etc, Alexa reports Device Malfunction until the device itself has completed updating. More on this below, screenshots and debug lines for testing if you’re interested.
  2. When using other apps such as the LIFX app, some devices can have their brightness set to 0 or 255, which causes Alexa to report Device doesn’t support requested value since the values are outside of the Hue API range of 1 - 254.

I added some debugging lines in hue_api.py (line numbers below reflect where they were inserted, in relation to the current version in Github) to see the requests made from Alexa to emulated Hue (both GET and PUT, as well as the immediate response sent back after a PUT) and tailed my log file to watch them:

223:        _LOGGER.warn('[HueOneLightStateView] <<< GET %s', json_response)
224:
...
268:        _LOGGER.warn('[HueOneLightChangeView] PUT >>> %s', request_json)
269:
...
422:        _LOGGER.warn('- RES - %s', json_response)
423:

I colored the output a bit to make it easier to see GET, PUT, and the immediate response to the PUT, using sed to replace things with color tags - here’s what I added to my .bash_profile:

CYEL=$(printf "\e[93m")
CGRN=$(printf "\e[92m")
CBLU=$(printf "\e[96m") 
CRES=$(printf "\e[0m")
alias tlog='tail -f /config/home-assistant.log | sed -r "s/(<<< GET)/${CYEL}\1${CRES}/g; s/(PUT >>>)/${CGRN}\1${CRES}/g; s/(- RES -)/${CBLU}\1${CRES}/g"' 

After that, I just use the tlog command to watch and color my log. What you see in the logs is as follows:

What’s happening is:

  1. Alexa tells the light to turn on (PUT >>> {'on': True}), and gets an immediate success reponse (- RES - [{'success': {'/lights/fan.bedroom_fan/state/on': True}}]).
  2. Alexa checks the light’s status, but sees that the light isn’t turned on (<<< GET {'state': {'on': False, ...).
  3. Alexa displays Device malfunction if you’re using the app, or says That device appears to be malfunctioning if you’re using voice control.
  4. After the device updates and Alexa checks its status again (perhaps a few times, actually), she sees the device is finally turned on ({'state': {'on': True, ...), the error goes away if you’re using the app, and everything looks good.

In the end, Alexa is reporting that there’s an error because as far as she can see the light isn’t responding successfully to her telling it to turn on. I don’t have a Hue bridge to test, but for some reason I have a feeling that they respond faster than our emulated Hue can respond. I don’t know a way to make this work and satisfy Alexa’s first immediate check, outside of emulated Hue “faking” the responses returned while also waiting for the entity to update before updating its own cached state. In the case of an eventual error, Alexa will first see that the device’s status was successfully changed, followed by our emulated Hue seeing a failure of some sort and updating its own internal cache/status, and Alexa later checking back in with emulated Hue and getting the updated (failed, but she doesn’t have to know that) state of the entity. At this point, Alexa will just display the device as turned off, no issue there.

Regarding the PRs submitted in an attempt to correct issues, neither of the two that I’ve seen (512759d and 3a34983) resolve the underlying issues discussed above, and both have additional shortcomings:

  1. 512759d does not strictly follow the Hue API, specifically regarding responses, although Home Assistant isn’t necessarily following the API either for the same reason.

    • Brightness can only range from 1 to 254, never be 0 or 255. Both Home Assistant’s emulated Hue and this PR have code that both allow, and in fact also specifically set, brightness to either 0 or 255 in several places.
    • This is possibly more of an issue with the Hue API itself, based on the principle that a byte ranges from 0 to 255 and arguably a brightness of 0 should constitute an OFF state, similar to how a value of 255 should constitute a fully-on state.
  2. Both PRs currently incorporate code or style that does not follow Home Assistant’s conventions:

    • 3a34983’s too-long line

    • 512759d typoed “HASS” as “HAS”, though this requires minimal effort to correct

    • 512759d incorrectly defines Extended color light requirements. Hue API’s definition states:

      On/off light (ZigBee Device ID: 0x0000), supports groups, scenes and on/off control
      Dimmable light (ZigBee Device ID: 0x0100), which supports groups, scenes, on/off and dimming.
      Color temperature light (ZigBee Device ID: 0x0220), which supports groups, scenes, on/off, dimming, and setting of a color temperature.
      Color light (ZigBee Device ID: 0x0200), which supports groups, scenes, on/off, dimming and color control (hue/saturation, enhanced hue, color loop and XY)
      Extended Color light (ZigBee Device ID: 0x0210), same as Color light, but which supports additional setting of color temperature

      The definition lacks the necessary SUPPORT_COLOR check and should read:

      613: if (entity_features & SUPPORT_BRIGHTNESS) and (
      614:     entity_features & SUPPORT_COLOR) and (
      615:     entity_features & SUPPORT_COLOR_TEMP
      616: ):
      617:     retval["type"] = "Extended color light"
      

I think in general, the emulated Hue component needs a pretty hefty rework with more attention paid to following the Hue API documentation more strictly. That’s only half of the problem though, as there’s so many devices that “don’t play nicely” (as in the case of my LIFX bulbs, for example), but that’s not their fault, as we’re really adding devices to the Hue hub that shouldn’t be there. I believe the best solution to address that, at least for now, would be to more strongly adhere to the Hue API, as this seems to be the cause of at least part of the issue people are experiencing. However, doing so is not without consequence as other apps like the LIFX app (and probably others) don’t see lights as 100% when set to the Hue API max of 254, depending on how they perform their round/ceil/floor operations on the brightness to percentage conversion. This does resolve Alexa’s Device doesn’t support requested value error, which in my opinion is more problematic as Alexa will fail to update devices in this state. One such check I’ve tested (working with the aforementioned limitation) is this:

...
69: # Define min/max values for device properties
70: HUE_API_STATE_BRI_MIN = 1.0
71: HUE_API_STATE_BRI_MAX = 254.0
72: HUE_API_STATE_HUE_MIN = 0.0
73: HUE_API_STATE_HUE_MAX = 65535.0
74: HUE_API_STATE_SAT_MIN = 0.0
75: HUE_API_STATE_SAT_MAX = 254.0
...
572:    # Clamp brightness, hue and saturation to valid values
573:    if data[STATE_BRIGHTNESS] is not None:
574:        data[STATE_BRIGHTNESS] = max(HUE_API_STATE_BRI_MIN, min(HUE_API_STATE_BRI_MAX, data[STATE_BRIGHTNESS]))
575:    if data[STATE_HUE] is not None:
576:        data[STATE_HUE] = max(HUE_API_STATE_HUE_MIN, min(HUE_API_STATE_HUE_MAX, data[STATE_HUE]))
577:    if data[STATE_SATURATION] is not None:
578:        data[STATE_SATURATION] = max(HUE_API_STATE_SAT_MIN, min(HUE_API_STATE_SAT_MAX, data[STATE_SATURATION]))
579:

This, despite needing line length breaks somewhere, has the effect of returning a clamped value between proper bounds when Alexa queries emulated Hue for a device’s brightness/hue/saturation. If LIFX or another app updates one of the entity’s supported values outside of that range, the value will not be modified; only the response sent to Alexa will be clamped. On the other side of things, if you use Alexa to change the device’s brightness/hue/saturation, the other app will see the clamped value. For example, setting a bulb to 100% with Alexa results in the LIFX app seeing the same bulb as 99% brightness as floor(254/255) = 0.996078... which rounds down to 99% in the LIFX app after calling the floor function and multiplying by 100 to get percentage.

I’d be willing to help out in any way I can, time permitted, and would love to see what others think and perhaps coordinate our efforts in one place. Currently we’ve got this topic on the community, issue #25047 on Github, at least two PRs (512759d and 3a34983) - neither of which fully resolve this problem, and likely other discussions about this. I’m fairly certain all of our collective brainpowers combined can come up with at least a mostly-functioning solution.

I’m currently working to re-write several bits to account for all of the problems the emulated Hue has to deal with, and I’ll be putting out a new pull request for others to grab my code and test it. I hate the state the emulated Hue is in, but the best I can do is fix the little things I’ve found and hopefully that’ll be good enough to work for everyone else.
 

- NobleKangaroo

5 Likes

It’s happening with many other home automation programs besides HA. Delays in responses from Alexa and error responses from Alexa when lights are responding seems to be a common feature since the latest round of Alexa firmware updates.

I don’t think Amazon are finished with their firmware updates either so Hue seems like its going to be an ongoing battle until everything settles down.

Hi, I tried all emulation and failed and wanted to try this.
May I know how did you installed this?
I’ve followed the git, and I’m stuck at-
cd: /etc/systemd/system: No such file or directory

Hi,
Im not sure what system you are trying to install ha-bridge on. It seems from your comment its not something that is using systemd (startup and service manager).

If I were you id start by just testing ha-bridge which is really simple to do as its a java app.

  1. Download the latest release ha-bridge-5.3.0.jar
  2. Make sure you have Java installed
  3. start HA-Bridge
    java -jar ha-bridge-5.3.0.jar

From there it should be running and you can test the integration with home assistant.
If all is well (it is still for me) then

  1. Backup your config database within HA-Bridge.
  2. Work on the start up scripts to auto start HA-Bridge.

I can help you with the last part if you let me know what system you are using (OS and Version). Or simply do a google search for your OS and ‘startup process’

Any problems, let me know.
Cheers

Yup, sure did. I used to use HA-Bridge with Domoticz before moving to Homeassistant.

I’m using hassio in Raspberry pi 4 (v0.100.2)
Tried this setup in github.

Nice work, I believe you are now the resident expert on this whole deal. Without having the api or the actual hardware, i’m not sure how far we can get in this whole ordeal.

1 Like

Hi,
So im not sure what v0.100.2 is. Are you using the hass.io HassOS? If thats the case I cant help as i havnt used that. I use HA on an ubuntu box. Ive installed on RP4 but on top of Raspbian.

Cheers

that’s alright. Thank you :slight_smile:

I’ve the same problems with my HA 0.100.3 on my Raspberry Pi. So whats the actual working fix or workaround? To login via Dev Port 22222 and edit the original hue_api.py, replacing line 574 like @Paul said?

My 2. problem with alexa is, that I have double entries of all my devices in the Alexa list “https://alexa.amazon.de/spa/index.html#appliances”. One device will show as Dimmable Light and second device with same name but as Royal Philips Electronics intelligentes Gerät.

Nice work! So if there is a timing issue, perhaps just delaying the success response back to Alexa would cure the problem. I am really no python coder but what is the line of code that gives back the success status?
Would like to test using sleep https://www.programiz.com/python-programming/time/sleep before that line of code.
Other option I guess is to perform the same test Alexa does and not return success until the “Alexa check” is also a success, perhaps with some kind of loop.

Replacing line 574 fixes the “Device doesn’t support requested value” error but most of the time I still get the “appears to malfunctioning” message from Alexa. Devices do successfully turn off/on so we are almost there…

Double amount of devices sounds like you just need to be sure to use “listen_port: 80” in the configuration.yaml for emulated_hue and go to https://alexa.amazon.com/spa/index.html#appliances and click “Forget All”. Then click Discover.

The 574 edit does fix Zwave devices, at least enough to allow them to work.

However, Zigbee devices still have an issue. On works, although Alexa can’t tell and claims a “device malfunction”. However, the app can’t turn it off since it thinks it’s already off. I’m not home to verify voice commands yet.

But trying to set a color for Zigbee devices in the Alexa app turns the light off with the “Device doesn’t support requested value” error.

I’m having the same Problem. I deleted everything related to the emulated hue from Home Assiastant. The config, the .json and even deleted the database. And Alexa is still discovering the old devices.

Is there no way to get rid of them?

edit: I had to completely reset my echo dot and set it up…

Try this:

1 Like

Would an HA update overwrite the changes made on the hue_api.py? I guess so, right?

@Paul
I’m logged in via SSH on port 22222. Where can I find the hue_api.py???

I seemed to fix the issue by first changing to port 80. Then I downloaded the proposed commit for hue_api.py from https://raw.githubusercontent.com/home-assistant/home-assistant/512759dbacd232d00abfa996bc9eac5847a5cde5/homeassistant/components/emulated_hue/hue_api.py.

Restarted home assistant, deleted all devices in Echo, and rediscovered. Everything working more or less perfectly in Alexa except for colors.

1 Like

You have to get the emulated hue integration files and make a custom component.

After you log in with SSH on port 22222 enter these commands:
login
docker exec -it homeassistant /bin/bash
find / -name hue_api.py

Suggest you take a copy of the orginal file before you modify it.
cd /usr/src/homeassistant/homeassistant/components/emulated_hue/
cp hue_api.py hue_api.py.orig

well, for those using Hass.io, all changes will be discarded on next homeassistant upgrade.You have to bind the hue_api.py from the host system to the container to make it work through out upgrades if they do not contain fixeds for this.