Where are the new entities for Hue (dimmer and tap) switches

As cc’ed by @Mariusthvdb, my POV on this

Last hue update now does some extra things:

  • It register as devices the Hue dimmer switch and the Hue Tap switch,
  • and in doing so it also enables device triggers to build automations from UI based on button presses.
  • For battery powered remotes (the Hue dimmer), it also adds a battery level entity.

That follows the general approach of mirroring real objects with “devices”, and expand their states as separate entities.

feels rather inadequate not having an entity in the states for an entity in real life…

It has now a device. It is like the motion sensors, which get into HA as 1 device with multiple entities (binary sensor for motion, temperature and light level sensors, etc.)

With official integrations, I believe anything like a button typically doesn’t have an entity that reflects the button press state. Events are used for that.

Exactly. In fact, this feature has been “inspired” (mostly copied) from the deCONZ ZigBee integration.

Having a sensor for the button press itself probably wouldn’t work with the Hue integration since the integration uses polling. Light, sensor, etc. states and events are updated every 5 seconds, so if you had a button press entity and you pressed the same button that was pressed before, the state likely wouldn’t change (it would go from up to down to up so quickly that it would likely be between polling cycles). That would make automations that are supposed to trigger on specific state changes likely not fire in that case.

That is totally true. The same zigbee remote device would work much better under a Conbee stick than in the Hue bridge, as deCONZ has pushing ability, so no button presses are lost.

BUT, that does not make this feature unusable. You need to take into account a possible delay of 0 to 5 secs between button press and HA action, but that is not the end of the world: the “hold” button press action is specially useful to create automations on devices that already are configured for simple presses on the Hue app.

I think OP was concerned that that was the only way the button presses could be used in automations.

The “hue_event” is always fired and can be used directly. Device triggers use it internally and they are just an easier way to do things from the UI.

@azogue has made a custom component event sensor, will give that a shot soon now this is in Core Ha.

There are some people (myself included) that want to “see” or record the state of the remotes in the house, so HA events are not enough. For those nerds I created that very simple custom integration to create a “sensor” entity from some specific event.

  • It is not needed to use the remotes in automations (better use the integrated device triggers)
  • It can be used for any kind of remote or other HA event emitter. In my case, I use it for all my remotes. Some are integrated in the hue bridge (they have the polling known delay) and others in the deconz (those have an instant response).
2 Likes

BTW, about the problem of the fixed polling interval in the main hue integration, I just did another custom integration :), details in a new thread:

Thanks for your input! Glad that, despite not being a dev, what I expressed was mostly accurate. Cool custom component too.

1 Like

HI!

just installed your CC, and it seems to be working fine :wink:

upon startup the sensors were unknown, but clicking made everything appear. Was a bit of a search t find the correct mapping for the dimmer switch, but think this should be it (using the ‘official’ Hue api event names:

  - platform: eventsensor
    name: Dimmer switch Living
    event: hue_event
    event_data:
      unique_id: 00:17:88:01:02:c3:74:f6-02-fc00
    state: event
    state_map:
      1000: 1 click
      1001: 1 hold
      1002: 1 click release
      1003: 1 hold release
      2000: 2 click
      2001: 2 hold
      2002: 2 click release
      2003: 2 hold release
      3000: 3 click
      3001: 3 hold
      3002: 3 click release
      3003: 3 hold release
      4000: 4 click
      4001: 4 hold
      4002: 4 click release
      4003: 4 hold release

maybe add that to the docs on your CC repo? secondly, could you add an icon to the different sensor models? Know we can customize,

homeassistant:
  customize_glob:
    sensor.dimmer_switch_*:
      icon: mdi:light-switch

  customize:
    sensor.tap_switch_auditorium:
      icon: mdi:google-circles-communities

but would be cool if it was already in the integration itself, like with Huecustom:

ICONS = {
    "SML": "mdi:motion",
    "RWL": "mdi:light-switch",
    "ROM": "mdi:remote",
    "ZGP": "mdi:google-circles-communities",
    "FOH": "mdi:light-switch",
}

making it look like:

1 question: after installing this, my sensors from @robmarkcole 's CC are no longer working and show as unavailable. Is this expected, or a bug somewhere.


thanks!

upon startup the sensors were unknown, but clicking made everything appear

That is expected and reasonable, as no state is set until some event gets fired. A good thing about the CC is that it uses RestoreEntity, so in the next HA restart it would ‘remember’ the last press.

Was a bit of a search t find the correct mapping for the dimmer switch

An example of the Hue Tap switch mapping is already in the docs, and one mapping for the dimmer switch is in my HA config repo. Also, if some “raw state” is received and it is not defined it would appear as the state, so you can always find out the states by just pressing buttons, so no big deal.

secondly, could you add an icon to the different sensor models? Know we can customize

You can customize it like any other HA entity. That CC is called eventsensor and meant to be used generically, not only for “hue remotes”, so setting specific icons is totally out of scope.

1 question: after installing this, my sensors from @robmarkcole 's CC are no longer working and show as unavailable. Is this expected, or a bug somewhere.

It is not related at all, so I don’t know what is going on on your HA, are you sure Robin’s CC is loaded? And most important: why do you need both??

If you have only Hue dimmer & tap remotes, and you’re already using the main hue, the best combination by far is:

  1. Doing nothing more :slight_smile:
  2. If you need/want a sensor entity for the button presses → use eventsensor
  3. If you need/want different polling intervals → use fasthue

No extra calls to the hue hub, no extra stress for HA.

I dont need both, I simply had it installed at the time I installed eventsensor. Which made the sensors made by CC unavailable as posted. Might be not related, but, there you have it :wink:

Ive taken out the CC now, so 100% sure they dont interfere now, but it might be interesting to understand why this happened.

as said, Ive already done that. Appreciate it being out of scope. no issue at all, thanks.

about fasthue: did you understand my question on the setting of the frequency by means of a slider in the frontend?

nevermind, just saw your response, thank you. Posted a suggested automation there, hope this works out nicely.

Sorry, it would be interesting for you only :slight_smile:. As I said, it is not related.

Which made the sensors made by CC unavailable as posted

No. Listening to an event does not make any other entity to fail, believe me. It is more probable that the behavior in the CC is related to changes in hue from HA Core, not from eventsensor

about fasthue: did you understand my question on the setting of the frequency by means of a slider in the frontend?

Yes, perfectly. I use that schema of input_number + automation to make other service calls from UI with a slider, like setting the speed of an air purifier fan:

But I would never ask for the xiaomi fan to generate that input_number from the integration, as it is out of the scope of it, don’t you think? The same applies here :slight_smile:

well, whatever, it is what happened, on a perfectly working CC with all sensors available. Didn’t update HA, I installed your eventsensor. Only reporting here.

dont think that is what I asked for, I only asked if we could use a frontend slider to set the update frequency, and posted this automation:

automation:
  - alias: set_hue_polling_interval:
    trigger: 
      platform: state
      entity_id: input_number.hue_polling_interval
    action:
      service: fasthue.set_update_interval
      entity_id: sensor.hue_polling_interval
      data_template:
        scan_interval:
          seconds: >
            {{states('input_number.hue_polling_interval')|int}}

will test later on.
edit
added |int to the template

Is not the eventsensor, I can asure you. It is probably one of the 2 CC’s (remotes or sensors) messing with the hue setup. What specific version of which specific CC were you trying?

will test later on.

It won’t work, please see my update on the other thread.

that might very well be true, and I wasn’t stating your eventsensor was the culprit. Merely noted that after installing that, my CC version of Hue_sensor stopped working.

It was a version from before the move to the remote platform, when Hue custom still used the hue_sensor. It worked just fine :wink:

"""
Sensor for checking the status of Hue sensors.

For more details about this platform, please refer to the documentation at
https://home-assistant.io/components/sensor.hue/
"""
import asyncio
import async_timeout
import logging
import threading
from datetime import timedelta

from homeassistant.components.sensor import PLATFORM_SCHEMA
from homeassistant.helpers.entity import Entity
from homeassistant.helpers.event import async_track_time_interval

DEPENDENCIES = ["hue"]

__version__ = "1.8"

_LOGGER = logging.getLogger(__name__)

SCAN_INTERVAL = timedelta(seconds=1)
TYPE_GEOFENCE = "Geofence"
ICONS = {
    "SML": "mdi:motion",
    "RWL": "mdi:light-switch",
    "ROM": "mdi:remote",
    "ZGP": "mdi:google-circles-communities",
    "FOH": "mdi:light-switch",
}
DEVICE_CLASSES = {"SML": "motion"}
ATTRS = {
    "SML": [
        "light_level",
        "battery",
        "last_updated",
        "lx",
        "dark",
        "daylight",
        "temperature",
        "on",
        "reachable",
        "sensitivity",
        "threshold",
    ],
    "RWL": ["last_updated","battery","on","reachable","name","type","modelid","uniqueid"],
    "ROM": ["last_updated", "battery", "on", "reachable"],
    "ZGP": ["last_updated","name","type","modelid","uniqueid"],
    "FOH": ["last_updated"],
}


def parse_hue_api_response(sensors):
    """Take in the Hue API json response."""
    data_dict = {}  # The list of sensors, referenced by their hue_id.

    # Loop over all keys (1,2 etc) to identify sensors and get data.
    for sensor in sensors:
        modelid = sensor["modelid"][0:3]
        if modelid in ["RWL", "ROM", "SML", "ZGP"]:
            _key = modelid + "_" + sensor["uniqueid"][:-5]
            if modelid == "RWL" or modelid == "ROM":
                data_dict[_key] = parse_rwl(sensor)
            elif modelid == "ZGP":
                data_dict[_key] = parse_zgp(sensor)

        elif modelid == "FOH":  ############# New Model ID
            _key = modelid + "_" + sensor["uniqueid"][-5:]  ###needed for uniqueness
            data_dict[_key] = parse_foh(sensor)

    return data_dict


def parse_zgp(response):
    """Parse the json response for a ZGPSWITCH Hue Tap."""
    TAP_BUTTONS = {34: "1", 16: "2", 17: "3", 18: "4"}
    press = response["state"]["buttonevent"]
    if press is None or press not in TAP_BUTTONS:
        button = "No data"
    else:
        button = TAP_BUTTONS[press]

    data = {
        "model": "ZGP",
        "name": response["name"],
        "state": button,
        "last_updated": response["state"]["lastupdated"].split("T"),
        "type": response["type"],
        "modelid": response["modelid"],
        "uniqueid": response["uniqueid"],
    }
    return data


def parse_rwl(response):
    """Parse the json response for a RWL Hue remote."""

    responsecodes = {"0":" click","1":" hold","2":" click release","3":" hold release"}

    button = None
    if response["state"]["buttonevent"]:
        press = str(response["state"]["buttonevent"])
        button = str(press)[0] + responsecodes[press[-1]]

    data = {
        "model": "RWL",
        "name": response["name"],
        "state": button,
        "on": response["config"]["on"],
        "battery": response["config"]["battery"],
        "reachable": response["config"]["reachable"],
        "last_updated": response["state"]["lastupdated"].split("T"),
        "type": response["type"],
        "modelid": response["modelid"],
        "uniqueid": response["uniqueid"],
    }
    return data


def parse_foh(response):
    """Parse the JSON response for a FOHSWITCH (type still = ZGPSwitch)"""
    FOH_BUTTONS = {
        16: "left_upper_press",
        20: "left_upper_release",
        17: "left_lower_press",
        21: "left_lower_release",
        18: "right_lower_press",
        22: "right_lower_release",
        19: "right_upper_press",
        23: "right_upper_release",
        100: "double_upper_press",
        101: "double_upper_release",
        98: "double_lower_press",
        99: "double_lower_release",
    }

    press = response["state"]["buttonevent"]
    if press is None or press not in FOH_BUTTONS:
        button = "No data"
    else:
        button = FOH_BUTTONS[press]

    data = {
        "model": "FOH",
        "name": response["name"],
        "state": button,
        "last_updated": response["state"]["lastupdated"].split("T"),
    }
    return data


def get_bridges(hass):
    from homeassistant.components import hue
    from homeassistant.components.hue.bridge import HueBridge

    return [
        entry
        for entry in hass.data[hue.DOMAIN].values()
        if isinstance(entry, HueBridge) and entry.api
    ]


async def update_api(api):
    import aiohue

    try:
        with async_timeout.timeout(10):
            await api.update()
    except (asyncio.TimeoutError, aiohue.AiohueException) as err:
        _LOGGER.debug("Failed to fetch sensors: %s", err)
        return False
    return True


async def async_setup_platform(hass, config, async_add_entities, discovery_info=None):
    """Initialise Hue Bridge connection."""
    data = HueSensorData(hass, async_add_entities)
    await data.async_update_info()
    async_track_time_interval(hass, data.async_update_info, SCAN_INTERVAL)


class HueSensorData(object):
    """Get the latest sensor data."""

    def __init__(self, hass, async_add_entities):
        """Initialize the data object."""
        self.hass = hass
        self.lock = threading.Lock()
        self.data = {}
        self.sensors = {}
        self.async_add_entities = async_add_entities

    async def update_bridge(self, bridge):
        available = await update_api(bridge.api.sensors)
        if not available:
            return

        data = parse_hue_api_response(
            sensor.raw
            for sensor in bridge.api.sensors.values()
            if sensor.type != TYPE_GEOFENCE
        )

        new_sensors = data.keys() - self.data.keys()
        updated_sensors = []
        for key, new in data.items():
            new["changed"] = True
            old = self.data.get(key)
            if not old or old == new:
                continue
            updated_sensors.append(key)
            if (
                old["last_updated"] == new["last_updated"]
                and old["state"] == new["state"]
            ):
                new["changed"] = False
        self.data.update(data)

        new_entities = {
            entity_id: HueSensor(entity_id, self) for entity_id in new_sensors
        }
        if new_entities:
            _LOGGER.debug("Created %s", ", ".join(new_entities.keys()))
            self.sensors.update(new_entities)
            self.async_add_entities(new_entities.values(), True)
        for entity_id in updated_sensors:
            self.sensors[entity_id].async_schedule_update_ha_state()

    async def async_update_info(self, now=None):
        """Get the bridge info."""
        locked = self.lock.acquire(False)
        if not locked:
            return
        try:
            bridges = get_bridges(self.hass)
            if not bridges:
                if now:
                    # periodic task
                    await asyncio.sleep(5)
                return
            await asyncio.wait(
                [self.update_bridge(bridge) for bridge in bridges], loop=self.hass.loop
            )
        finally:
            self.lock.release()


class HueSensor(Entity):
    """Class to hold Hue Sensor basic info."""

    ICON = "mdi:run-fast"

    def __init__(self, hue_id, data):
        """Initialize the sensor object."""
        self._hue_id = hue_id
        self._data = data.data  # data is in .data

    @property
    def should_poll(self):
        """No polling needed."""
        return False

    @property
    def name(self):
        """Return the name of the sensor."""
        data = self._data.get(self._hue_id)
        if data:
            return data["name"]

    @property
    def unique_id(self):
        """Return the ID of this Hue sensor."""
        return self._hue_id[+4:][:-3]

    @property
    def state(self):
        """Return the state of the sensor."""
        data = self._data.get(self._hue_id)
#        if data and data["changed"]:
        return data["state"]

    @property
    def icon(self):
        """Icon to use in the frontend, if any."""
        data = self._data.get(self._hue_id)
        if data:
            icon = ICONS.get(data["model"])
            if icon:
                return icon
        return self.ICON

    @property
    def device_class(self):
        """Return the class of this device, from component DEVICE_CLASSES."""
        data = self._data.get(self._hue_id)
        if data:
            device_class = DEVICE_CLASSES.get(data["model"])
            if device_class:
                return device_class

    @property
    def device_state_attributes(self):
        """Attributes."""
        data = self._data.get(self._hue_id)
        if data:
            return {key: data.get(key) for key in ATTRS.get(data["model"], [])}

It was a version from before the move to the remote platform, when Hue custom still used the hue_sensor. It worked just fine

We moved past that because that is not true, and I’m very tired of trying to explain it to you. That code is hammering the hue bridge and making HA fail. I will not lose my time on this anymore, sorry.

sorry to hear.
it’s not hammering any more than your version of the current huecustom CC does at 0.5 seconds. in fact, it does 2x fewer… or less when I set it to a lower frequency (as I have always had it)

nevermind. reinstalled that version once more to be sure to have the latest of the latest.

having said all that, does your own CC for Hue Polling interval interfere with the polling interval set in the HueCustom integration? I ask since they both interact with the core HA update frequency?

I just hit upon this topic, and as I am already using @robmarkcole 's CC, I thought it worth a try to replace this with this new and seemingly promising CC. My first findings after installation (for now in parallel with the ‘old’ CC) are as follows :
The good news : my ‘old’ sensors keep working, so no ‘malicious’ interaction - as confirmed by @azogue and contrary to the findings of @Mariusthvdb
The sort of bad news : events seem to be missed. Clicking a button sometimes generates a ‘final’ state of e.g. 4002, sometimes 4000 (testing with a Hue Dimmer) - seemingly missing either the down or the up event at times.
And although the raw event monitoring shows the same ‘malfunction’, suggesting the Hue integration being the real culprit, it is surprising that @robmarkcole’s sensor captures ALL up/down clicks without missing a beat …

Hi @pav,

That is, sadly, expected. As the hue integration is of polling type, it depends on the manual request to the bridge. The differences that you see between the CCs and the main hue are because of the different polling interval (the CC makes it at ~1Hz and the main hue each 5s)

Anyway, as both are “polling” the hub, there will be always missed events, and that’s why only the “_up” or “release” presses are registered to be used as device triggers.

The Hue bridge register all remote presses, and stores the last state. The components just ‘take a photo’ of these states at some fixed rate (each 5s or each 1s). When the photo changes, events are fired (or states are updated, if there are entities involved).

To make the main hue faster, so the CC’s for remotes is not needed, use the new CC described in Custom integration to set polling interval for Friends of Hue devices - #5 by azogue

About the eventsensor CC, as you suspected, it is only mirroring the lack of some events missed by the main hue. If used over a deconz_event (using the same ZigBee gear but linked to a ConBee stick), it doesn’t miss any event.

Hi Azogue,

does your new CC for setting the polling interval conflict with the latest version of CC Huesensor, where polling is set fixed in the CC?

Hi @azogue, thx for confirming my suspicion.

That may be as it is, but I guess that doesn’t prevent the device triggers built upon these selectively ‘registered presses’ be error-prone because of missed presses (because the release may just as well be missed as the click) … ?

Aah, don’t we all hate to have to resort to polling devices :frowning_face:
I now tend to think that the best way to deal with my remotes is to move them over to zigbee2mqtt, which I’m using already for some other devices. No more Hue bridge hammering/polling, and all necessary and useful sensors automatically at the ready …

That being said, a big thx anyway for a CC that no doubt will sooner or later prove to be very useful :+1:

1 Like

It shouldn’t, as the bridge always stores the last action (–> the ‘release’ event).
It could happen that ‘the photo’ of 1 update gets in the middle of an action (in that case the ‘click’ is processed), but in the next update it would get the release. So the release is not lost, only will have a lot of delay! (a full update interval).

Aah, don’t we all hate to have to resort to polling devices :frowning_face:

Yes, we do :frowning:

In my case, I moved part of my zigbee gear to one conbee stick (connected to the RPI4 running HA).
In particular, I moved the motion sensors and 3 of 6 remotes, and also a pair of rooms with lights that I never control with the Hue app (to maintain a second zigbee mesh to support those sensors), and I’m very happy with the results.

I didn’t move everything, as the alternative is not without setbacks:

  • The Hue mobile app is really good for manual setting of lights and ambiances, much better than current lovelace UI,
  • The Philips Ambilight + Hue integration for the TV, can’t miss that :slight_smile:
  • Some little things, like Hue labs ‘candles’ simulations and alike.
  • Moving a remote to the conbee means reprogramming it completely, and in most cases also will introduce a HA dependency for it: no more independent subsystems (remotes working with HA down)

But as it looks like Philips is never going to give us a pushing API, if instant response is in need, that is the way to do it.

Yes, of course. But there are details:

  • HueSensor CC is already deprecated, and had the remotes removed, so it only implements things that are already in main hue. It works with its own schedule update, so no, it doesn’t have any conflict. But there is no reason to use it now.
  • HueRemotes CC in the last version is sharing the data update coordinator with the main hue, and changing it to its own fixed fast refresh rate, so yes, if you also use the new CC it will go crazy changing the update interval between the 2 values (remote CC vs fasthue CC). So don’t use both!

The reason to make fasthue is to make HueRemotes CC unnecessary, as the only real feature that is adding now is the fast refresh rate. With fasthue you have a dynamic refresh rate, independent for each bridge, plus UI config without HA restart, and it is basically a hack to change the fixed 5s scan_interval.

My motivation to implement it was just to hack my hue integration to poll the 3 remaining remotes in the hue network faster than /5s (2s for me is perfect, as I have automations triggering on hold button releases, and simple presses configured in Hue app).

As I don’t use anything more to talk with the Hue bridge (only HA + Ambilight TV + ocasional manual usage of Hue app), and I know my system is very stable even polling at 1Hz, that is the best configuration for me.

Combined with the eventsensor CC to create sensor entities for all remotes (hue + deconz zigbee meshes), because I like to see the history of button presses, that is by far the optimal combination for me, and I think it may be useful for anyone complaining about the 5s polling of hue, as it will not extra-hammer the hub nor create any interference with the main hue (what it does is equivalent to manually edit the components/hue/sensor_base.py file to change the 5s to anything >= 1s)

thanks @azogue

thanks for confirming

I feared as much. Think this warrants a warning on both repo’s since people might not realize this.

Since 108.4, I’ve noticed my Hue system go much more reliable, and the constant going ‘unavailable’ of the core Hue entities seems to have calmed down enormously !
So, because of that, I’ve taken out the CC Huesensor/remote integrations completely. I do miss some ‘attributes’, most prominently the ‘config.on’ for the binary_sensors, which I need to be able to switch the motion sensors on/off.

if we would have the ‘config.on’ attribute in core we could simply create a template switch. Or maybe even a core switch would be even better…

Without it we need solutions like this:

      corridor_office_sensor_motion_switch:
        friendly_name: 'Corridor office motion sensor switch'
        command_on: >
          curl -X PUT -d '{"on":true}' "http://192.168.1.212/api/dmredactedQq/sensors/28/config/"
        command_off: >
          curl -X PUT -d '{"on":false}' "http://192.168.1.212/api/dmredactedQq/sensors/28/config/"
        command_state: curl http://192.168.1.212/api/dmredactedQq/sensors/28/
        value_template: >
          {{value_json.config.on}}

to switch from within HA Lovelace frontend, in backend automations/logic.

But other than that, the system indeed seems very reliable, hope that is here to stay.

I have 2 more wishes to go into core: the hue device_tracker, with its attributes, and the daylight sensor, which I now make using

  - platform: command_line
    name: Hue Daylight
    command: !secret hue_daylight_command
    value_template: >
      {{'on' if value_json.state.daylight == True else 'off'}}
    json_attributes:
      - state
      - config
      - name
      - type
      - modelid
      - manufacturername
      - swversion
    scan_interval: 300

Your eventsensor is working beautifully, and, since I don’t use the remotes to control from within HA, but simply want a monitoring system, it suffices 100%. Again: thanks!

A bit hesitant to test the polling adjustment in fasthue, now my system has finally stabilized, but as I read your suggestions in the repo, it doesn’t hurt, and taking it out brings back the system to its core state :wink:

Cheers.

It won’t break anything, it is just a silly thing to do (set a scan_interval of X in one CC and another Y in the other)

A bit hesitant to test the polling adjustment in fasthue , now my system has finally stabilized

If you don’t need to poll faster why would you try it? Try always to reduce complexity, not the opposite :slight_smile:

I have 2 more wishes to go into core: the hue device_tracker, with its attributes, and the daylight sensor, which I now make using

You won’t see me trying to implement that, sorry. For me (and others) it has no sense, as they are virtual sensors (no real devices involved) and they don’t solve anything that is better solvable other way.

  • What do you need from the daylight sensor that you cannot get from the sun component, which is much more powerful?
  • There are lots of better tracking tools available in HA that the one implemented by the Hue bridge, IMHO. For example, have you tried the mobile_app tracking system?