All my hue lights randomly becoming unavailable

I think most of the connection problems are with Hue, I get unresponsive lights all the time using the hue app, but it still works if I use the native app even with the error. My issue is HA or the HomeKit app won’t control them if they say unresponsive… very frustrating indeed!

That should be fixed by adding:

  allow_unreachable: true

to your hue config. Unfortunately this does not seem to be available yet through the UI config

2 things I did added to updating to 108.+ :

  • exclude domain light in Logger and history
  • update custom card Button (which is a main interface in my setup, and has been updated to only evaluate entities upon their change, instead of constantly checking all entities…

lastly, Ive been reporting unavailable’s upon Lovelace refresh for ever since Lovelace was introduced, and been told that was impossible.
I am glad I insisted it was the case, as Balloob has just recently added a new tweak that prevents this, and finally it was acknowledged…albeit in a completely unexpected cause.

that wont fix HA not being able to control them when they are unavailable, it simply makes the HA interface show the state the lights were in when available before, as recorded in the Hub.
HA shows unavailable in 2 cases:

  • no communication between Hub and Ha (this would be a true error, caused by possibly many things)
  • no communication between Hub and lights (this would be regular operation indicating the light had no power, or is broken eg, but the Hub still communicates with Ha. Hue sets the ‘reachable’ attribute to false, which is taken into account in the Hue logic)
    I’ve asked the core dev team to relay that attribute into Hue core attributes, but was denied unfortunately. It does make a difference as you can see above, and would be really nice if we could distinguish between the 2 situations. Right now we cant see if we have an HA-issue with the Hub, or if a light simply has no power.

In both case the lights can’t be controlled of course because they are unreachable…
check the source-code

have a look, my Garden terrace light is cut off power (because of a light sensor) and is reporting to be On with 50% brightness. Which of course it isn’t, but its the state the Hue Hub records it to be, and that is being reported back to Ha because of the allow_unreachable: true :

before you ask: I have a command_line sensor for my outside lights to reveal the reachable attribute so I can use the in logic for the buttons:

  - platform: command_line
    name: Hue lights config
    command: !secret hue_lights_command
    value_template: >
      {{value_json|length}}
    json_attributes:
      - '10' #mobile
      - '13' #driveway
      - '16' #garden_terrace
      - '17' #garden_backyard
      - '41' #gate_outdoors
      - '42' #porch_outdoors
      - '50' #bureau
      - '53' #backdoor_outdoors
    scan_interval: 300

I’ve set it to update each 5 minutes, as not to ‘Hammer the Hub’

correct, these are extra detailed settings we need to set in the config, as explained here

Note: you can however act on the Hub representation of the light, so upon next connection that state is set to the light. Eg, switching my Garden Terrace light ‘off’ now (while actually being unreachable) this happens in the Ha/Lovelace frontend:

Schermafbeelding 2020-04-21 om 09.49.28

the light is still Unreachable, but now is switched ‘Off’, and has no brightness available (should probably adjust that in the button template, but for now illustrates perfectly what is going on.)

So this appears to still be an issue as I’ve been seeing it for the last couple of months (currently on 111.4) - Are other folks still seeing this?

Both the HA (running HassOS on an Rpi) and the Philips Hue are wired into the same switch, and as others are reporting above, the Hue app will see the lights as available when the issue occurs.

image

image

I’m having some big issues at the moment with the entire Hue hub losing links to HA every 2 minutes.
Both HA and Hue ethernet cable linked to the same router so not wifi issue.
I’m going to start diagnosing tonight but any advice on where to start appreciated.

Just a thought but “Driveway light changed to unavailable” doesn’t necessarily mean Home Assistant is disconnecting from the Philips Hue Bridge. The status unavailable can also mean the Hue Bridge has lost communication with the light (i.e. Zigbee communication issue).

there are about 85 posts of suggestions in this thread alone… also, search for the combination Hue and Unavailable on this community, or in the Github issues. You’ll need some time consuming those.

Since 110 though, this has improved hugely. Every once in a while the lights still go unavailable, mostly because of the system is having momentary internal timing issue, caused by other integrations/internals. Depending on your system, try to minimize processor intensive processes. Have History and Logbook only log what you need and exclude all others. Try that first and see if this helps.

It practically never is the Hub losing communication with the lights. That is rock solid, and easy to check, in the App. The Hue app that is…

1 Like

Thanks Taras. Every hue light disconnects at the same time and reconnects at the same time but your point noted. thanks

Thanks Marius. appreciated.

check your sensors, they will be unavailable too. This is 100% certain HA losing connection to the Hub.

Yeah - I think that’s different than what others in the thread have been seeing.

After reading this and other threads, I was thinking it was load related on my Rpi3, so I went ahead and bought a Mini PC. I migrated a snapshot over to it last night and we’ll see if I notice any disconnects.

You’re right, that does sound like a LAN connection issue. To confirm, I would ping the Hue Bridge (from the Home Assistant Server) when the issue occurs. If ping fails then it points to a LAN disconnection. If ping succeeds then there’s a Zigbee issue.

Nope, its not a Lan issue, it’s a Home Assistant (- Hue) issue. HA polls the Hub. Only has a limited time frame for that, and if that is exceeded (because of timing issues on the HA instance) it throws unavailable. It’s a long standing problem, caused by the fact Hue doesn’t alert on changes, but needs to be polled by HA.

As said, it should be much better now. On my Rpi4 and fully maxed out Hue Hub, I almost never see these unavailable anymore.

I did take extreme measures though, on anything. Guard all templates, logger, history, etc etc. Take out almost all Rest sensors, and the ones left, set a large timeframe for them. Separate my Zwave, and Mqtt in to separate Rpi’s to not have that traffic cause lockups, etc etc.

What you described sound plausible if the RPi is at 100% CPU load. If not, I don’t see how the integration fails to get its allotted share of execution time.

well, all I can say is ‘join the club’ :wink:

btw, it’s not that the integration doesn’t get its allotted execution time, it’s a hard limit set in the Ha Hue core code within the polling should return. If not, it returns ‘unavailable’. it’s all in the source code, and heavily debated.

set logger to

homeassistant.components.hue: debug

and you’ll see when it’s happening.


I’ve asked to split the logic on Api attribute ‘reachable’ (Hue communicating with Light) and ‘unavailable’ (HA communicating with Hue hub) but Paulus didn’t want that. Nor was it accepted to add the attribute itself. While it is very handy to have that, on lights that don’t have power. It would show a true unreachable (Hub cant see the light) and wouldn’t have to show ‘Unavailable’ (HA/Hue comm error)…

We can only hope Hue will one day change its api to a push, so HA won’t need to poll the Hub any longer. Since that is very unlikely, (we’ve all added to the Hue developers community and asked for that) all we can do is optimize the HA instance as much as possible.

Heck, I’ve even created a sensor to check for the Hub not being overloaded.

As said, it has been very reliable lately, and fingers crossed that will stay.

1 Like

FWIW, I occasionally read another community forum for a home automation product developed by one person for over ten years. This individual is a very experienced software developer and I now recall he was dumbfounded by how a company with all of Philips’ resources chose to create a polling-based API.

Is there any way to pinpoint the top 2 or 3 of these that might have helped you? Watching my new migrated instance on a Mini PC that has almost no load, and it’s still happening.

image

image

The mind-boggling thing is that this setup has been fine for YEARS, even on an RPi2. I can’t really remember when it started, but it’s within the last month or two.

I might start removing integrations one at a time just to isolate it, though I don’t really have a whole lot of them to begin with.

FWIW, I have a test instance of Home Assistant OS on an RPI3. Using the SSH Add-on, when I ping my Philips Hue Bridge it returns sub-second responses.

Screenshot from 2020-06-26 13-20-41

Unless your DNS has an entry for the Hue Bridge, you’ll have to ping it using its IP address.

The response times from my production server (running on an old laptop) are slightly faster (under a half-second). All devices are connected to the same 10/100 switch.

Only my production server communicates with the Hue Bridge and I haven’t noticed incidents where all Hue devices become unavailable (I currently only have a half-dozen Hue Color bulbs).

anything else you added? or changed in the HA setup?

start with the recorder and logbook. simply only include what you need, and exclude anything else, including events.

exclude:
  event_types:
    - service_removed
    - service_executed
    - platform_discovered
    - homeassistant_start
    - homeassistant_stop
    - feedreader
    - service_registered
    - call_service
    - component_loaded
    - logbook_entry
    - system_log_event
    - automation_triggered
    - script_started
    - timer_out_of_sync

  domains:
    - alert
    - automation
    - binary_sensor
    - camera
    - climate
    - counter
    - customizer
    - device_tracker
    - group
    - input_boolean
    - input_datetime
    - input_number
    - input_select
    - input_text
    - light
    - media_player
    - proximity
    - scene
    - sensor
    - script
    - sun
    - switch
    - persistent_notification
    - person
    - remote
    - timer
    - updater
    - variable
    - weather
    - zone