All my hue lights randomly becoming unavailable

Can you share your Python script ? Becauseif more people are having performance issues with their python scripts with just normal commands, we might need to evaluate that component and see where things go wrong.

sure, id be glad to.

What im trying to figure out is which causes what: the automation is triggered by changes in the lights, which also happens when the hue lights become unavailable and back again. so a possible loop of events.

Only happening on the Hue lights. Half of the lights are Tradfri, and there is zero issue there. They are checked by the same 2 scripts and same automations.
Hope this helps.

please see below:

whats_on.py:

##########################################################################################
# Whats_on.py, previously known as Anything_on...
#
# interrelated with the Summary.py, which doesn't show the entities per badge (yet)
# modified and customized by https://community.home-assistant.io/u/Mariusthvdb/summary
# big hand in Python by https://community.home-assistant.io/u/petro/summary
# tbd: timezone correction a bit of a challenge, solved with a little hack (=2)
##########################################################################################

familyEntities = 'group.family'
switchEntities = 'group.iungo_switch_switches_template'
lightEntities = 'group.all_lights_only'
applianceEntities = 'group.iungo_switch_appliances_template'

filteredLightEntities = [ entity_id for entity_id in lightEntities if ' ' not in entity_id ]

def CountMyEntities(hass, entity_list, entity_state):
    cnt = 0

#  using a full list:  for entity_id in entity_list:
#  using groups in a [list] :  for group_entity in entity_list:
#   for entity_id in hass.states.get(group_entity).attributes['entity_id']:
    for entity_id in hass.states.get(entity_list).attributes['entity_id']:
        state = hass.states.get(entity_id)
        if state.state == entity_state:
            cnt = cnt + 1
    return cnt

familyHome = CountMyEntities(hass, familyEntities, 'home')
switchesOn = CountMyEntities(hass, switchEntities, 'on')
appliancesOn = CountMyEntities(hass, applianceEntities, 'on')
lightsOn = CountMyEntities(hass, lightEntities, 'on') #if needed, use filteredLightEntities

totalOn = switchesOn + lightsOn + appliancesOn
whichIcon = 'mdi:lightbulb-on-outline' if totalOn else 'mdi:lightbulb-outline' 
status = 'on' if totalOn else 'off' # will evaluate "off" if totalOn == 0.

hass.states.set('input_boolean.whats_on', status, { 
    'friendly_name': "What's On?",
    'icon': whichIcon,
    'family_home': familyHome,
    'lights_on': lightsOn,
    'switches_on': switchesOn,
    'appliances_on': appliancesOn,
    'total_on': totalOn,
    'extra_data_template':'{total_on} on'
})

##########################################################################################
# Lights:
# Badges images: /config/www/lights
##########################################################################################

lights_on = []
#show only lights, not light groups
#excluded = ['light.custom_group_for_group','light.custom_group_for_lights_2']

#total_lights_on = hass.states.get('input_boolean.whats_on').attributes.get('lights_on')
state = hass.states.get('automation.sense_lights_change')

if state:
    total_on = hass.states.get('input_boolean.whats_on').attributes.get('lights_on')
    for entity_id in hass.states.get(lightEntities).attributes['entity_id']:
        dt = state.last_updated + datetime.timedelta(hours=2)
        time = '%02d:%02d' % (dt.hour, dt.minute)

        if hass.states.get(entity_id).state is 'on': #and entity_id not in excluded
            lights_on.append(hass.states.get(entity_id).attributes['friendly_name'])

    if len(lights_on) > 0:
        picture = '/local/lights/hue_pl.png'
        message = ', '.join(lights_on)
        sensor_message = 'Lights on: ' + message
        lights_desc = '=- Lights on: {} -- {} since {}'.format(total_on, message,time)
        uom = 'Lights'

        if len(lights_on) == 1:
            uom = 'Light'
    else:
        picture = '/local/lights/bulboff.png'
        message= ''
        sensor_message= 'No lights on'
        uom = 'Lights'

    sensor_lights_desc = '{}'.format(sensor_message)

    hass.states.set('sensor.lights_badge', total_on, {
#        'custom_ui_state_card': 'state-card-value_only',
        'text': sensor_message,
        'unit_of_measurement': uom,
        'friendly_name': time,
        'entity_picture': picture
         })

##########################################################################################
# Switches:
# Badges images: /config/www/buttons
##########################################################################################

switches_on = []

#total_switches_on = hass.states.get('input_boolean.whats_on').attributes.get('switches_on')
state = hass.states.get('automation.sense_switches_change')

if state:
    total_on = hass.states.get('input_boolean.whats_on').attributes.get('switches_on')
    for entity_id in hass.states.get(switchEntities).attributes['entity_id']:
        dt = state.last_updated + datetime.timedelta(hours=2)
        time = '%02d:%02d' % (dt.hour, dt.minute)

        if hass.states.get(entity_id).state is 'on':
            switches_on.append(hass.states.get(entity_id).attributes['friendly_name'])

    if len(switches_on) > 0:
        picture = '/local/buttons/button_power_on.png'
        message = ', '.join(switches_on)
        sensor_message = 'Switches on: ' + message
        switches_desc = '#- Switches on: {} -- {} since {}'.format(total_on, message,time)
        uom = 'Switches'

        if len(switches_on) == 1:
           uom = 'Switch'
    else:
        picture = '/local/buttons/button_power_off.png'
        message= ''
        sensor_message= 'No switches on'
        uom = 'Switches'

    sensor_switches_desc = '{}'.format(sensor_message)

    hass.states.set('sensor.switches_badge', total_on, {
#        'custom_ui_state_card': 'state-card-value_only',
            'text': sensor_message,
            'unit_of_measurement': uom,
            'friendly_name': time,
            'entity_picture': picture
         })

##########################################################################################
# Appliances:
# Badges images: /config/www/buttons
##########################################################################################
#appliancesGroup = 'group.iungo_switch_appliances_template'
appliances_on = []

#total_appliances_on = hass.states.get('input_boolean.whats_on').attributes.get('appliances_on')
state = hass.states.get('automation.sense_appliances_change')

if state:
    total_on = hass.states.get('input_boolean.whats_on').attributes.get('appliances_on')
    for entity_id in hass.states.get(applianceEntities).attributes['entity_id']:
        dt = state.last_updated + datetime.timedelta(hours=2)
        time = '%02d:%02d' % (dt.hour, dt.minute)

        if hass.states.get(entity_id).state is 'on':
            appliances_on.append(hass.states.get(entity_id).attributes['friendly_name'])

    if len(appliances_on) > 0:
        picture = '/local/buttons/button_power_on.png'
        message = ', '.join(appliances_on)
        sensor_message = 'Appliances on: ' + message
        uom = 'Appliances'

        if len(appliances_on) == 1:
            uom = 'Appliance'
    else:
        picture = '/local/buttons/button_power_off.png'
        message= ''
        sensor_message= 'No appliances on'
        appliances_desc = '!|-> No appliances on since {}'.format(time)
        uom = 'Appliances'

    sensor_appliances_desc = '{}'.format(sensor_message)

    hass.states.set('sensor.appliances_badge', total_on, {
    #        'custom_ui_state_card': 'state-card-value_only',
            'text': sensor_message,
            'unit_of_measurement': uom,
            'friendly_name': time,
            'entity_picture': picture
         })

summary.py (slightly bigger):

##########################################################################################
# Resources:
# https://github.com/maattdiy/home-assistant-config @Mviezzer
# https://home-assistant.io/components/python_script/
# https://home-assistant.io/docs/configuration/state_object/
# modified and customized by https://community.home-assistant.io/u/Mariusthvdb/summary
# big hand in Python by https://community.home-assistant.io/u/petro/summary
##########################################################################################

debug = False

if debug:
    event = data.get('event')
    logger.error("\n\nSUMMARY: " + str(event))

show_badges = True
show_card = True

##########################################################################################
# Group count
# Groups config:
# Entities summary by group name 
# groups = need to have same max member_count
# groups_format = # Message prefix
# groups_filter = # Filter to list
# groups_badge = # Badge 'belt' (unit_of_measurement)
# groups_badge_pic = # none, one, or a list of pictures (picture position = match the count)
# groups_min_show = # Mininum count to show
# groups_theme =  # Theme template
# groups_desc = # Can set the default description, for use in case count = 0
# themes build up or down according to counted value
##########################################################################################
group_count = 0
group_desc = ''
summary = ''
idx = 0
dt_prevstate = None

on ='black_badge|red_badge|yellow_badge|grey_badge|blue_badge|orange_badge|brown_badge|purple_badge|green_badge'

off ='green_badge|purple_badge|brown_badge|orange_badge|blue_badge|grey_badge|yellow_badge|red_badge|black_badge'

groups = ['group.family',
          'group.hubs_binary_pinged',
          'group.critical_devices_state',
          'group.media_player_media',
          'group.device_tracker_media',
          'group.all_lights_only',
          'group.iungo_switch_switches_template',
          'group.iungo_switch_appliances_template',
          'group.binary_sensors_active_template']

groups_format = ['+- Home: {} -- {}',
                 '!|-> Offline: {} -- {}',
                 '!|-> Status - Critical device offline: {} -- {}',
                 '#- Playing: {} -- {}',
                 '#- Tracked: {} -- {}',
                 '=- Lights on: {} -- {}',
                 '#- Switches on: {} -- {}',
                 '#- Appliances on: {} -- {}',
                 '+- Active: {} -- {}']

groups_filter = ['home',
                 'off|idle|not_home', 
                 'off|idle|unknown|not_home',
                 'home|playing|on',
                 'home|playing|on',
                 'on',
                 'on',
                 'on',
                 'on'] # was: 'home|playing|on'

groups_badge = ['Home','Hubs','Status','Play','Track','Lamps','Switch','Appli','Active']

groups_badge_pic = ['','ok|bug|critical','ok|bug|critical','','','','','','']

groups_min_show = [0,0,0,0,0,0,0,0,0]

groups_theme = [on,off,off,on,on,on,on,on,on]

groups_desc = ['!|-> Nobody home since ',
               '+- Hubs: all online',
               '+- Status: all online',
               '%|-> Nothing playing',
               '%|-> Nothing tracked',
               '!|-> No lights on since ',
               '!|-> No switches on since ',
               '!|-> No appliances on since ',
               '%|-> No Appliances active since ']

groups_count = [0, 0, 0, 0, 0, 0, 0, 0, 0]

for group in groups:
    group_count = 0
    group_desc = ''

    for entity_id in hass.states.get(group).attributes['entity_id']:
        state = hass.states.get(entity_id)
        filter = groups_filter[idx]

        if (state.state in filter.split('|') or debug):
            dt = state.last_updated + datetime.timedelta(hours= 2)
            time = '%02d:%02d' % (dt.hour, dt.minute)
            ch = state.last_changed + datetime.timedelta(hours= 2)
            ch_time = '%02d:%02d' % (dt.hour, dt.minute)
          # If state changed in the past days show the date too
            if dt.date() < datetime.datetime.now().date():
                time = '{} {}'.format('%02d/%02d' % (dt.day, dt.month), time)

            group_count = group_count + 1
            group_desc = '{} {} ({}), '.format( group_desc, state.name, time)

        else:
            if (dt_prevstate is None):
                dt_prevstate = state.last_changed
            else:
                if (not state.last_changed is None):
                    if (state.last_changed > dt_prevstate):
                        dt_prevstate = state.last_changed

    # Final format for this group
    if (group_count >= groups_min_show[idx]):
        if (group_count == 0):
            group_desc = groups_desc[idx]
           # If there is none 'On/Home' state in group, show since
            if (group_desc.find(' since ') > 0): #if (group_desc != ''):
                dt = dt_prevstate + datetime.timedelta(hours= 2)
                group_desc = '{} {}'.format(group_desc, '%02d:%02d' % (dt.hour, dt.minute))

        else:
            group_desc = groups_format[idx].format(group_count, group_desc[:-2])

        groups_desc[idx] = group_desc
        groups_count[idx] = group_count

    idx = idx + 1

##########################################################################################
# Badges updates
##########################################################################################

idx = 0

if show_badges:
    for badge in groups_badge:
        if (badge != ''):
            entity_id = 'sensor.{}_badge'.format(badge.replace(' ', '').lower());
            hidden = False if (groups_count[idx] >= groups_min_show[idx] or debug) else True
            fname = groups_desc[idx] if debug else ' '
            picture = groups_badge_pic[idx].replace(' ', '').lower()

            themelist = groups_theme[idx].split('|')
            if len(themelist) > 1:
                try:
                    theme = themelist[groups_count[idx]]
                except IndexError:
                    theme = 'green_badge' # Oops, error badge.1 ;-))
            else:
                theme = 'black_badge' # this will be the default theme

            # Check for picture X index/count
            if (picture != ''):
                picturelist = picture.split('|')
                if (len(picturelist) in [1, groups_count[idx]]):
                    picture = picturelist[-1] #was mylist[len(mylist)-1]
                else:
                    picture = picturelist[groups_count[idx]]

                if (picture != ''):
                    picture = '/local/badges/{}.png'.format(picture)

            hass.states.set(entity_id, groups_count[idx], {
              'friendly_name': fname,
              'unit_of_measurement': badge,
              'entity_picture': picture,
              'templates': { 'theme': theme }
            })

        idx = idx + 1

##########################################################################################
# Alarm clock
# https://github.com/maattdiy/home-assistant-config/blob/master/config/packages/alarmclock.yaml
##########################################################################################

alarms_prefix = ['alarmclock_wd', 'alarmclock_we']
alarms_wfilter = [range(1,6), range(6,8)]
alarms_desc = ''
mydict = {'hidden':hidden}

for entity_id, filter in zip(alarms_prefix, alarms_wfilter):
    state = hass.states.get('input_boolean.{}_enabled'.format(entity_id))
    if state:
        if state.state is 'on' and datetime.datetime.now().isoweekday() in filter:
            state = hass.states.get('sensor.{}_time_template'.format(entity_id))
            alarms_desc = '{}{}, '.format(alarms_desc, state.state)

if (alarms_desc == ''):
    mydict['entity_picture'] = '/local/badges/alarm_off.png'
    mydict['friendly_name'] =  'Relax'
    mydict['unit_of_measurement'] = 'Off'
    mydict['theme'] = 'grey_badge'
    alarms_desc = '%- Alarm clock is not set, relax'
else:
    mydict['entity_picture'] = '/local/badges/alarm.png'
    mydict['friendly_name'] = alarms_desc[:-2]
    mydict['unit_of_measurement'] = 'On'
    mydict['theme'] = 'orange_badge'
    alarms_desc = '/- Alarm clock set at ' + alarms_desc[:-2]

hass.states.set('sensor.alarms_badge', '', mydict)

##########################################################################################
# Mode:
# Inspired by: https://github.com/maattdiy/home-assistant-config/blob/master/config/packages/profiles.yaml
#              https://github.com/maattdiy/home-assistant-config/blob/master/config/packages/developer.yaml
# Badges images: /config/www/modes
##########################################################################################

mode_desc = ''
## Get Mode description
state = hass.states.get('input_select.mode')

if state:
    dt = state.last_changed + datetime.timedelta(hours= 2)
    time = "%02d:%02d" % (dt.hour, dt.minute)

    mode_desc = '{}*- {} mode selected at: {}\n'.format(summary, state.state, time)

    hass.states.set('sensor.mode_badge',state.state, {
         'entity_picture': '/local/modes/{''}.png'.format(state.state.replace(' ','').lower()),
         'friendly_name': time,
         'unit_of_measurement': 'Mode'
          })

##########################################################################################
# Activity:
# Badges images: /config/www/activities
##########################################################################################

activity_desc = ''
## Get Activity description
state = hass.states.get('input_select.activity')

if state:
    dt = state.last_changed + datetime.timedelta(hours= 2)
    time = "%02d:%02d" % (dt.hour, dt.minute)

    activity_desc = '{}$- {} activity selected at: {}\n'.format(summary, state.state, time)

    hass.states.set('sensor.activity_badge', state.state, {
            'friendly_name': time,
            'entity_picture': '/local/activities/{' '}.png'.format(state.state.replace(' ','').lower()),
            'unit_of_measurement': 'Act'
            })

##########################################################################################
# Summary update
# Custom card: /custom_ui/state-card-value_only.html
##########################################################################################

for group_desc in groups_desc:
    if group_desc != '' and not group_desc.endswith(': '):
        summary = '{}{}\n'.format(summary, group_desc)

# Add to final summary
summary = '*============== Status ============\n' \
          '{}\n' \
          '*============= Settings ===========\n' \
          '{}\n' \
          '*============== Modes =============\n' \
          '{}\n' \
          '{}' \
          .format(summary, 
                  alarms_desc, 
                  activity_desc, 
                  mode_desc)

if show_card:
    hass.states.set('sensor.summary', '', {
        'custom_ui_state_card': 'state-card-value_only',
        'text': summary
    })

##########################################################################################
# Codes for text_colors declared in 
# Custom card: /custom_ui/state-card-value_only.html
# changed to use below customizing options:
##########################################################################################

#      case "*": return "bold";
#      case "/": return "italic";
#      case "!": return "red";
#      case "+": return "green";
#      case "=": return "yellow";
#      case "%": return "grey";
#      case "$": return "brown";
#      case "#": return "blue";
#      default:  return "normal";

Hey all, @balloob

back again, after extensive testing, both on Hassio and Hassos, I can report that the flapping issue seems to be caused by several ‘listeners’, rather than the python_script.

If I run the python scripts per time period, now set to each 15 seconds, the flapping stops, and responses of the Hue system seem alright. On the Hassos system, where none of these scripts is installed, but the Hue Custom component is, Hue system is dead right, and no issues arise.

That being said, one needs to be able to trigger from light state changes now and then to automate the Home, and thats when the issues arise. I have tested the following automations, which in them selves works fine, but cause the Hue system to go flapping, ie, show the lights as unavailable for a certain time, and then come back again. Also the disconnecting to the Hue ip address is shown in the logs.

Automations are:

based on event call_service in domain light:

- alias: 'Call Service Event Light'
  id: 'Call Service Event Light'
  trigger:
    platform: event
    event_type: call_service
    event_data:
      domain: light
  action:
    - delay: 00:00:02
    - service: python_script.whats_on
    - delay: 00:00:02
    - service: python_script.summary

based on individual light state (with added condition not being in home mode party, because thats another listener to the lights):

- alias: 'Sense Lights change'
  id: 'Sense Lights change'
#  hide_entity: True
#  initial_state: 'on'
  trigger:
    platform: state
    entity_id:
#Hue lights
      - light.dining_1
      - light.table_lamp_1
      - light.table_lamp_2
      - light.table_lamp_3
      - light.5
      - light.6
      - light.home_theater
      - light.frontdoor
      - light.9
      - light.10
      - light.hue_go_1
      - light.driveway
#Tradfri lights
      - light.chair_long
      - light.drawer
      - light.hobby_tafel
      - light.tv_backlight
      - light.bureau_left
      - light.bureau_right
      - light.inside
      - light.outside
      - light.pantry
  condition:
    condition: state
    entity_id: input_boolean.home_mode_party
    state: 'off'
  action:
    - delay: 00:00:02
    - service: python_script.whats_on
    - delay: 00:00:02
    - service: python_script.summary

and one on state change event, with template selecting the appropriate lights, rather a short and elegant automation:

- alias: 'State changed Light'
  id: 'State changed Light'
  trigger:
    platform: event
    event_type: state_changed
  condition:
    condition: template
    value_template: >
      {{ trigger.event.data.entity_id in state_attr('group.all_lights_only','entity_id') }}
  action:
    - delay: 00:00:02
    - service: python_script.whats_on
    - delay: 00:00:02
    - service: python_script.summary

There is some difference in system response, with the latter being the most robust, or better said least causing issues.
Still, they all are causing the Hue system to disconnect and reconnect after a short while, making this an unreliable source for further Home-automation ( because after reconnecting all last_changed attributes restart, and after disconnecting the condition for a light being in a certain state is not valid anymore) . which is what we all are here for aren’t we.

That the above is only happening in the Hue system implementation could be established by this automation, checking only my Tradfri lights:

- alias: 'State changed Ikea Light'
  id: 'State changed Ikea Light'
  trigger:
    platform: event
    event_type: state_changed
  condition:
    condition: template
    value_template: >
      {{ trigger.event.data.entity_id in state_attr('group.ikea_tradfri_lights','entity_id') }}
  action:
    - delay: 00:00:02
    - service: python_script.whats_on
    - delay: 00:00:02
    - service: python_script.summary

Immediate response, no hubs disconnecting.

Somehow the Hassio - Hue system (integration) doesn’t like listeners

Hope this helps in further exploring this issue, and be glad to help in any way.
before you ask: of course I did not run these automotations simultaneously…

Cheers,
Marius

I thought I had gotten past this as well. Running 75.3 now. Created this automation the other day. Worked great for one day, then this morning, it was stuck, could not get it to work, took me a few minutes to realize that sure enough, several of my hue bulbs were unavailable. I’ll double check, but I don’t have many conditions based on hue, I have several triggers based on other things, that have actions with Hue (pretty standard stuff). Definitely no python scripts to check status.

- alias: Set alcove light color based on Traffic to work
  trigger:
  - platform: state
    entity_id: input_select.house_mode
    from: 'Night'
  - platform: state
    entity_id: sensor.person_traffic_density_to_work
  - platform: time
    at: '07:35:00'
  condition:
  - condition: state
    entity_id: binary_sensor.workday_sensor
    state: 'on'
  - condition: state
    entity_id: input_select.house_mode
    state: Home
  - condition: time
    after: '07:35:00'
    before: '08:35:00'
  action:
  - service: light.turn_on
    data_template:
      entity_id: light.alcove_light
      brightness_pct: 100
      color_name: >
        {% set map = {'Normal': 'green', 'Moderate': 'orange', 'Heavy': 'red'} %}
        {% set state = states('sensor.person_traffic_density_to_work') %}
        {{ map[state] if state in map else 'white' }}

This bulb and another bulb physically nearby, but not part of any recent automation were suddenly unavailable. Just passing along for a data point. I will see how this runs tomorrow. After restarting HA it worked as expected. Also I had restarted HA yesterday, so it was not up and running for days before this happened

Just wanted to add this too, doubt it’s related, but much of my lighting for hue (not the example above), is controlled by calling hue scenes with scripts. So morning, runs two scenes for two groups of lights, turn on tv after dark, runs a different set of scenes, come home after dark, etc. Anyone else with issues, calling hue scenes?

ive been digging some further, and found out that not only does the Hue integration suffer from lights related automations calling the python scripts, but other processor heavy automations (listeners) cause the Hue lights to go unavailable immediately and start the ‘flapping’

I have an automation that triggers on each automation being triggered (of course without being triggered by itself…or several other often triggered ones) and another automation triggered by state change of my power sensors.

enabling either of these throws the Hue into ‘flapping’.

And only the Hue hub, the rest of the system runs fine. This is a timing issue leading to frustration. I can prevent it from happening (well, almost, even without these automations and pythons the hue shows unavailable now and then) by disabling these automations. But, these automations are there to monitor the House and its main systems. Exactly what HomeAssistant should be doing.

Hope the dev’s can optimize the Hue integration (before the asyncio move this didnt happen) and stop this unstable Hue behavior.

these are the 2 automations.

- alias: Automation ran
#  hide_entity: True
  initial_state: 'off'
  trigger:
    platform: event
    event_type: state_changed
  condition:
    condition: template
    value_template: >
      {% set skip_list = ['automation_ran', 'count_warnings', 'count_errors',
                           'activate_map_sensors_actueel'] %}
      {{ trigger.event.data.entity_id.startswith('automation.') and
         trigger.event.data.entity_id.split('.')[1] not in skip_list and
         'old_state' in trigger.event.data and 'new_state' in trigger.event.data }}
  action:
    - condition: template
      value_template: >
        {{ trigger.event.data.new_state.attributes.last_triggered !=
           trigger.event.data.old_state.attributes.last_triggered }}
    - service: notify.filed_automations
      data_template:
        message: >
         {{ as_timestamp(trigger.event.data.new_state.last_updated)|timestamp_custom("%d %b: %X") }}: {{ trigger.event.data.new_state.name }}
    - service: python_script.last_automation
      data_template:
        event: '{{ trigger.event }}'

- alias: 'Activate Map sensors actueel'
  id: 'Activate Map sensors actueel'
#  hide_entity: True
  initial_state: 'off'
  trigger:
    platform: event
    event_type: state_changed
    event_data:
      domain: sensor
  condition:
    condition: template
    value_template: >
      {{ trigger.event.data.entity_id.endswith('actueel')}}
  action:
    service: python_script.map_sensors_actueel
1 Like

I’m running 76.2 and am still having this issue from time to time. Recently I added a new automation that uses one of my hue bulbs and have noticed that the issue is much, more common now. To be fair, I’m not always looking and the UI, or logs, so It’s possible they are “unavailable” but on other times during the day.

Here’s my usual Hue light control. Wake up in the morning, turn on a couple of hue light strips. Leave the house, turn off all lights (inlcuding those strips). Come home turn on the light strips again, if its dark, also turn on a couple other strips and a bulb. Turn on TV after dark, change the color/brightness of a some of the strips. Turn off the TV, change brigtness/etc. all of the lights are changed by calling Hue Scenes via a script. The lights at night are 99% stable. I cannot remember the last time they did not work.
The lights in the morning used to be stable, until I added this recent automation. It is more “intensive”. I’ll post in a second. Just to be clear, and to avoid useful recommendations, I’ll add that this has not always been an issue, its not my wi-fi, or channel, etc. Been there done that.

So here is what happens in the morning, and this is what has seemed to be causing my lights to become unavailable

- alias: Good Morning Routine
  trigger:
  - platform: state
    entity_id: binary_sensor.motion_sensor_living_room_motion
    from: 'off'
    to: 'on'
  - platform: state
    entity_id: binary_sensor.motion_sensor_kitchen_motion
    from: 'off'
    to: 'on'
  condition:
  - condition: state
    entity_id: input_select.security_system
    state: 'Armed (Home)'
  - condition: time
    after: '05:30:00'
    before: '11:30:00'
  action:
  - service: script.turn_on
    entity_id:
      - script.ecobee_resume_mode
      - script.kitchen_evening_lights_hue_scene
  - service: lock.unlock
    entity_id: lock.lock_garage_door_lock
  - service: input_select.select_option
    data_template:
      entity_id: input_select.security_system
      option: Disarmed
  - service: input_select.select_option
    data_template:
      entity_id: input_select.house_mode
      option: Home

Here is the script for the scene
kitchen_evening_lights_hue_scene:
  alias: Hue Scene Evening Lights (Kitchen)
  sequence:
    - service: hue.hue_activate_scene
      data:
        group_name: "Kitchen"
        scene_name: "Evening Lights"

Additional Automations
- alias: Set alcove light color based on Traffic to AVIS
  trigger:
  - platform: state
    entity_id: input_select.house_mode
    from: 'Night'
  - platform: state
    entity_id: sensor.tracys_traffic_density_to_avis
  - platform: time
    at: '06:35:00'
  condition:
  - condition: state
    entity_id: binary_sensor.workday_sensor
    state: 'on'
  - condition: state
    entity_id: input_select.house_mode
    state: Home
  - condition: time
    after: '06:35:00'
    before: '07:34:00'
  action:
  - service: light.turn_on
    data_template:
      entity_id: light.alcove_light
      brightness_pct: 100
      color_name: >
        {% set map = {'Normal': 'green', 'Moderate': 'orange', 'Heavy': 'red'} %}
        {% set state = states('sensor.tracys_traffic_density_to_avis') %}
        {{ map[state] if state in map else 'white' }}

- alias: Set alcove light color based on Traffic to Base
  trigger:
  - platform: state
    entity_id: input_select.house_mode
    from: 'Night'
  - platform: state
    entity_id: sensor.pauls_traffic_density_to_base
  - platform: time
    at: '07:35:00'
  condition:
  - condition: state
    entity_id: binary_sensor.workday_sensor
    state: 'on'
  - condition: state
    entity_id: input_select.house_mode
    state: Home
  - condition: time
    after: '07:35:00'
    before: '08:35:00'
  action:
  - service: light.turn_on
    data_template:
      entity_id: light.alcove_light
      brightness_pct: 100
      color_name: >
        {% set map = {'Normal': 'green', 'Moderate': 'orange', 'Heavy': 'red'} %}
        {% set state = states('sensor.pauls_traffic_density_to_base') %}
        {{ map[state] if state in map else 'white' }}

Additional details. Looking through the logs for time stamps as I was writing this, I see the three lights in my kitchen hue group, starting going unavailable, available at 5am exactly. Which does not seem to tie into this after all, but it is weird that it’s just this group of lights. The other 4-5 hue lights in different groups did not go unavailable at all during that time.

Well, I guess I believed the traffic automation was the cause, but I think that it just brought it more to my attention. It is weird that it is only one hue group though.

just a heads-up the dev’s have now enabled debug on the hue component to see if updating causes these disconnects. Ive posted some of my findings on the Github issue:

please have a look if you could do so too, so the dev’s have as much useful info as possible to be able to mitigate the issue here.
thx

1 Like

binary sensors ping (and mqtt) causing Hue unavailable? please see https://github.com/home-assistant/home-assistant/issues/16689#issuecomment-442094226 and check if this would help in your setups.

cheers.

Is there any update on this? I started to have the same problem yesterday when couple of my Hue lights started becoming unavailable suddenly. Noticed this when one of my automations was not working even when HA showed that it triggered normally. And when investigating I noticed that everything works but some of the lights are becoming unavailable and then suddenly available again.

This started when I added device_tracker and couple of automations that use that.

@Mariusthvdb are you still suffering from this or did you solved it by changing the python script and/or binary_sensor ping?

According to the Github issue, someone did a reset on their Hub and it started working. I actually plan on doing this since there are so many useless scenes etc. created by 3rd party andoird and iOS apps but don’t know if that is going to help with this.

Edit: Sorry for tagging you Mariusthvb

no this still prevails, and reseting the hub wont work…
there are many many posts on this.
that being said, I have 2 instances and the one which has my mqtt broker on it (the hassio add-on) suffers most. I can not have any automation rely on lights state, because of that.
It apparently has to do with I/O going on. And we must see the issue as a signal of something else in the system taking processor time/goin wrong. thats what the HA founder told me.

If you add show_last_changed to the lights, you can exactly see for how long they have been seen by the system.
On my Mqtt system that mostly is about a few minutes, and if I am lucky, several 10’s of minutes up to an hour.
On my 2nd system, which I use to show in the app and use as a front-end, light are available most of the time. But reloading Lovelace heavy cards (weather custom cards etc) might again cause Unavailable.

So long answer short: its still there.

see below, showing the same lights/groups in my

mqtt system:

23

and main system:

35

Well this is too bad. It is actually kind of breaking thing and might move people away from Home Assistant in the end if it is not fixed.

I can investigate this on my side but most likely nothing new and useful will be found :frowning:

As a final thing to try take a look here

Since I boosted the wifi signal in my house the lights haven’t shown as unavailable once.

Thanks, I will take that into account. Most likely that has nothing to do with my lights going unavailable since I have a powerful WIFI AP that is only a WIFI AP so it has pretty strong signal everywhere in my house. But this is still a good point to look :+1:

well, not ‘final’ as in this solved it…

this has been discussed before, as has changing the zigbee channels. It might solve some issues in particular for some users,because of their specific home settings. It isnt a solution to the issue in itself though.

It has to do with I/O on your local system and the Hue needing to be polled regularly, as it is implemented now, rather than receiving pushed changed from the Hue side. (like eg the Tradfri system does)

To stress the importance of this issue, we could however all hop over on the Hue developers forum and support @balloob’s call for a push API on the HUE system: https://developers.meethue.com/forum/t/2018-and-2019-hue-api-and-sdk-features/5996/21

I am having this issue also. I don’t think it is related to interference by WiFi, wouldn’t that make the lights unavailable on all platforms? I have eero’s mesh wifi and you can’t select channel in the config so I can’t really test that. My hue lights work perfectly with Amazon Echo, They work perfectly with the App and they work perfectly with two Flic Button Hubs I have on my system. Only HA has them bouncing between unavailable and active. I have two lights which are the worst and I have them automated to turn on when I arrive home and when a zigbee button is pressed to toggle. One or the other fails almost all the time. It’s rare that both are online at the same time.
I am on 0.81.0 but see no reason to upgrade for this issue since people are reporting same problem with 0.86.3 which is the current version as of this post.

Yeah, there seem to be two common causes of “unavailable”, but often people get them confused.

One possible common cause, is wifi/channel/network related, this will affect all apps, controllers, etc

The issue that this thread is trying to resolve is why HA is marking some lights as unavailable even when they seem to be available to other systems.

I will say that there has been a lot of back and forth, and no resolution. For me and many others my lights go unavailable one or two at a time for a minute or so, throughout the day. But 95% of the time it does not affect me as I dont turn my lights off and on every 5 minutes, etc. For people who have the hub going off line, or all lgihts going offline for a long period of time, you are probably dealing with the 1st issue.

Mine are offline much more often. Checking right now. Two are offline

stairs_top and stairs_bottom are offline every morning which is when I use them. I no longer expect them to work at all.

A few seconds later now 4 are offline

I pretty much can’t control Hue with HA anymore.

few seconds later and different 4 offline now.

Few seconds later different set again (still 4 offline).

Basically 2-4 of my 12 Hue lights are offline at any given moment.
I can just watch this panel and they rotate around continuously. I never see them all online.

I cannot say for sure, but to me it does sound like a wifi/channel/HA/network issue. Most of my issues (and several others like me), are an occasional drop out of a bulb at a time. During a normal day, even in the middle of the night when nothing is going on, I’ll see a light or two go unavailable, and then back available in the next minute.

Are you running on a RPI? Do you have a lot going on? One of the comments above seemed to point to the HA server being too busy to verify connectivity to the hue entities

This is the one I suspect. I am running on RPI but I have nothing else running on this RPI. I have MQTT on it’s own PI, I have several other RPI’s about doing various choirs.

I just tried disabling the recorder, I do have mysql running on this RPI so maybe that is an issue. I will disable the mySQL server and see if that helps. I am using an external drive NOT SD card so performance is pretty good on this RPI which is a 3b+

On a side note I found that my recorder has never purged. Documentation says default is to keep 10 days worth and purge every day. I have found 505,858 events in the DB with a min created date of Sept 6 up until today. Can I just delete all the data in each table to manually purge?

I still say if it was WiFi interference my other methods of connection would be failing. Alexa is 100% never fails.

UPDATE: disabled mysql cpu at 3% max lights still bouncing offline/online in HA only.