All my hue lights randomly becoming unavailable

Hi Marius,

It looks a bit different, i strongly believe your issue has a different nature as you conclude. Now again I turned on my pizero with presence detection (bluetooth scanning). Directly I have the issue below.

2018-07-04 21:43:25 ERROR (MainThread) [homeassistant.components.light.hue] Unable to reach bridge 10.0.1.3
2018-07-04 21:43:25 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.hue_ambiance_spot_4, old_state=<state light.hue_ambiance_spot_4=off; min_mireds=153, max_mireds=500, friendly_name=Hue ambiance spot 4, supported_features=43 @ 2018-07-04T21:38:37.857220+02:00>, new_state=<state light.hue_ambiance_spot_4=unavailable; friendly_name=Hue ambiance spot 4, supported_features=43 @ 2018-07-04T21:43:25.853328+02:00>>
2018-07-04 21:43:25 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.hue_ambiance_spot_5, old_state=<state light.hue_ambiance_spot_5=off; min_mireds=153, max_mireds=500, friendly_name=Hue ambiance spot 5, supported_features=43 @ 2018-07-04T21:38:37.844316+02:00>, new_state=<state light.hue_ambiance_spot_5=unavailable; friendly_name=Hue ambiance spot 5, supported_features=43 @ 2018-07-04T21:43:25.860572+02:00>>
2018-07-04 21:43:25 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.hue_white_lamp_2, old_state=<state light.hue_white_lamp_2=off; friendly_name=Hue white lamp 2, supported_features=41 @ 2018-07-04T21:38:37.881026+02:00>, new_state=<state light.hue_white_lamp_2=unavailable; friendly_name=Hue white lamp 2, supported_features=41 @ 2018-07-04T21:43:25.867617+02:00>>
2018-07-04 21:43:25 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.hue_ambiance_spot_1, old_state=<state light.hue_ambiance_spot_1=off; min_mireds=153, max_mireds=500, friendly_name=Hue ambiance spot 1, supported_features=43 @ 2018-07-04T21:38:37.887188+02:00>, new_state=<state light.hue_ambiance_spot_1=unavailable; friendly_name=Hue ambiance spot 1, supported_features=43 @ 2018-07-04T21:43:25.874279+02:00>>
2018-07-04 21:43:25 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.hue_white_lamp_3, old_state=<state light.hue_white_lamp_3=off; friendly_name=Hue white lamp 3, supported_features=41 @ 2018-07-04T21:38:37.892953+02:00>, new_state=<state light.hue_white_lamp_3=unavailable; friendly_name=Hue white lamp 3, supported_features=41 @ 2018-07-04T21:43:25.882457+02:00>>
2018-07-04 21:43:25 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.hue_ambiance_lamp_1, old_state=<state light.hue_ambiance_lamp_1=off; min_mireds=153, max_mireds=500, friendly_name=Hue ambiance lamp 1, supported_features=43 @ 2018-07-04T21:38:37.862976+02:00>, new_state=<state light.hue_ambiance_lamp_1=unavailable; friendly_name=Hue ambiance lamp 1, supported_features=43 @ 2018-07-04T21:43:25.890735+02:00>>
2018-07-04 21:43:25 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.hue_ambiance_spot_3, old_state=<state light.hue_ambiance_spot_3=off; min_mireds=153, max_mireds=500, friendly_name=Hue ambiance spot 3, supported_features=43 @ 2018-07-04T21:38:37.874820+02:00>, new_state=<state light.hue_ambiance_spot_3=unavailable; friendly_name=Hue ambiance spot 3, supported_features=43 @ 2018-07-04T21:43:25.901226+02:00>>
2018-07-04 21:43:25 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.hue_ambiance_spot_6, old_state=<state light.hue_ambiance_spot_6=off; min_mireds=153, max_mireds=500, friendly_name=Hue ambiance spot 6, supported_features=43 @ 2018-07-04T21:38:37.868975+02:00>, new_state=<state light.hue_ambiance_spot_6=unavailable; friendly_name=Hue ambiance spot 6, supported_features=43 @ 2018-07-04T21:43:25.914141+02:00>>
2018-07-04 21:43:25 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.hue_ambiance_spot_2, old_state=<state light.hue_ambiance_spot_2=off; min_mireds=153, max_mireds=500, friendly_name=Hue ambiance spot 2, supported_features=43 @ 2018-07-04T21:38:37.851177+02:00>, new_state=<state light.hue_ambiance_spot_2=unavailable; friendly_name=Hue ambiance spot 2, supported_features=43 @ 2018-07-04T21:43:25.924260+02:00>>
2018-07-04 21:43:28 INFO (MainThread) [homeassistant.components.light.hue] Reconnected to bridge 10.0.1.3

ok, but thats another possible interfering protocol: bluetooth scanning, as reported by others indeed. Thats why I have already disabled the bluetooth add-on…

no conclusions yet, Im still trying to figure out if the Hub causes the timing issues, or if the timing issues cause the Hub to make the lights unavailable …

ok, with regards to the log, which log did you dump in you previous post? as the layout looked quite smooth and organised, or is that the way you posted it?

Maybe it would be nice to know the logs from the other people look like, just to get a better understanding of the issue in general.

simply the home-assistant.log…

this is the current debug setting for it:

# https://home-assistant.io/components/logger/
# Possible log severities: critical, fatal, error, warning, warn, info, debug, notset
default: error
logs:
  homeassistant.components.device_tracker: warn
  homeassistant.components.mqtt: warn
#  homeassistant.components.device_tracker.mqtt: info
  homeassistant.components.logbook: warn
  homeassistant.components.logger: warn
  homeassistant.components.hassio: debug
  homeassistant.components.weather: warn
  homeassistant.components.recorder: info
#  homeassistant.components.python_script: debug
#  homeassistant.components.script: debug
#  homeassistant.components.scripts: debug
#    homeassistant.components.nmap_tracker: error
#    homeassistant.helpers.entity: critical
  homeassistant.setup: debug
  homeassistant.core: warn
#    homeassistant.helpers.condition: critical
#  homeassistant.config.automation: debug
#  homeassistant.components.websocket_api: warn
#  homeassistant.components.switch: info
#  homeassistant.components.binary_sensor.mqtt: debug
#  py.warnings: critical
  homeassistant.components.light.hue: debug
1 Like

other than identifying wifi/zigbee issues, i’ve been hard cutting in some scripts and automations, as @balloob told me to…

I am able to report rather significant progress on this, that is to say I can identify a big decrease in disconnections/unavailabilty of Hue lights, when I turn-off 1 automation. This automation sense_lights_change checks the states of the lights, and if changed runs 2 python scripts, simply counting these lights and showing their names.

I really like to use it, cause it is part of a monitoring system showing whats happening when. Taking it out makes me need to rely on a time based automation running these python scripts, which takes away much of the elegance.

I was most prominently forced to check this during Party mode… An automation that randomly turns 4 Hue lights on , with random colors. The Hue lights were kicked out almost immediately, and hardly ever came back, even so bad I couldn’t stop the Partymode, since the lights were unavailable to the system.

Right now, ive adapted the Partymode to first switch off the sense_lights_change automation, which is a huge improvement.

Not sure why this only started after moving to Asyncio, (had it al working before without issues), and the many Ikea lights don’t suffer this behavior.

This is in no way an effort of mine to deny your wifi/Zigbee issues. Simply an effort to disclose issues experienced here, so you could take advantage of that.

Can you share your Python script ? Becauseif more people are having performance issues with their python scripts with just normal commands, we might need to evaluate that component and see where things go wrong.

sure, id be glad to.

What im trying to figure out is which causes what: the automation is triggered by changes in the lights, which also happens when the hue lights become unavailable and back again. so a possible loop of events.

Only happening on the Hue lights. Half of the lights are Tradfri, and there is zero issue there. They are checked by the same 2 scripts and same automations.
Hope this helps.

please see below:

whats_on.py:

##########################################################################################
# Whats_on.py, previously known as Anything_on...
#
# interrelated with the Summary.py, which doesn't show the entities per badge (yet)
# modified and customized by https://community.home-assistant.io/u/Mariusthvdb/summary
# big hand in Python by https://community.home-assistant.io/u/petro/summary
# tbd: timezone correction a bit of a challenge, solved with a little hack (=2)
##########################################################################################

familyEntities = 'group.family'
switchEntities = 'group.iungo_switch_switches_template'
lightEntities = 'group.all_lights_only'
applianceEntities = 'group.iungo_switch_appliances_template'

filteredLightEntities = [ entity_id for entity_id in lightEntities if ' ' not in entity_id ]

def CountMyEntities(hass, entity_list, entity_state):
    cnt = 0

#  using a full list:  for entity_id in entity_list:
#  using groups in a [list] :  for group_entity in entity_list:
#   for entity_id in hass.states.get(group_entity).attributes['entity_id']:
    for entity_id in hass.states.get(entity_list).attributes['entity_id']:
        state = hass.states.get(entity_id)
        if state.state == entity_state:
            cnt = cnt + 1
    return cnt

familyHome = CountMyEntities(hass, familyEntities, 'home')
switchesOn = CountMyEntities(hass, switchEntities, 'on')
appliancesOn = CountMyEntities(hass, applianceEntities, 'on')
lightsOn = CountMyEntities(hass, lightEntities, 'on') #if needed, use filteredLightEntities

totalOn = switchesOn + lightsOn + appliancesOn
whichIcon = 'mdi:lightbulb-on-outline' if totalOn else 'mdi:lightbulb-outline' 
status = 'on' if totalOn else 'off' # will evaluate "off" if totalOn == 0.

hass.states.set('input_boolean.whats_on', status, { 
    'friendly_name': "What's On?",
    'icon': whichIcon,
    'family_home': familyHome,
    'lights_on': lightsOn,
    'switches_on': switchesOn,
    'appliances_on': appliancesOn,
    'total_on': totalOn,
    'extra_data_template':'{total_on} on'
})

##########################################################################################
# Lights:
# Badges images: /config/www/lights
##########################################################################################

lights_on = []
#show only lights, not light groups
#excluded = ['light.custom_group_for_group','light.custom_group_for_lights_2']

#total_lights_on = hass.states.get('input_boolean.whats_on').attributes.get('lights_on')
state = hass.states.get('automation.sense_lights_change')

if state:
    total_on = hass.states.get('input_boolean.whats_on').attributes.get('lights_on')
    for entity_id in hass.states.get(lightEntities).attributes['entity_id']:
        dt = state.last_updated + datetime.timedelta(hours=2)
        time = '%02d:%02d' % (dt.hour, dt.minute)

        if hass.states.get(entity_id).state is 'on': #and entity_id not in excluded
            lights_on.append(hass.states.get(entity_id).attributes['friendly_name'])

    if len(lights_on) > 0:
        picture = '/local/lights/hue_pl.png'
        message = ', '.join(lights_on)
        sensor_message = 'Lights on: ' + message
        lights_desc = '=- Lights on: {} -- {} since {}'.format(total_on, message,time)
        uom = 'Lights'

        if len(lights_on) == 1:
            uom = 'Light'
    else:
        picture = '/local/lights/bulboff.png'
        message= ''
        sensor_message= 'No lights on'
        uom = 'Lights'

    sensor_lights_desc = '{}'.format(sensor_message)

    hass.states.set('sensor.lights_badge', total_on, {
#        'custom_ui_state_card': 'state-card-value_only',
        'text': sensor_message,
        'unit_of_measurement': uom,
        'friendly_name': time,
        'entity_picture': picture
         })

##########################################################################################
# Switches:
# Badges images: /config/www/buttons
##########################################################################################

switches_on = []

#total_switches_on = hass.states.get('input_boolean.whats_on').attributes.get('switches_on')
state = hass.states.get('automation.sense_switches_change')

if state:
    total_on = hass.states.get('input_boolean.whats_on').attributes.get('switches_on')
    for entity_id in hass.states.get(switchEntities).attributes['entity_id']:
        dt = state.last_updated + datetime.timedelta(hours=2)
        time = '%02d:%02d' % (dt.hour, dt.minute)

        if hass.states.get(entity_id).state is 'on':
            switches_on.append(hass.states.get(entity_id).attributes['friendly_name'])

    if len(switches_on) > 0:
        picture = '/local/buttons/button_power_on.png'
        message = ', '.join(switches_on)
        sensor_message = 'Switches on: ' + message
        switches_desc = '#- Switches on: {} -- {} since {}'.format(total_on, message,time)
        uom = 'Switches'

        if len(switches_on) == 1:
           uom = 'Switch'
    else:
        picture = '/local/buttons/button_power_off.png'
        message= ''
        sensor_message= 'No switches on'
        uom = 'Switches'

    sensor_switches_desc = '{}'.format(sensor_message)

    hass.states.set('sensor.switches_badge', total_on, {
#        'custom_ui_state_card': 'state-card-value_only',
            'text': sensor_message,
            'unit_of_measurement': uom,
            'friendly_name': time,
            'entity_picture': picture
         })

##########################################################################################
# Appliances:
# Badges images: /config/www/buttons
##########################################################################################
#appliancesGroup = 'group.iungo_switch_appliances_template'
appliances_on = []

#total_appliances_on = hass.states.get('input_boolean.whats_on').attributes.get('appliances_on')
state = hass.states.get('automation.sense_appliances_change')

if state:
    total_on = hass.states.get('input_boolean.whats_on').attributes.get('appliances_on')
    for entity_id in hass.states.get(applianceEntities).attributes['entity_id']:
        dt = state.last_updated + datetime.timedelta(hours=2)
        time = '%02d:%02d' % (dt.hour, dt.minute)

        if hass.states.get(entity_id).state is 'on':
            appliances_on.append(hass.states.get(entity_id).attributes['friendly_name'])

    if len(appliances_on) > 0:
        picture = '/local/buttons/button_power_on.png'
        message = ', '.join(appliances_on)
        sensor_message = 'Appliances on: ' + message
        uom = 'Appliances'

        if len(appliances_on) == 1:
            uom = 'Appliance'
    else:
        picture = '/local/buttons/button_power_off.png'
        message= ''
        sensor_message= 'No appliances on'
        appliances_desc = '!|-> No appliances on since {}'.format(time)
        uom = 'Appliances'

    sensor_appliances_desc = '{}'.format(sensor_message)

    hass.states.set('sensor.appliances_badge', total_on, {
    #        'custom_ui_state_card': 'state-card-value_only',
            'text': sensor_message,
            'unit_of_measurement': uom,
            'friendly_name': time,
            'entity_picture': picture
         })

summary.py (slightly bigger):

##########################################################################################
# Resources:
# https://github.com/maattdiy/home-assistant-config @Mviezzer
# https://home-assistant.io/components/python_script/
# https://home-assistant.io/docs/configuration/state_object/
# modified and customized by https://community.home-assistant.io/u/Mariusthvdb/summary
# big hand in Python by https://community.home-assistant.io/u/petro/summary
##########################################################################################

debug = False

if debug:
    event = data.get('event')
    logger.error("\n\nSUMMARY: " + str(event))

show_badges = True
show_card = True

##########################################################################################
# Group count
# Groups config:
# Entities summary by group name 
# groups = need to have same max member_count
# groups_format = # Message prefix
# groups_filter = # Filter to list
# groups_badge = # Badge 'belt' (unit_of_measurement)
# groups_badge_pic = # none, one, or a list of pictures (picture position = match the count)
# groups_min_show = # Mininum count to show
# groups_theme =  # Theme template
# groups_desc = # Can set the default description, for use in case count = 0
# themes build up or down according to counted value
##########################################################################################
group_count = 0
group_desc = ''
summary = ''
idx = 0
dt_prevstate = None

on ='black_badge|red_badge|yellow_badge|grey_badge|blue_badge|orange_badge|brown_badge|purple_badge|green_badge'

off ='green_badge|purple_badge|brown_badge|orange_badge|blue_badge|grey_badge|yellow_badge|red_badge|black_badge'

groups = ['group.family',
          'group.hubs_binary_pinged',
          'group.critical_devices_state',
          'group.media_player_media',
          'group.device_tracker_media',
          'group.all_lights_only',
          'group.iungo_switch_switches_template',
          'group.iungo_switch_appliances_template',
          'group.binary_sensors_active_template']

groups_format = ['+- Home: {} -- {}',
                 '!|-> Offline: {} -- {}',
                 '!|-> Status - Critical device offline: {} -- {}',
                 '#- Playing: {} -- {}',
                 '#- Tracked: {} -- {}',
                 '=- Lights on: {} -- {}',
                 '#- Switches on: {} -- {}',
                 '#- Appliances on: {} -- {}',
                 '+- Active: {} -- {}']

groups_filter = ['home',
                 'off|idle|not_home', 
                 'off|idle|unknown|not_home',
                 'home|playing|on',
                 'home|playing|on',
                 'on',
                 'on',
                 'on',
                 'on'] # was: 'home|playing|on'

groups_badge = ['Home','Hubs','Status','Play','Track','Lamps','Switch','Appli','Active']

groups_badge_pic = ['','ok|bug|critical','ok|bug|critical','','','','','','']

groups_min_show = [0,0,0,0,0,0,0,0,0]

groups_theme = [on,off,off,on,on,on,on,on,on]

groups_desc = ['!|-> Nobody home since ',
               '+- Hubs: all online',
               '+- Status: all online',
               '%|-> Nothing playing',
               '%|-> Nothing tracked',
               '!|-> No lights on since ',
               '!|-> No switches on since ',
               '!|-> No appliances on since ',
               '%|-> No Appliances active since ']

groups_count = [0, 0, 0, 0, 0, 0, 0, 0, 0]

for group in groups:
    group_count = 0
    group_desc = ''

    for entity_id in hass.states.get(group).attributes['entity_id']:
        state = hass.states.get(entity_id)
        filter = groups_filter[idx]

        if (state.state in filter.split('|') or debug):
            dt = state.last_updated + datetime.timedelta(hours= 2)
            time = '%02d:%02d' % (dt.hour, dt.minute)
            ch = state.last_changed + datetime.timedelta(hours= 2)
            ch_time = '%02d:%02d' % (dt.hour, dt.minute)
          # If state changed in the past days show the date too
            if dt.date() < datetime.datetime.now().date():
                time = '{} {}'.format('%02d/%02d' % (dt.day, dt.month), time)

            group_count = group_count + 1
            group_desc = '{} {} ({}), '.format( group_desc, state.name, time)

        else:
            if (dt_prevstate is None):
                dt_prevstate = state.last_changed
            else:
                if (not state.last_changed is None):
                    if (state.last_changed > dt_prevstate):
                        dt_prevstate = state.last_changed

    # Final format for this group
    if (group_count >= groups_min_show[idx]):
        if (group_count == 0):
            group_desc = groups_desc[idx]
           # If there is none 'On/Home' state in group, show since
            if (group_desc.find(' since ') > 0): #if (group_desc != ''):
                dt = dt_prevstate + datetime.timedelta(hours= 2)
                group_desc = '{} {}'.format(group_desc, '%02d:%02d' % (dt.hour, dt.minute))

        else:
            group_desc = groups_format[idx].format(group_count, group_desc[:-2])

        groups_desc[idx] = group_desc
        groups_count[idx] = group_count

    idx = idx + 1

##########################################################################################
# Badges updates
##########################################################################################

idx = 0

if show_badges:
    for badge in groups_badge:
        if (badge != ''):
            entity_id = 'sensor.{}_badge'.format(badge.replace(' ', '').lower());
            hidden = False if (groups_count[idx] >= groups_min_show[idx] or debug) else True
            fname = groups_desc[idx] if debug else ' '
            picture = groups_badge_pic[idx].replace(' ', '').lower()

            themelist = groups_theme[idx].split('|')
            if len(themelist) > 1:
                try:
                    theme = themelist[groups_count[idx]]
                except IndexError:
                    theme = 'green_badge' # Oops, error badge.1 ;-))
            else:
                theme = 'black_badge' # this will be the default theme

            # Check for picture X index/count
            if (picture != ''):
                picturelist = picture.split('|')
                if (len(picturelist) in [1, groups_count[idx]]):
                    picture = picturelist[-1] #was mylist[len(mylist)-1]
                else:
                    picture = picturelist[groups_count[idx]]

                if (picture != ''):
                    picture = '/local/badges/{}.png'.format(picture)

            hass.states.set(entity_id, groups_count[idx], {
              'friendly_name': fname,
              'unit_of_measurement': badge,
              'entity_picture': picture,
              'templates': { 'theme': theme }
            })

        idx = idx + 1

##########################################################################################
# Alarm clock
# https://github.com/maattdiy/home-assistant-config/blob/master/config/packages/alarmclock.yaml
##########################################################################################

alarms_prefix = ['alarmclock_wd', 'alarmclock_we']
alarms_wfilter = [range(1,6), range(6,8)]
alarms_desc = ''
mydict = {'hidden':hidden}

for entity_id, filter in zip(alarms_prefix, alarms_wfilter):
    state = hass.states.get('input_boolean.{}_enabled'.format(entity_id))
    if state:
        if state.state is 'on' and datetime.datetime.now().isoweekday() in filter:
            state = hass.states.get('sensor.{}_time_template'.format(entity_id))
            alarms_desc = '{}{}, '.format(alarms_desc, state.state)

if (alarms_desc == ''):
    mydict['entity_picture'] = '/local/badges/alarm_off.png'
    mydict['friendly_name'] =  'Relax'
    mydict['unit_of_measurement'] = 'Off'
    mydict['theme'] = 'grey_badge'
    alarms_desc = '%- Alarm clock is not set, relax'
else:
    mydict['entity_picture'] = '/local/badges/alarm.png'
    mydict['friendly_name'] = alarms_desc[:-2]
    mydict['unit_of_measurement'] = 'On'
    mydict['theme'] = 'orange_badge'
    alarms_desc = '/- Alarm clock set at ' + alarms_desc[:-2]

hass.states.set('sensor.alarms_badge', '', mydict)

##########################################################################################
# Mode:
# Inspired by: https://github.com/maattdiy/home-assistant-config/blob/master/config/packages/profiles.yaml
#              https://github.com/maattdiy/home-assistant-config/blob/master/config/packages/developer.yaml
# Badges images: /config/www/modes
##########################################################################################

mode_desc = ''
## Get Mode description
state = hass.states.get('input_select.mode')

if state:
    dt = state.last_changed + datetime.timedelta(hours= 2)
    time = "%02d:%02d" % (dt.hour, dt.minute)

    mode_desc = '{}*- {} mode selected at: {}\n'.format(summary, state.state, time)

    hass.states.set('sensor.mode_badge',state.state, {
         'entity_picture': '/local/modes/{''}.png'.format(state.state.replace(' ','').lower()),
         'friendly_name': time,
         'unit_of_measurement': 'Mode'
          })

##########################################################################################
# Activity:
# Badges images: /config/www/activities
##########################################################################################

activity_desc = ''
## Get Activity description
state = hass.states.get('input_select.activity')

if state:
    dt = state.last_changed + datetime.timedelta(hours= 2)
    time = "%02d:%02d" % (dt.hour, dt.minute)

    activity_desc = '{}$- {} activity selected at: {}\n'.format(summary, state.state, time)

    hass.states.set('sensor.activity_badge', state.state, {
            'friendly_name': time,
            'entity_picture': '/local/activities/{' '}.png'.format(state.state.replace(' ','').lower()),
            'unit_of_measurement': 'Act'
            })

##########################################################################################
# Summary update
# Custom card: /custom_ui/state-card-value_only.html
##########################################################################################

for group_desc in groups_desc:
    if group_desc != '' and not group_desc.endswith(': '):
        summary = '{}{}\n'.format(summary, group_desc)

# Add to final summary
summary = '*============== Status ============\n' \
          '{}\n' \
          '*============= Settings ===========\n' \
          '{}\n' \
          '*============== Modes =============\n' \
          '{}\n' \
          '{}' \
          .format(summary, 
                  alarms_desc, 
                  activity_desc, 
                  mode_desc)

if show_card:
    hass.states.set('sensor.summary', '', {
        'custom_ui_state_card': 'state-card-value_only',
        'text': summary
    })

##########################################################################################
# Codes for text_colors declared in 
# Custom card: /custom_ui/state-card-value_only.html
# changed to use below customizing options:
##########################################################################################

#      case "*": return "bold";
#      case "/": return "italic";
#      case "!": return "red";
#      case "+": return "green";
#      case "=": return "yellow";
#      case "%": return "grey";
#      case "$": return "brown";
#      case "#": return "blue";
#      default:  return "normal";

Hey all, @balloob

back again, after extensive testing, both on Hassio and Hassos, I can report that the flapping issue seems to be caused by several ‘listeners’, rather than the python_script.

If I run the python scripts per time period, now set to each 15 seconds, the flapping stops, and responses of the Hue system seem alright. On the Hassos system, where none of these scripts is installed, but the Hue Custom component is, Hue system is dead right, and no issues arise.

That being said, one needs to be able to trigger from light state changes now and then to automate the Home, and thats when the issues arise. I have tested the following automations, which in them selves works fine, but cause the Hue system to go flapping, ie, show the lights as unavailable for a certain time, and then come back again. Also the disconnecting to the Hue ip address is shown in the logs.

Automations are:

based on event call_service in domain light:

- alias: 'Call Service Event Light'
  id: 'Call Service Event Light'
  trigger:
    platform: event
    event_type: call_service
    event_data:
      domain: light
  action:
    - delay: 00:00:02
    - service: python_script.whats_on
    - delay: 00:00:02
    - service: python_script.summary

based on individual light state (with added condition not being in home mode party, because thats another listener to the lights):

- alias: 'Sense Lights change'
  id: 'Sense Lights change'
#  hide_entity: True
#  initial_state: 'on'
  trigger:
    platform: state
    entity_id:
#Hue lights
      - light.dining_1
      - light.table_lamp_1
      - light.table_lamp_2
      - light.table_lamp_3
      - light.5
      - light.6
      - light.home_theater
      - light.frontdoor
      - light.9
      - light.10
      - light.hue_go_1
      - light.driveway
#Tradfri lights
      - light.chair_long
      - light.drawer
      - light.hobby_tafel
      - light.tv_backlight
      - light.bureau_left
      - light.bureau_right
      - light.inside
      - light.outside
      - light.pantry
  condition:
    condition: state
    entity_id: input_boolean.home_mode_party
    state: 'off'
  action:
    - delay: 00:00:02
    - service: python_script.whats_on
    - delay: 00:00:02
    - service: python_script.summary

and one on state change event, with template selecting the appropriate lights, rather a short and elegant automation:

- alias: 'State changed Light'
  id: 'State changed Light'
  trigger:
    platform: event
    event_type: state_changed
  condition:
    condition: template
    value_template: >
      {{ trigger.event.data.entity_id in state_attr('group.all_lights_only','entity_id') }}
  action:
    - delay: 00:00:02
    - service: python_script.whats_on
    - delay: 00:00:02
    - service: python_script.summary

There is some difference in system response, with the latter being the most robust, or better said least causing issues.
Still, they all are causing the Hue system to disconnect and reconnect after a short while, making this an unreliable source for further Home-automation ( because after reconnecting all last_changed attributes restart, and after disconnecting the condition for a light being in a certain state is not valid anymore) . which is what we all are here for aren’t we.

That the above is only happening in the Hue system implementation could be established by this automation, checking only my Tradfri lights:

- alias: 'State changed Ikea Light'
  id: 'State changed Ikea Light'
  trigger:
    platform: event
    event_type: state_changed
  condition:
    condition: template
    value_template: >
      {{ trigger.event.data.entity_id in state_attr('group.ikea_tradfri_lights','entity_id') }}
  action:
    - delay: 00:00:02
    - service: python_script.whats_on
    - delay: 00:00:02
    - service: python_script.summary

Immediate response, no hubs disconnecting.

Somehow the Hassio - Hue system (integration) doesn’t like listeners

Hope this helps in further exploring this issue, and be glad to help in any way.
before you ask: of course I did not run these automotations simultaneously…

Cheers,
Marius

I thought I had gotten past this as well. Running 75.3 now. Created this automation the other day. Worked great for one day, then this morning, it was stuck, could not get it to work, took me a few minutes to realize that sure enough, several of my hue bulbs were unavailable. I’ll double check, but I don’t have many conditions based on hue, I have several triggers based on other things, that have actions with Hue (pretty standard stuff). Definitely no python scripts to check status.

- alias: Set alcove light color based on Traffic to work
  trigger:
  - platform: state
    entity_id: input_select.house_mode
    from: 'Night'
  - platform: state
    entity_id: sensor.person_traffic_density_to_work
  - platform: time
    at: '07:35:00'
  condition:
  - condition: state
    entity_id: binary_sensor.workday_sensor
    state: 'on'
  - condition: state
    entity_id: input_select.house_mode
    state: Home
  - condition: time
    after: '07:35:00'
    before: '08:35:00'
  action:
  - service: light.turn_on
    data_template:
      entity_id: light.alcove_light
      brightness_pct: 100
      color_name: >
        {% set map = {'Normal': 'green', 'Moderate': 'orange', 'Heavy': 'red'} %}
        {% set state = states('sensor.person_traffic_density_to_work') %}
        {{ map[state] if state in map else 'white' }}

This bulb and another bulb physically nearby, but not part of any recent automation were suddenly unavailable. Just passing along for a data point. I will see how this runs tomorrow. After restarting HA it worked as expected. Also I had restarted HA yesterday, so it was not up and running for days before this happened

Just wanted to add this too, doubt it’s related, but much of my lighting for hue (not the example above), is controlled by calling hue scenes with scripts. So morning, runs two scenes for two groups of lights, turn on tv after dark, runs a different set of scenes, come home after dark, etc. Anyone else with issues, calling hue scenes?

ive been digging some further, and found out that not only does the Hue integration suffer from lights related automations calling the python scripts, but other processor heavy automations (listeners) cause the Hue lights to go unavailable immediately and start the ‘flapping’

I have an automation that triggers on each automation being triggered (of course without being triggered by itself…or several other often triggered ones) and another automation triggered by state change of my power sensors.

enabling either of these throws the Hue into ‘flapping’.

And only the Hue hub, the rest of the system runs fine. This is a timing issue leading to frustration. I can prevent it from happening (well, almost, even without these automations and pythons the hue shows unavailable now and then) by disabling these automations. But, these automations are there to monitor the House and its main systems. Exactly what HomeAssistant should be doing.

Hope the dev’s can optimize the Hue integration (before the asyncio move this didnt happen) and stop this unstable Hue behavior.

these are the 2 automations.

- alias: Automation ran
#  hide_entity: True
  initial_state: 'off'
  trigger:
    platform: event
    event_type: state_changed
  condition:
    condition: template
    value_template: >
      {% set skip_list = ['automation_ran', 'count_warnings', 'count_errors',
                           'activate_map_sensors_actueel'] %}
      {{ trigger.event.data.entity_id.startswith('automation.') and
         trigger.event.data.entity_id.split('.')[1] not in skip_list and
         'old_state' in trigger.event.data and 'new_state' in trigger.event.data }}
  action:
    - condition: template
      value_template: >
        {{ trigger.event.data.new_state.attributes.last_triggered !=
           trigger.event.data.old_state.attributes.last_triggered }}
    - service: notify.filed_automations
      data_template:
        message: >
         {{ as_timestamp(trigger.event.data.new_state.last_updated)|timestamp_custom("%d %b: %X") }}: {{ trigger.event.data.new_state.name }}
    - service: python_script.last_automation
      data_template:
        event: '{{ trigger.event }}'

- alias: 'Activate Map sensors actueel'
  id: 'Activate Map sensors actueel'
#  hide_entity: True
  initial_state: 'off'
  trigger:
    platform: event
    event_type: state_changed
    event_data:
      domain: sensor
  condition:
    condition: template
    value_template: >
      {{ trigger.event.data.entity_id.endswith('actueel')}}
  action:
    service: python_script.map_sensors_actueel
1 Like

I’m running 76.2 and am still having this issue from time to time. Recently I added a new automation that uses one of my hue bulbs and have noticed that the issue is much, more common now. To be fair, I’m not always looking and the UI, or logs, so It’s possible they are “unavailable” but on other times during the day.

Here’s my usual Hue light control. Wake up in the morning, turn on a couple of hue light strips. Leave the house, turn off all lights (inlcuding those strips). Come home turn on the light strips again, if its dark, also turn on a couple other strips and a bulb. Turn on TV after dark, change the color/brightness of a some of the strips. Turn off the TV, change brigtness/etc. all of the lights are changed by calling Hue Scenes via a script. The lights at night are 99% stable. I cannot remember the last time they did not work.
The lights in the morning used to be stable, until I added this recent automation. It is more “intensive”. I’ll post in a second. Just to be clear, and to avoid useful recommendations, I’ll add that this has not always been an issue, its not my wi-fi, or channel, etc. Been there done that.

So here is what happens in the morning, and this is what has seemed to be causing my lights to become unavailable

- alias: Good Morning Routine
  trigger:
  - platform: state
    entity_id: binary_sensor.motion_sensor_living_room_motion
    from: 'off'
    to: 'on'
  - platform: state
    entity_id: binary_sensor.motion_sensor_kitchen_motion
    from: 'off'
    to: 'on'
  condition:
  - condition: state
    entity_id: input_select.security_system
    state: 'Armed (Home)'
  - condition: time
    after: '05:30:00'
    before: '11:30:00'
  action:
  - service: script.turn_on
    entity_id:
      - script.ecobee_resume_mode
      - script.kitchen_evening_lights_hue_scene
  - service: lock.unlock
    entity_id: lock.lock_garage_door_lock
  - service: input_select.select_option
    data_template:
      entity_id: input_select.security_system
      option: Disarmed
  - service: input_select.select_option
    data_template:
      entity_id: input_select.house_mode
      option: Home

Here is the script for the scene
kitchen_evening_lights_hue_scene:
  alias: Hue Scene Evening Lights (Kitchen)
  sequence:
    - service: hue.hue_activate_scene
      data:
        group_name: "Kitchen"
        scene_name: "Evening Lights"

Additional Automations
- alias: Set alcove light color based on Traffic to AVIS
  trigger:
  - platform: state
    entity_id: input_select.house_mode
    from: 'Night'
  - platform: state
    entity_id: sensor.tracys_traffic_density_to_avis
  - platform: time
    at: '06:35:00'
  condition:
  - condition: state
    entity_id: binary_sensor.workday_sensor
    state: 'on'
  - condition: state
    entity_id: input_select.house_mode
    state: Home
  - condition: time
    after: '06:35:00'
    before: '07:34:00'
  action:
  - service: light.turn_on
    data_template:
      entity_id: light.alcove_light
      brightness_pct: 100
      color_name: >
        {% set map = {'Normal': 'green', 'Moderate': 'orange', 'Heavy': 'red'} %}
        {% set state = states('sensor.tracys_traffic_density_to_avis') %}
        {{ map[state] if state in map else 'white' }}

- alias: Set alcove light color based on Traffic to Base
  trigger:
  - platform: state
    entity_id: input_select.house_mode
    from: 'Night'
  - platform: state
    entity_id: sensor.pauls_traffic_density_to_base
  - platform: time
    at: '07:35:00'
  condition:
  - condition: state
    entity_id: binary_sensor.workday_sensor
    state: 'on'
  - condition: state
    entity_id: input_select.house_mode
    state: Home
  - condition: time
    after: '07:35:00'
    before: '08:35:00'
  action:
  - service: light.turn_on
    data_template:
      entity_id: light.alcove_light
      brightness_pct: 100
      color_name: >
        {% set map = {'Normal': 'green', 'Moderate': 'orange', 'Heavy': 'red'} %}
        {% set state = states('sensor.pauls_traffic_density_to_base') %}
        {{ map[state] if state in map else 'white' }}

Additional details. Looking through the logs for time stamps as I was writing this, I see the three lights in my kitchen hue group, starting going unavailable, available at 5am exactly. Which does not seem to tie into this after all, but it is weird that it’s just this group of lights. The other 4-5 hue lights in different groups did not go unavailable at all during that time.

Well, I guess I believed the traffic automation was the cause, but I think that it just brought it more to my attention. It is weird that it is only one hue group though.

just a heads-up the dev’s have now enabled debug on the hue component to see if updating causes these disconnects. Ive posted some of my findings on the Github issue:

please have a look if you could do so too, so the dev’s have as much useful info as possible to be able to mitigate the issue here.
thx

1 Like

binary sensors ping (and mqtt) causing Hue unavailable? please see https://github.com/home-assistant/home-assistant/issues/16689#issuecomment-442094226 and check if this would help in your setups.

cheers.

Is there any update on this? I started to have the same problem yesterday when couple of my Hue lights started becoming unavailable suddenly. Noticed this when one of my automations was not working even when HA showed that it triggered normally. And when investigating I noticed that everything works but some of the lights are becoming unavailable and then suddenly available again.

This started when I added device_tracker and couple of automations that use that.

@Mariusthvdb are you still suffering from this or did you solved it by changing the python script and/or binary_sensor ping?

According to the Github issue, someone did a reset on their Hub and it started working. I actually plan on doing this since there are so many useless scenes etc. created by 3rd party andoird and iOS apps but don’t know if that is going to help with this.

Edit: Sorry for tagging you Mariusthvb

no this still prevails, and reseting the hub wont work…
there are many many posts on this.
that being said, I have 2 instances and the one which has my mqtt broker on it (the hassio add-on) suffers most. I can not have any automation rely on lights state, because of that.
It apparently has to do with I/O going on. And we must see the issue as a signal of something else in the system taking processor time/goin wrong. thats what the HA founder told me.

If you add show_last_changed to the lights, you can exactly see for how long they have been seen by the system.
On my Mqtt system that mostly is about a few minutes, and if I am lucky, several 10’s of minutes up to an hour.
On my 2nd system, which I use to show in the app and use as a front-end, light are available most of the time. But reloading Lovelace heavy cards (weather custom cards etc) might again cause Unavailable.

So long answer short: its still there.

see below, showing the same lights/groups in my

mqtt system:

23

and main system:

35

Well this is too bad. It is actually kind of breaking thing and might move people away from Home Assistant in the end if it is not fixed.

I can investigate this on my side but most likely nothing new and useful will be found :frowning:

As a final thing to try take a look here

Since I boosted the wifi signal in my house the lights haven’t shown as unavailable once.

Thanks, I will take that into account. Most likely that has nothing to do with my lights going unavailable since I have a powerful WIFI AP that is only a WIFI AP so it has pretty strong signal everywhere in my house. But this is still a good point to look :+1:

well, not ‘final’ as in this solved it…

this has been discussed before, as has changing the zigbee channels. It might solve some issues in particular for some users,because of their specific home settings. It isnt a solution to the issue in itself though.

It has to do with I/O on your local system and the Hue needing to be polled regularly, as it is implemented now, rather than receiving pushed changed from the Hue side. (like eg the Tradfri system does)

To stress the importance of this issue, we could however all hop over on the Hue developers forum and support @balloob’s call for a push API on the HUE system: https://developers.meethue.com/forum/t/2018-and-2019-hue-api-and-sdk-features/5996/21