Need help selecting entities in grouped groups, in Python script

Mariusthvdb · October 10, 2020, 9:31pm

for checking which and how many lights are on, I use this python script, with the entity selecting like this:

for entity_id in hass.states.get(group).attributes['entity_id']:
    state = hass.states.get(entity_id)

which is quite the default way, nothing spectaculair here…

full script below.

But, this is done with the group.all_lights_only, that lists all lights individually, so the entity_id’s in that group are in fact the individual lights.

I have changed my lights groups setup though, so that I nest a few groups, of which the main nestings are:

  all_lights_only:
    name: All lights only
    icon: mdi:lightbulb-outline
    entities:
      - group.all_inside_lights
      - group.all_outside_lights

  all_inside_lights:
    name: All inside lights
    icon: mdi:lightbulb-outline
    entities:
      - group.main_inside_lights
      - group.guest_inside_lights
      - group.living_ceiling_spots

  all_outside_lights:
    name: All outside lights
    icon: mdi:flood-light
    entities:
      - group.outside_flood_lights
      - group.terrace_outdoors_spots

which of course doesn’t work with the current python_script, because it now sees one of these groups as being the entity_id. I now would need to iterate these groups, which consist of several groups, (that contain groups again), containing lights.

What would be the best way to do so in Python?

I’ve started with this, explicitly listing all groups (to make it easier and not have to iterate groups that contain yet more groups)

light_groups = ['group.main_inside_lights','group.guest_inside_lights',
                'group.living_ceiling_spots','group.outside_flood_lights',
                'group.terrace_outdoors_spots']

for group in light_groups:
    for entity_id in hass.states.get(group).attributes['entity_id']:
        light_ids.append(entity_id)

for entity_id in light_ids: #hass.states.get(group).attributes['entity_id']:
    state = hass.states.get(entity_id)

this now does seem to work. But it feels inadequate… As worded in

This is not as elegant as the expand() function we now can easily use in Jinja…
Would appreciate your help here,
thanks for having a look!

dt_prevstate = None
utc_offset = hass.states.get('sensor.utc_offset').state
timeDifference = float(utc_offset)

group = 'group.all_lights_only'
fname = 'Lights'
on_format = 'Lights on: {} -- {}'
off_format = 'No lights on since '
filter = 'on'
pic = 'lamp_off|lamp'
icon = 'mdi:lightbulb'

# min_show = 0
count = 0
desc = ''

for entity_id in hass.states.get(group).attributes['entity_id']:
    state = hass.states.get(entity_id)

    if (state.state == filter or debug):
        dt = state.last_changed + datetime.timedelta(hours= timeDifference)
        time = '%02d:%02d' % (dt.hour,dt.minute)

      # If state changed in the past days show the date too
        if dt.date() < datetime.datetime.now().date():
            time = '{} {}'.format('%02d/%02d' % (dt.day,dt.month),time)

      # check if hue lights are 'reachable' is needed because of allow_unreachable: true
        binary_sensor_id = 'binary_sensor.{}_reachable'.format(state.object_id)
        binary_sensor = hass.states.get(binary_sensor_id)
        if binary_sensor is None or binary_sensor.state == 'on':
            count = count + 1
            desc = '{} {} ({}), '.format(desc,state.name,time)

    else:
        if (dt_prevstate is None):
            dt_prevstate = state.last_changed
        else:
            if (not state.last_changed is None):
                if (state.last_changed > dt_prevstate):
                    dt_prevstate = state.last_changed

# Final format for the group

picturelist = pic.split('|')

if (count == 0):
    desc = off_format
    picture = picturelist[0]
   # If there is none 'On/Home' state in group, show 'since'
    if (desc.find(' since ') > 0):
        dt = dt_prevstate + datetime.timedelta(hours= timeDifference)
        desc = '{} {}'.format(desc,'%02d:%02d' % (dt.hour,dt.minute))
else:
    desc = on_format.format(count, desc[:-2])
    picture = picturelist[1]

##########################################################################################
# Create sensor and badge
##########################################################################################

badge_id = 'sensor.{}_badge'.format(fname.replace(' ', '').lower());
sensor_id = 'sensor.{}_summary'.format(fname.replace(' ', '').lower());

picture = '/local/badges/{}.png'.format(picture)

# badge
hass.states.set(badge_id, count, {
  'friendly_name': '',
  fname: desc,
  'unit_of_measurement': fname + ': ' + str(count),
  'entity_picture': picture,
#  'icon': icon,
})

# sensor
hass.states.set(sensor_id, count, {
  'friendly_name': fname,
  fname: desc,
#  'unit_of_measurement': badge + ': ' + str(count),
  'entity_picture': picture,
#  'icon': icon,
})

Steven_Rollason · October 19, 2020, 12:02pm

A hacky way of leveraging the expand() functionality from Jinja templates, which may work (I’ve done similar in one of my Python scripts in order to take advantage of Jinja templating) would be:

Create a template sensor:

sensor:
  - platform: template
    sensors:
      light_entity_ids:
        value_template: >-
          {{ expand("group.all_lights_only") | map(attribute="entity_id") | list | join(",") }}

Then in the Python script, simply use

light_ids = hass.states.get("sensor.light_entity_ids").split(",")

If that doesn’t work, due to the string exceeding the maximum state size (255), you could expose it as an attribute instead:

sensor:
  - platform: template
    sensors:
      light_entity_ids:
        value_template: >-
          {{ expand("group.all_lights_only") | map(attribute="entity_id") | list | count }}
        attribute_templates:
          entity_ids: >-
            {{ expand("group.all_lights_only") | map(attribute="entity_id") | list | join(",") }}

and then use

light_ids = hass.states.get("sensor.light_entity_ids").attributes("entity_ids").split(",")

Mariusthvdb · October 19, 2020, 12:16pm

thanks! a bit hacky indeed, but yet another interesting option.

Still, I can’t imagine, there isn’t a python replacement for

     {{ expand("group.all_lights_only") | map(attribute="entity_id") | list | join(",") }}

with the expanded group containing (multiple nested) groups.

petro · October 19, 2020, 12:40pm

Add this function to your python script after desc = ''

def flatten(entity_id, result=[], searched=[]):
    state = hass.states.get(entity_id)

    if state is not None:
        entity_ids = state.attributes.get("entity_id")
        if entity_ids is not None:
            searched.append(entity_id)
            for entity_id in entity_ids:
                if entity_id not in result and entity_id not in searched:
                    flatten(entity_id, result, searched)
        else:
            result.append(entity_id)

then change the beginning of your for loop from

for entity_id in hass.states.get(group).attributes['entity_id']:

to

entity_ids = []
flatten(group, entity_ids)
for entity_id in entity_ids:

the flatten function mimics expand()

Mariusthvdb · October 19, 2020, 1:13pm

thanks!
had been experimenting with the flatten function, but thought it would be available already, and not required the definition in the script itself.

can report success

this is so much better.

petro · October 19, 2020, 1:44pm

Report any issues with it. It’s a recursive function and I tried to trap against circular group configurations. You could get max recursion exceptions, hopefully you won’t.

Mariusthvdb · October 19, 2020, 2:02pm

ok I will be on the lookout.

though, the groups I use don’t have any circular reference. As posted above the top group holds a few sub groups which again hold a few groups, until the last group that only holds single entities. None of the entities (lights) are in any other group.

The whole exercise was about that, so I have been careful in the group configurations.

petro · October 19, 2020, 3:17pm

Yeah I wouldn’t worry about it, I tested it with this setup and it works:

all_lights:
  entities:
    - group.foo
    - group.bar
    - light.bed_light
foo:
  entities:
    - light.ceiling_lights
    - group.bar
bar:
  entities:
    - light.kitchen_lights
    - group.foo

Mariusthvdb · October 28, 2020, 9:05am

found an issue! Or, more of a challenge really

but not in the python, in a js template referencing the now toplevel group.all_inside_lights:

  - type: custom:button-card
    entity: group.all_inside_lights
    template: button_title_counter
    triggers_update: input_boolean.run_lights_summary
    label: >
      [[[
      var i;
      var entities = entity.attributes.entity_id;
      var count = 0;
      for (i = 0; i < entities.length; i++) {
        var state = states[entities[i]].state;
        if (state == 'on') {count += 1;}
       }
      if (count == entities.length) return 'All ' + entities.length + ' lights on';
      if (count == 0) return 'No lights on';
      return 'Lights on: ' + count + ' of ' + entities.length;
      ]]]

counted the lights on before, but now, as is to be expected shows only the count of the groups (which now are the entities inside the group.all_inside_lights

so, the question is, can we also ‘flatten’ groups in JS…? some googling suggests we can, but I havent found a real JS replacement for what you made in the python_script.

If you’d have a moment and see a challenge, Id appreciate it. Guess it could be very useful for others too, having a counter iterating groups for entities
Otherwise I fear I’d see myself forced to re-introduce the verbose group…

petro · October 28, 2020, 1:26pm

try converting the python flatten function to JS yourself.

Mariusthvdb · October 28, 2020, 4:58pm

given the fact that this: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/flat would be available, shouldn’t I use that, instead of trying to literally convert the python to js?

would have been nice if something along then lines of

      [[[
      var i;
      var flattened = entity.flat(4);
      var entities = flattened.attributes.entity_id;
      var count = 0;
      for (i = 0; i < entities.length; i++) {
        var state = states[entities[i]].state;
        if (state == 'on') {count += 1;}
       }
      if (count == entities.length) return 'All ' + entities.length + ' lights on';
      if (count == 0) return 'No lights on';
      return 'Lights on: ' + count + ' of ' + entities.length;
      ]]]

would have been possible.

new territories, finding/learning available JS functions…

petro · October 28, 2020, 5:04pm

You don’t have nested arrays, you have nested objects with arrays. You’ll have to make this by hand.

Mariusthvdb · October 28, 2020, 5:10pm

ok will study after dinner

thanks

petro · October 28, 2020, 5:22pm

Yeah your googling the wrong information. You want to google recursive functions. But if you simply went through the python script and tried to replace each piece with the correct call/syntax for JS, you’d get what you want.

Mariusthvdb · October 29, 2020, 3:39pm

finding myself in new territory trying to define a recursive function in JS, and the translation from Python…

first structural attempt would be something like this, where entity_id is the main group, entities the group members, and e an individual light that will be appended (or not):

function flatten(entity, lights_result, lights_searched){
  return entity; # ??< dont yet see what I should do here..
  }

var entities = entity.attributes.entity_id;
var lights_result = []
var lights_searched = []

if (entity && entities ) return
              searched.append(entity);  # set the first main group to be the entity
  for (e in entities) {(return e not in lights_result && e not in lights_searched) # check if an entity is in either list
                       ? flatten(e, lights_result, lights_searched) : null ;} # if no, append, if yes, do nothing

return result.append(e);

could you please comment on the most basic of errors…

petro · October 29, 2020, 9:10pm

Just go line by line. You almost have the first line correct.

python

def flatten(entity_id, result=[], searched=[]):

js

function flatten(entity_id, result=[], searched=[])
{

}

Now what’s the second line

python

state = hass.states.get(entity_id)

js

… You should know this. How do you get a state using the entity_id in JS?

the line would go here:

function flatten(entity_id, result=[], searched=[])
{
       # <- line goes here.
}

Mariusthvdb · October 29, 2020, 9:47pm

are we looking for

entity.attributes.entity_id

making that line

var state = entity.attributes.entity_id

?

petro · October 29, 2020, 11:43pm

Close, we are trying to just get the state from the state machine using the entity_id

Mariusthvdb · October 30, 2020, 9:11am

meaning entity.entity_id, like this in Jinja

function flatten(entity_id, result=[], searched=[])
{
       var state = entity.entity_id
}

and in the next step we need to get the attributes list:

set entities =  entity.attributes.entity_id

recursively:

to finally reach the lights:

(replaced only one group per level for illustrational purposes )

making the full function like this then:

      [[[
         function flatten(entity, lights_result= [], lights_searched= []){
           var state = entity.entity_id;
           return state;
           
           if (state != None) {
             var entities = entity.attributes.entity_id;
               if (entities != None) {
                 searched.append(entities);
                   for (e in entities) {
                     (! lights_result.includes(e) && ! lights_searched.includes(e))
                       ? flatten(e, lights_result, lights_searched) : null
                                        }
                                      }
                               }
              return result.append(entities);
}

and then use this in the actual template…

[[[
  function flatten(entity, lights_result= [], lights_searched= []){
  var state = entity.entity_id;
           return state;
           
           if (state != None) {
             var entities = entity.attributes.entity_id;
               if (entities != None) {
                 searched.append(entities);
                   for (e in entities) {
                     (! lights_result.includes(e) && ! lights_searched.includes(e))
                       ? flatten(e, lights_result, lights_searched) : null
                                        }
                                      }
                               }
              return result.append(entities);
}
#   var entities = entity.attributes.entity_id; #<--- this needs to become the flattened list of lights so:
   var light_ids = []; # <-- new variable to hold the light_ids
   var entitities = flatten(entity, light_ids);  # <-- flatten group 'entity' in list 'light_ids'

   var i;
   var count = 0;
   for (i = 0; i < entities.length; i++) {
     var state = states[entities[i]].state;
     if (state == 'on') {count += 1;}
     }
   if (count == entities.length) return 'All ' + entities.length + ' lights on';
   if (count == 0) return 'No lights on';
   return 'Lights on: ' + count + ' of ' + entities.length;
]]]

RomRider · October 30, 2020, 3:11pm

That’s how I deal with it in button-card (the entity being a group). This returns the number of entities which are on in the group and works for nested groups.

        if (!entity) return 0;
        function loop(list, value) {
          list.forEach(entity => { // This entity is different from the button-card entity, I should have named it differently...
            if (states[entity])
              if (entity.split('.')[0] === "group")
                value = loop(states[entity].attributes.entity_id, value);
              else if (["open", "on"].includes(states[entity].state))
                value++;
          });
          return value;
        }
        return loop(entity.attributes.entity_id, 0);