A label is not part of an entity’s attributes. Therefore you cannot use attributes.label in a selectattr.
There are specific functions for working with labels.
The following example gets a list of entity_ids representing all entity’s with the “activity” label, gets their state, selects only the ones that are on, then returns the quantity.
{{ label_entities('activity') | select('is_state', 'on') | count }}
Be advised that you stated you were attempting to “count the number of devices” with a template that begins with states (which contains information about all entities).
The word “device” has special meaning in Home Assistant and is different from another term named “entity”. Your template, had it worked, would have reported the number of entities that are on (not devices).
A device is a virtual model of a physical device and consists of one or more entities.