Just to share how I did it.

To explain a bit:
- My second floor is organized in 4 areas.
- I have renamed all lights that are part of another light to something like “2nd Floor xyz”. These are for instance IKEA bulbs in a lamp with 3 bulbs. I have organized this using light groups into one light.
- I use custom button cards for the buttons. Check out https://github.com/custom-cards/button-card.
- platform: template
sensors:
lights_number_on_in_2nd_floor:
friendly_name: 'Lights on in 2nd floor'
unique_id: lights_number_on_in_2nd_floor
value_template: >
{%- set search_state = 'on' %}
{%- set search_areas = ['Office', 'Loft', 'Bedroom', 'Master bedroom'] %}
{%- set ns = namespace(lights=[]) %}
{%- for light in states.light | selectattr('state','eq', search_state) %}
{%- for area in search_areas %}
{% if area_name(light.entity_id) == area and "2nd Floor" not in state_attr(light.entity_id, "friendly_name") %}
{%- set ns.lights = ns.lights + [ light.entity_id ] %}
{% endif%}
{%- endfor %}
{%- endfor %}
{{ ns.lights| list | length }}
Edit: Thanks to @petro and everyone else for the help in the thread.