There are a couple of ways to do it.
One way is to use the “Change device type of a switch” helper to create 28 matching light entities, one for each switch. Then you can use the UI to create a light group of all 32 light entities. Light groups are special because they maintain the ability to set brightness levels of the group members.
To convert a light entity into a switch you would have to use template switch helpers instead. If you’re grouping them anyway you could probably make one template switch that turns all four lights on and off and sets their brightness level. Again you can then create a normal group of all the switches.
Lastly you might be able to define an old-school group in YAML that contains both the lights and switch entities but note this requires using homeassistant.turn_on/off actions to control it.