Aetoc Z-wave dimmer does not respond correctly

I have an Aeotec DSC19103 Micro Smart Dimmer (2nd Edition) which is connected to a few halogen spotlights.
The entity i can control on/off with is called

light.vardagsrum_spotlights_level

It behaves a bit weird, if I’m using homeassistant.turn_off / on it seems to work fine
If i flip the “switch” in the group i have visible in the gui it also repsonds, but if i flip the switch for “all lights” for the group it does not turn on/off.

If i try to use automations with light.toggle it will not respond.

Does this has something to do with the spotlights and dimmer not working together or might there been some issue with including it and i should have more than the “.level” entity? I dont have any other z-wave dimmers installed so cant compare it to anything.

a few examples that works / dont work.

Does not work, only the light.livingroom_window will toggle

action:
  - service: light.toggle
    entity_id: light.vardagsrum_spotlights_level
  - service: light.toggle
    entity_id: light.livingroom_window

Works:

  home_theatre_on:
    sequence:
     - service: light.turn_off
       entity_id: light.vardagsrum_spotlights_level
     - service: light.turn_off
       entity_id: light.livingroom_window