Is there a way to template out automation (dynamic) attributes?

How do I use a service_template / data_template to do dynamic attributes? For example, I have a remote that, when a button is pushed, it turns off the lights. Whereas another button would dim the lights.

- alias: "Cube Controller"
  hide_entity: false
  trigger:
    - platform: state
      entity_id: sensor.cube_controller
  condition:
    condition: and
    conditions:
      - condition: template
        value_template: "{% if is_state('sensor.cube_controller', 'inactive') %}false{% else %}true{% endif %}"  
  action:
    - service_template: "{% if trigger.to_state.state == 'Knock' %}light.turn_off{% else %}light.turn_on{% endif %}"
      data_template:
        entity_id: >
          {% if trigger.to_state.state == 'Rotate left' or trigger.to_state.state == 'Rotate right' %}
            light.office
          {% else %}
            light.office
          {% endif %}
        brightness: >
          {% if states.light.office.attributes.brightness is defined %}
            {% if trigger.to_state.state == 'Rotate left' %}
              {% if states.light.office.attributes.brightness|int - 75 <= 0 %}
                5
              {% else %}
                {{ states.light.office.attributes.brightness|int - 75 }}
              {% endif %}
            {% elif trigger.to_state.state == 'Rotate right' %}
              {% if states.light.office.attributes.brightness|int + 75 >= 255 %}
                255
              {% else %}
                {{ states.light.office.attributes.brightness|int + 75 }}
              {% endif %}
            {% endif %}
          {% else %}
            5
          {% endif %}
    - delay: 00:00:01
    - service: mqtt.publish
      data:
        topic: 'smartthings/Cube Control/button'
        payload: 'inactive'
        retain: 'true'

This is all well and good, but if I have a service_template that does ‘light.turn_off’, well, it doesn’t like the ‘brightness’ configuration being there.