Input_number increment with custom button

I have a custom button card that sets a setpoint temperature via MQTT with up/down buttons. It works but it is slow. I would like to use the up/down buttons to increment/decrement an input_number and then once the desired value is obtained, then publish MQTT with a Save button. For the life of me, I cannot get the up/down buttons to change the input_number value. Do I need to do this with a script? I thought I could do this from within the buttons but now I’m totally confused.
Any help would be appreciated…
custom button card
Here is the custom button code using the Publish MQTT service

type: horizontal-stack
cards:
  - type: custom:button-card
    color_type: card
    color: rgb(223, 255, 97)
    icon: mdi:arrow-up-bold-outline
    size: 100%
    styles:
      card:
        - height: 40px
        - width: 60px
        - '--mdc-ripple-color': green
        - '--mdc-ripple-press-opacity': 0.5
      tap_action:
        action: call-service
        service: mqtt.publish
        service_data:
          topic: BackGarage/Thermostat/UpSwitch/switch
          payload: 'ON'
  - type: custom:button-card
    tap_action:
      action: call-service
      service: mqtt.publish
      service_data:
        topic: BackGarage/Thermostat/DownSwitch/switch
        payload: 'OFF'
    color_type: card
    color: rgb(223, 255, 97)
    icon: mdi:arrow-down-bold-outline
    size: 100%
    styles:
      card:
        - height: 40px
        - width: 60px
        - '--mdc-ripple-color': green
        - '--mdc-ripple-press-opacity': 0.5
  - type: custom:button-card
    color_type: blank-card
    styles:
      card:
        - height: 30px
        - width: 10px
  - type: custom:button-card
    color_type: blank-card
  - type: custom:button-card
    color_type: card
    name: Setpoint
    show_name: false
    entity: input_number.setpoint
    icon: none
    size: 25%
    styles:
      card:
        - height: 40px
        - width: 60px
        - font-size: 24px
    show_state: true
  - type: custom:button-card
    tap_action:
      action: call-service
      service: mqtt.publish
      service_data:
        topic: BackGarage/Thermostat/SaveSwitch/switch
        payload: 'OFF'
    color_type: card
    color: rgb(223, 255, 97)
    name: Save
    styles:
      card:
        - height: 40px
        - width: 60px
        - font-size: 24px
        - '--mdc-ripple-color': green
        - '--mdc-ripple-press-opacity': 0.5
  - type: custom:button-card
    color_type: blank-card
    styles:
      card:
        - height: 30px
        - width: 10px

the tap_action would be a service call to increment or decrement the input number based on the defined step variable. the save button could then publish the setpoint.

So that’s what I thought, that you could set the input_number value from the tap_action. But I still can’t get the input box display to change. I modified the up/down buttons to just set a value based on examples I found. But nothing changes. Here is the modified code:

type: custom:button-card
tap_action:
  service: input_number.set_value
  data:
    entity: input_number.setpoint
    value: 50
color_type: card
color: rgb(223, 255, 97)
icon: mdi:arrow-up-bold-outline
size: 100%
styles:
  card:
    - height: 40px
    - width: 60px
    - '--mdc-ripple-color': green
    - '--mdc-ripple-press-opacity': 0.5

I must be missing something.