Homekit: Home Assistant switch state doesn't update

Hey there,
I built my own alarm system which works great inside my smart home system, however it looks like Homekit doesn’t update the state of the switch of the alarm. It is just in Homekit though. I see the real state both in Home Assistant as well as in Alexa.

  - platform: template
    switches:
     alarm_switch:
       value_template: "{{ is_state('input_boolean_alarm_state', 'on')}}"
       turn_on:
         service: input_boolean.turn_on
         entity_id: input_boolean.alarm_state
       turn_off:
         service: input_boolean.turn_off
         entity_id: input_boolean.alarm_state

both the switch and boolean number show as off in Home Assistant. And the Alexa app also shows it as off. However in Homekit it still shows as “on”.
All of my other Home Assistant switches update just fine in my Homekit setup. It is weird

Functionality of the alarm system:
When I press my Homekit connected Philips Hue Button next to the bedroom, the alarm switches to On.
In the mornings there is a Hue motion sensor next to my kitchen which triggers an Alexa routine to switch the alarm off. Everything works great, just Homekit still shows it as “on” even though it is not.

Any ideas?