Managed to get this working without giving up markdown parsing (I like it), here it goes, hope it’s useful to more people:
- service: notify.telegram_bot
data_template:
title: '*Alarm triggered*'
message: >-
{% set trigger_name = trigger.to_state.entity_id %}
🚨 Triggered by {{ trigger_name.replace('_','\_') }}