Erik has responded to the Github issue and provided changes to the my fan definition as above. Apparently we should put quotes around all string values in yaml files as otherwise it might get interpreted as a boolean false.
So I have changed my yaml file as per below (added a bunch of quotes).
This version of my fan configuration is working for me with Home Assistant core 2021.3.1
fan OfficeFan:
- platform: mqtt
name: "Office Fan"
command_topic: "cmnd/fan1/FanSpeed"
speed_command_topic: "cmnd/fan1/FanSpeed"
state_topic: "stat/fan1/RESULT"
speed_state_topic: "stat/fan1/RESULT"
state_value_template: >
{% if value_json.FanSpeed is defined %}
{% if value_json.FanSpeed == 0 -%}0{%- elif value_json.FanSpeed > 0 -%}4{%- endif %}
{% else %}
{% if states.fan.office_fan.state == 'off' -%}0{%- elif states.fan.office_fan.state == 'on' -%}4{%- endif %}
{% endif %}
speed_value_template: "{{ value_json.FanSpeed }}"
availability_topic: tele/fan1/LWT
payload_off: "0"
payload_on: "4"
payload_low_speed: "1"
payload_medium_speed: "2"
payload_high_speed: "3"
payload_available: "Online"
payload_not_available: "Offline"
speeds:
- "off"
- "low"
- "medium"
- "high"