I’m using Harmony to control ATV, and I have off.

Not using the remote from ATV, but apparently there is off no doubt… at least Harmony can turn off the atv just fine and the media_player.atv would read off.
Furthermore, I did a quick template sensor, so that the sensor would say off when off, and would say what is running when playing.
template:
- sensor:
- name: "ATV"
unique_id: atv_playing
state: >
{% if is_state ('media_player.atv', 'playing') %}
{{ state_attr('media_player.atv', 'app_name') }}
{% elif is_state ('media_player.atv', 'paused') %}
{{ state_attr('media_player.atv', 'app_name') }}
{% else %}
{{ states('media_player.atv')}}
{% endif %}
attributes:
data_source: "media_player.atv"