As you surmised, the entity’s state is probably unknown upon startup when the template sensor is first evaluated. However, the error message is produced because of this:
value_template: '{{ states.remote.family_room.attributes.current_activity }}'
When you explicitly request an entity’s state or attribute, whose value is unknown, the evaluation halts and throws an error.
In contrast, if you use a function to request the entity’s state or attribute, whose value is unknown, the function returns None and evaluation continues.
value_template: "{{ state_attr('remote.family_room', 'current_activity') }}"
To make you code more fault-tolerant, it’s preferable to use the states and state-attr functions.
Regarding the icon_template, here’s another way to do it:
icon_template: >-
{% set activities =
{ 'Dish':'mdi:satellite-uplink',
'Xbox':'mdi:xbox',
'Roku':'mdi:alpha-r-box-outline',
'Blu-ray':'mdi:disc-player',
'TV':'mdi:television-classic',
'PowerOff':'mdi:power-plug-off',
'None':'mdi:help-circle' } %}
{% set current = state_attr('remote.family_room', 'current_activity') %}
{% set icon = activities[current] if current in activities %}
{{icon}}