You can’t do it entirely via the UI, but if you edit the yaml of your trigger you can specify ‘not_from’. See the documentation here: Automation Trigger - Home Assistant
Here’s one of my automation triggers as an example:
trigger:
- platform: state
entity_id:
- alarm_control_panel.home_alarm
id: MANUALLY_DISARMED
to: disarmed
not_from:
- unknown
- unavailable