You have to adjust your start time. now() always returns the current day. So when midnight hits, it’s now set to today at hour 20. So you should subtract 1 day from the start time if the current time is less than the start time.
- platform: history_stats
name: PowerUp
entity_id: binary_sensor.power_up
state: "on"
type: time
start: >
{% set t = now() %}
{% set start = t.replace(hour=20, minute=0, second=0) %}
{% if start > t %}
{{ start - timedelta(days=1) }}
{% else %}
{{ start }}
{% endif %}
duration: 12:00