The ESPHome sensors should have last_updated and last_changed attributes that HA creates that you can work with further on the HA side (for instance create further template sensors from them).
(Edit: my use of the term attribute above is wrong here I think).
Search for those terms for more examples on the forum.
A Status sensor with a delayed_off filter could be another approach.