Many thanks Michael for taking the time to read this and try to figure out a fix. I have been trying to google this as well with no luck (this post came after a few days of trying to figure it out without asking). What you’re suggesting seems like a good alternative (if i understood it correctly), replace the binary sensor with whichever other type is best best suited to hold the time in the form of a date entity.
Edit: just found those two posts that address the issue of adding state changes at specific times instead of now:
https://community.home-assistant.io/t/import-old-energy-readings-for-use-in-energy-dashboard/341406
https://community.home-assistant.io/t/improved-support-for-long-term-historic-data/379659