I have a data provider that exposes the previous day’s data at (roughly) midnight.
I would like to ingest this data as if it had been received live.
I’ve tried creating a sensor - but it doesn’t appear Home Assistant has any ability to provide a timestamp for reference with the data. The value is the value, now.
I’ve tried MQTT publishing the data - but again - can’t seem to travel in time.
Some more examples of the data are below.
There is no way to bring this data in “live” as the power readings are not available until the next (sometimes next business) day.
Surely there is a way to bring in Time Series data when it’s not available live?
I don’t think there is… but it also seems there’s no desire to implement something like this.
It’s a little frustrating with the “Energy” push in the current version - there are millions of people that have access to their energy data, but it’s delayed by 24 hours.
You could implement that by changing the core and it looks worthwhile.
The simplest would be a new method on the recorder interface that would insert this data into the state history table. E,g, batch_load_state
You could also, just connect to Postgres and stick it in the table (assumes you are using Postgres for recorder)
Feeding it into the state machine as current values will be problematic because HA is not a change based system it is a current state system. For example if the state updates 3 times very quickly, the clients will likely only see the last value. AFAIK there is no good way to provide a queue of state changes in the system
Hi there @mpschr
Sadly - there is no dev/community interest in adding this, and I barely have time to blink my eyes at the moment - so I can’t really contribute anything of value.
I’ve not come up with a creative way around it either.
I am working on DIY sleep/breath monitor and figured out that I don’t need to send data from ESP32 in realtime, and decided to implement a batch sync in the morning.
Thus, I faced this issue too. And looks like I just found a method to attach a timestamp to a value
It looks quite simple - you just have to send a bunch of events like this, with your entity id (which should be already created in system), state value, and last_updated timestamp.
add yesterday’s sum to the running total (this is what I do now, and the statistics say everything happened today at 6am), or will the hourly events add to the total?
loop through the hourly data, emitting events for each hour
emit one more event to subtract the subtotal from this morning’s 6am data from the statistics?
In my case, the daily data is added to an input_number that feeds a template sensor, and is reset to 0 monthly on the billing date
I was able to set data for dates both in the past and the future. Although it did not update the value in the UI, it did update the history graph in the recorder, which is mostly what I care about.
Edit: See my thread for more details, but as of now, I think it’s not possible to send these Events from scripts or automations. It seems to only work from the dev tools. This is due to how HA is storing the new state as a Python Dictionary instead of a HA State object when it puts the Event on the bus.
If helpful, I posted some reasonably robust Python code to insert backdated state data into the states table. The source can be either a CSV file or a SQLite db – with 2 columns (one for timestamp in UTC, other for data to be entered)