Importing historic values (async_import_statistics) resulting in entries at HH:26

Development
1

Hey there,

It’s my first time trying to hack together a custom integration (which I’ll eventually intend to share on the community store, but it’s too hacky so far).
It’s a relatively straightforward idea: my electricity provider provides a web interface for historic consumption data (kWh in 15min intervals). Unfortunately, there’s a significant delay in the data (multiple hours).
So my plan was to build on top of GitHub - klausj1/homeassistant-statistics: This HA integration allows to import long term statistics from a file like csv or tsv to import historic values. I understand that the long term statistics are only possible with hourly resolution, and I’m okay with that (I simply combine all entries of an hour up to just one entry). But the result of my code is something weird…

history_26
history_261209×404 19.5 KB

As you can see, the entries somehow start at the 26th minute, not the 0th minute. This is thoroughly confusing me. Furthermore, no matter what I try with setting sum and state and last_reset on the StatisticsData, what I end up with in the energy dashboard is a whole load of nonsense:

energy_dashboard
energy_dashboard1379×435 15.1 KB

I’m frankly lost with this. There’s not a single negative state (I don’t produce electricity. It’s all consumption from the grid) in the data, yet the energy dashboard thinks there is??

I’m hoping for a miracle that somebody here encountered something similar before and can point me to the right direction. I’m sharing my code just in case, but beware: it’s a mess, as I’m a bit out of my comfort zone with python and the magic in the home assistant code base (I would have expected quite a bit more “hooking things up”, but apparently with the right method and file names(??) magic happens).

Here are the relevant code pointers:
adding statistics
entity

Any help is greatly appreciated.

Best regards,
Stefan

2

Have you tried putting breakpoints in your code to see why it’s getting a :26 timestamp?

As for how the energy dashboard displays the values, please show a dump here of example data of what values your integration is inserting into statistics, e.g. for start, state, sum.

IIRC state does not matter, and sum must be always increasing.

3

After deleting the entire DB (lucky me didn’t run this in production) and doing a fresh import, no :26 entries appear anywhere. I’m still puzzled by it though because the import_statistics method explicitly requires MM:SS to be 00:00.

Now I just have to figure out how to query the history, such that I can minimize the API calls to the electricity provider (otherwise they might get unhappy with me). Internally to the code, _statistics_exists seems to be there, but I don’t see an equivalent in the public API.