Import old energy readings for use in energy dashboard

Feature Requests
4

Same here, I’m using Home Assistant for a few years and now my energy provider allows me to download historic energy data as simple CSV files with , e.g.:

Date;Time;Data [kWh]
2021-09-04;00:15:00.000;0,152;
2021-09-04;00:30:00.000;0,150;
2021-09-04;00:45:00.000;0,150;

It would be great to import this historic data to make it available for use in the energy dashboard.

2 Likes
5

I would also like this. I have solar panels since Sep’ 20 and have always monitored them with Shelly’s. I also have a Shelly EM that measures the “Grid Import” / “Grid Export”, so I have all data needed for this.

The Shelly Cloud has the possibility to download all history into a .csv file. This file contains accumulated energy with 10 minute intervals (in my grid Shelly: import and export are separated).

1 Like
6

@fversteegen you might be interested in this feature request as well, since it is strongely related to this one:

1 Like
7

I support the request to be able to import historical values since energy consumption on hourly basis is only available from last day in Quebec.

8

:+1: from me!
I am getting my readings (imported & exported energy + gas readings) from my supplier’s Restful API (Octopus Energy).
These readings are for 30 minutes blocks, however they are only available up to a previous day (so effectively they are always delayed by up to a day)
That would be very handy if I could use import these data in Home-Assistant!

1 Like
9

Just wanted to note for any interested developers it seems there is a way for an integration to add data in the past to history without changing the current state of entities. It can be done by using async_add_external_statistics:

There are currently two integrations in core making use of this if you’re looking for an example, Demo:

And tibber:

To be clear, this isn’t a solution to this FR. But if anyone is interested in building an integration to contribute to core or make available in HACS which does import of historical data I think it can be done using this API in core without the need to write a lot of custom SQL.

13 Likes
10

It seems that there is an import api through websocket events api recorder/import_statistics

11

When playing with the demo-integration, it seems that the async_add_external_statistics always creates a new entity and thus it seems to be unable to link the historical data to the already existing sensor entity. As a result, after importing the historical data, you will have two entities (one containing the imported data, one containing the measured-by-HASS-data) which is annoying.

12

yes, but websocket events api recorder/import_statistics
switches import method based on the id (xyz:something = external, xyz.somehting = existing sensor)

if valid_entity_id(metadata["statistic_id"]):
        async_import_statistics(hass, metadata, stats)
    else:
        async_add_external_statistics(hass, metadata, stats)

I successfully imported external data to an external entity, and can update it. But having some problems with continuity. Only way I was able to have the right charts is to get the sum of the last inserted value and continue inserting increasing sums without ever resetting. The last_rest does not seem to work, at least for yearly energy graph.

2 Likes
13

Thanks, I am now using async_import_statistics to import my energy statistics, which seems to work perfectly.

14

Any chance you might do a mildly detailed run through of how you do this? Dominion recently began offering detailed statistics for my area, though there’s no way to hoover them into HA that I can see. I’d love to be able to track them in HA if possible.

Thanks.

3 Likes
15

same request here :slight_smile:
Could you please give us some details on the procedure ?

Thx a lot !!!

1 Like
16

I am still in the process of improving my script.

I have one part which parses my energy supplier’s Dashboard and stores them in a database.
I then have additional scripts to aggregate and send the data to influxdb or/and home assistant.
For home assistant I aggregate the data to hourly values, since this is the smallest unit supported in the statistics history screen.

The websocket api upload can be done with any programming language. I used Javascript/Node here the relevant function node.js script to upload external data to home assistant via the websocket api · GitHub

It retrieves the last current value for a date and appends the new values by adjusting the sum based on this existing stat. It is the only way I found to have graphs without errors, because any combination of reset with last_reset did not work.

I use long lived access token which can be generated in your profile page in home assistant.

2 Likes
17

My integration is still a work in progress, but the part that is doing the import of the statistics in the existing sensor is here: huawei_fusionsolar/sensor.py at ea2b58ee8a537b02ab1a367107f77c5960ac9f7a · janmolemans/huawei_fusionsolar · GitHub

18

So this script wouldn’t exactly work if I need to prepend data?

I am looking to fill my historical energy values from my energy supplier, from before I was monitoring my home meter.

19

The websocket api can be used to add data to existing entites you must use the “type.name” syntaxe instead of “external:name”. But you might get problems with sum changes between imported data and the one already inserted.

20

Wow, this looks like far beyond my skills …

On my side I have a year of data to import that i can manipulate easily (CSV file)

Would one of these scripts be able to do that ?

21

made a WTH entry, suggest all vote to get this on the radar

22

With already 86 votes I have good hopes this will be picked up for the month of What the heck :smiley:

Only issue is that this one is in the wrong forum. Can one of the admins move this to wth? @frenck @petro ?

1 Like
23

This is a feature request, not a WTH. It will not be moved.

1 Like