[Guide] Migrate / copy energy statistics & history to new entity - keep full history in Energy Dashboard (2026)

[Guide] Migrate / copy energy statistics & history to new entity - keep full history in Energy Dashboard (2026)

Preamble

My setup

I have a few shelly’s EMs in my switch box monitoring all major loads in my house:

  • Mains
  • Wall Connector (EV Charging)
  • A/C
  • Hot Water Cylinder
  • Oven/HOB

There are also miscellaneous power plugs / native integration that monitor other loads (such as Fridge, Washer, Dryer, Server room, etc).

All of them setup in energy dashboard as is (i.e what-ever ‘energy’ a given entity integration provides)

Problem

First time I’ve faced limitation with this setup - when server’s room TPLink EM plug stopped reporting sane values, and I had to replace it with a new EM plug,
Once I’ve updated the entity to the new plug - old values no longer appeared in the dashboard.
But back then I’ve written off this issue, and moved on.

Over the years my I’ve expanded my energy monitoring, and coming to the present day - similar issue about to happen once again, but with my ‘mains’ entity (provided by shelly integration directly),
The problem is, I want to ‘split’ my mains into ‘peak’ and ‘free’ energy entities, but I DO want to retain all my history (I want to move current values into new peak entity).

More visually, what I want is:

  • sensor.shellyem_244cab417eaf_channel_1_energy ← has all historical data for ‘mains’ and current live data

Using Utility Meter integration I’ve created 2 entities:

  • sensor.mains_utility_test_peak
  • sensor.mains_utility_test_free

But I want sensor.mains_utility_test_peak to ‘inherit’ all historical values from sensor.shellyem_244cab417eaf_channel_1_energy up till now

So when I replace the old mains entity with 2 new once in the energy dashboard - all my historical data will be intact.

After countless googling and reading forums - I’ve come to conclusion that other answers - no longer apply to 2026.1.3 HA version which I’m running at the time of writing this post.

So I’ve decided to poke into HA’s internals and figure out myself how to COPY the data from one entity to another, preserving the old entity data too.

Migrating ‘cost’ values

If you want to migrate cost entities (which energy dashboard creates under the hood) for entities that track cost (i.e Grid consumption) and not individual devices.

Make sure to add your new sensor to the Energy dashboard,
Most likely you will see warning as such:

Statistics not defined. Some entities currently have no statistics metadata. 

That’s fine, we just need to wait since the sensor does not have new data yet.

Once this warning goes away (should be as soon as your entity has some valid data in it)

Then go to Settings → Entities → search for ‘cost’ → wait till your new cost entity shows up, and gets a few data point updates. (i.e wait till you have 5 min aggregated graph)

For ‘cost’ entities we only need to do SQL migration.

Solution

Create ‘proxy’ entity (Utility meter)

As mentioned before, once you start using an entity in HA’s energy - you are locked in into that entity, so we want to avoid using entities that are provided by devices itself, as if that device dies/needs replacement/whatever - by switching to another entity you lose the data,

So the solution is to ‘create’ a proxy entity, which will point to current entity.

For energy dashboard - I’ve setted on Utility meter - as it can work for cases such as: mains(where you may need to count peak/off-peak) as well as individual devices.

This step is pretty self explanatory, create new helper - ‘Utility meter’ - setting up device that is providing the readout, as well as tariffs if needed)

For me, since I’m planning to ‘copy’ data into ‘peak’ entity.

Make sure to let HA run for a bit, so new entity can populate some data, so this entity will be easily visible in database and HA’s config files.

SQL Shenanigans

Needles to say, what we are gonna is risky and may break in the future, and we may corrupt HA state/db, so please do FULL backup, test migration in not production instance of HA, and then if feeling ok - do this migration on production system.
Also, do all manipulation that I will be talking here, while HA is off, so we don’t conflict with a running system.

Here is my SQL migration file, that will do:

  • Delete all data in ‘new’ entity
  • Copy long term statistics from OLD into NEW
  • Copy short term statistics from OLD into NEW
  • Copy ‘states’ history from OLD into NEW

Copy these SQL statements into a file on your PC, and do a bit of find-replace.

  • Find: sensor.shellyem_244cab417eaf_channel_1_energy | Replace this ID with ID of your old entity (where we will be copying data from)
  • Find: sensor.mains_utility_test_peak | Replace this ID with ID of your new entity (where we will be copying data into, and deleting old data)
-- Pre checks for old entity
SELECT
    id AS old_metadata_id,
    statistic_id,
    (SELECT COUNT() FROM statistics WHERE metadata_id = statistics_meta.id) AS long_rows_old,
    (SELECT COUNT() FROM statistics_short_term WHERE metadata_id = statistics_meta.id) AS short_rows_old
FROM statistics_meta
WHERE statistic_id = 'sensor.shellyem_244cab417eaf_channel_1_energy';

-- New entity (must exist already)
SELECT
    id AS new_metadata_id,
    statistic_id
FROM statistics_meta
WHERE statistic_id = 'sensor.mains_utility_test_peak';

-- Check if new already has data
SELECT COUNT(*) AS new_has_long
FROM statistics
WHERE metadata_id = (SELECT id FROM statistics_meta WHERE statistic_id = 'sensor.mains_utility_test_peak');
SELECT COUNT(*) AS new_has_short
FROM statistics_short_term
WHERE metadata_id = (SELECT id FROM statistics_meta WHERE statistic_id = 'sensor.mains_utility_test_peak');


-- Delete old data for the new entity
DELETE FROM statistics
WHERE metadata_id = (
    SELECT id
    FROM statistics_meta
    WHERE statistic_id = 'sensor.mains_utility_test_peak'
);

-- Delete old data for the new entity
DELETE FROM statistics_short_term
WHERE metadata_id = (
    SELECT id
    FROM statistics_meta
    WHERE statistic_id = 'sensor.mains_utility_test_peak'
);

-- Copy short_term from old entity to new entity
INSERT INTO statistics_short_term (
    created,
    created_ts,
    metadata_id,
    start,
    start_ts,
    mean,
    min,
    max,
    last_reset,
    last_reset_ts,
    state,
    sum,
    mean_weight
)
SELECT
    created,
    created_ts,
    (SELECT id FROM statistics_meta WHERE statistic_id = 'sensor.mains_utility_test_peak') AS metadata_id,
    start,
    start_ts,
    mean,
    min,
    max,
    last_reset,
    last_reset_ts,
    state,
    sum,
    mean_weight
FROM statistics_short_term
WHERE metadata_id = (
    SELECT id
    FROM statistics_meta
    WHERE statistic_id = 'sensor.shellyem_244cab417eaf_channel_1_energy'
);

-- Verify duplicate (counts should match old short_rows_old)
SELECT
    COUNT(*) AS short_duplicated,
    MIN(start_ts) AS oldest,
    MAX(start_ts) AS newest,
    MAX(sum) AS max_sum  -- should match old max
FROM statistics_short_term
WHERE metadata_id = (
    SELECT id FROM statistics_meta WHERE statistic_id = 'sensor.mains_utility_test_peak'
);


-- Copy long term from old entity to new entity
INSERT INTO statistics (
    created,
    created_ts,
    metadata_id,
    start,
    start_ts,
    mean,
    min,
    max,
    last_reset,
    last_reset_ts,
    state,
    sum,
    mean_weight
)
SELECT
    created,
    created_ts,
    (SELECT id FROM statistics_meta WHERE statistic_id = 'sensor.mains_utility_test_peak') AS metadata_id,
    start,
    start_ts,
    mean,
    min,
    max,
    last_reset,
    last_reset_ts,
    state,
    sum,
    mean_weight
FROM statistics
WHERE metadata_id = (
    SELECT id
    FROM statistics_meta
    WHERE statistic_id = 'sensor.shellyem_244cab417eaf_channel_1_energy'
);

-- Verify duplicate (counts should match old long_rows_old)
SELECT
    COUNT(*) AS long_duplicated,
    MIN(start_ts) AS oldest,
    MAX(start_ts) AS newest,
    MAX(sum) AS max_sum  -- should match old max
FROM statistics
WHERE metadata_id = (
    SELECT id FROM statistics_meta WHERE statistic_id = 'sensor.mains_utility_test_peak'
);

-- pre-checks for states (old entity)
SELECT
    metadata_id AS old_metadata_id,
    entity_id,
    (SELECT COUNT(*) FROM states WHERE metadata_id = states_meta.metadata_id) AS states_rows_old
FROM states_meta
WHERE entity_id = 'sensor.shellyem_244cab417eaf_channel_1_energy';

-- pre-checks: new states_meta exists (new entity)
SELECT
    metadata_id AS new_metadata_id,
    entity_id
FROM states_meta
WHERE entity_id = 'sensor.mains_utility_test_peak';

-- Clear existing states for a new entity
DELETE FROM states
WHERE metadata_id = (
    SELECT metadata_id
    FROM states_meta
    WHERE entity_id = 'sensor.mains_utility_test_peak'
);

-- Copy states from old entity to new entity
INSERT INTO states (
    entity_id,
    state,
    attributes,
    event_id,
    last_changed,
    last_changed_ts,
    last_reported_ts,
    last_updated,
    last_updated_ts,
    old_state_id,
    attributes_id,
    context_id,
    context_user_id,
    context_parent_id,
    origin_idx,
    context_id_bin,
    context_user_id_bin,
    context_parent_id_bin,
    metadata_id
)
SELECT
    'sensor.mains_utility_test_peak'                  AS entity_id,          -- new entity
    state,
    attributes,
    event_id,
    last_changed,
    last_changed_ts,
    last_reported_ts,
    last_updated,
    last_updated_ts,
    NULL                                               AS old_state_id,       -- break chain
    attributes_id,                                                                -- keep shared attrs
    context_id,
    context_user_id,
    context_parent_id,
    origin_idx,
    context_id_bin,
    context_user_id_bin,
    context_parent_id_bin,
    (SELECT metadata_id
     FROM states_meta
     WHERE entity_id = 'sensor.mains_utility_test_peak') AS metadata_id
FROM states
WHERE metadata_id = (
    SELECT metadata_id
    FROM states_meta
    WHERE entity_id = 'sensor.shellyem_244cab417eaf_channel_1_energy'
);

-- Verify state copy
SELECT
    COUNT(*) AS states_duplicated,
    MIN(last_updated_ts) AS oldest_ts,
    MAX(last_updated_ts) AS newest_ts
FROM states
WHERE metadata_id = (
    SELECT metadata_id
    FROM states_meta
    WHERE entity_id = 'sensor.mains_utility_test_peak'
);


-- Last state for old sensor to copy into new entity restore_state
SELECT state from statistics_short_term where metadata_id = (SELECT id FROM statistics_meta WHERE statistic_id = 'sensor.shellyem_244cab417eaf_channel_1_energy') ORDER BY start_ts DESC LIMIT 1;

Run migration

You can run the migration with command like this (Linux):

sqlite3 /PATH/TO/YOUR/HA/DB/home-assistant_v2.db < /PATH/TO/MIGRATION/FILE/migration.sql

Alternatively, while developing this script, I was using DataGrip, which can execute this from it’s console, select all and press Ctrl+Enter.

Migrating cost

If you are migrating/copying cost values too, you have these additional steps to follow.

Copy the migration file, and do search replace like above, BUT adding _cost at the end of entities.

For example: sensor.shellyem_244cab417eaf_channel_1_energy will become sensor.shellyem_244cab417eaf_channel_1_energy_cost
And likewise, sensor.mains_utility_test_peak will become sensor.mains_utility_test_peak_cost

State file shenanigans

After running SQL migration script, the very last line of the output will print the last value of your old sensor (from which we have copied the data)
We will need this value in this step. For example, my value was 26709.2073

Open config/.storage/core.restore_state file and find your new sensor entity id,

It should look something like this:

    {
      "state": {"entity_id":"sensor.mains_utility_test_peak","state":"70.4808","attributes":{"state_class":"total_increasing","status":"collecting","last_period":"0","last_valid_state":"26759.7279","tariff":"Peak","last_reset":"2026-01-31T19:46:16.269642+00:00","unit_of_measurement":"kWh","device_class":"energy","friendly_name":"Mains Utility Test Peak"},"last_changed":"2026-02-02T07:40:59.787910+00:00","last_reported":"2026-02-02T07:40:59.787910+00:00","last_updated":"2026-02-02T07:40:59.787910+00:00","context":{"id":"01KGEMRDGB3B1HMMEMTSDE2Z8X","parent_id":null,"user_id":null}},
      "extra_data": {
        "native_value": {
          "__type": "<class 'decimal.Decimal'>",
          "decimal_str": "70.4808"
        },
        "native_unit_of_measurement": "kWh",
        "last_period": "0",
        "last_reset": "2026-01-31T19:46:16.269642+00:00",
        "last_valid_state": "26759.7279",
        "status": "collecting",
        "input_device_class": "energy"
      },
      "last_seen": "2026-02-02T07:41:13.811823+00:00"
    },

Now replace state, decimal_str, last_valid_state (2 fields) - with the value we have got from SQL migration.
(Note: if your last_valid_state is set to None/null - there is no need to change it then)

So it should look something like this now:

    {
      "state": {"entity_id":"sensor.mains_utility_test_peak","state":"26709.2073","attributes":{"state_class":"total_increasing","status":"collecting","last_period":"0","last_valid_state":"26709.2073","tariff":"Peak","last_reset":"2026-01-31T19:46:16.269642+00:00","unit_of_measurement":"kWh","device_class":"energy","friendly_name":"Mains Utility Test Peak"},"last_changed":"2026-02-02T07:40:59.787910+00:00","last_reported":"2026-02-02T07:40:59.787910+00:00","last_updated":"2026-02-02T07:40:59.787910+00:00","context":{"id":"01KGEMRDGB3B1HMMEMTSDE2Z8X","parent_id":null,"user_id":null}},
      "extra_data": {
        "native_value": {
          "__type": "<class 'decimal.Decimal'>",
          "decimal_str": "26709.2073"
        },
        "native_unit_of_measurement": "kWh",
        "last_period": "0",
        "last_reset": "2026-01-31T19:46:16.269642+00:00",
        "last_valid_state": "26709.2073",
        "status": "collecting",
        "input_device_class": "energy"
      },
      "last_seen": "2026-02-02T07:41:13.811823+00:00"
    },

Test it

Done! You can now boot up your (test) instance of home-assistant and:

  • Check that new entity has got history copied over
  • Also, suggest to wait until it has got a new reading update to make sure after it gets new value - there are no issues.
  • As I’ve noticed, if we don’t modify the core.restore_state file - it would seem fine, until new reading comes along, and then state would reset.
  • Go to energy dashboard and remove old sensor, while adding new sensor.
  • All the history should be intact

If you are happy with the results, do this migration again, but on production home assistant :slight_smile:

My 2 cents

From now on, if I want to add a new entity into energy dashboard - I must create a Utility Sensor first, so it acts as a proxy, so in case I need to switch the actual device that is monitoring - I would just update Utility Sensor which entity it’s referencing, and not execute any SQL migrations :slight_smile:

Result

Here is a quick result after doing the migration - as you can see the data matches up between old and new entity.

Hope this helps someone!

8 Likes

Thanks! It is exactly what I’ve been looking for.
So far, so good and energy panel looks ok with utility meters with copied statistics…

1 Like

This is cool @deishelon - It would be helpful if this “need” would be thought as an actual functionality/feature in HA (migrate statistics from old to new entities) - Maybe someone already placed this in their backlog.

Reading your share:

  • What do you mean by “do all manipulation that I will be talking here, while HA is off”?
  • What is your current setup (installed in VM, RPI?) and how do you possibly do this (connect to the HA DB)?

I’m running HA in docker container, the host is a Linux box. So ‘off’ means docker compose down so HA is not running while we are touchng db.

If you are running in VM - it would mean stopping the HA VM.

Im not connecting to HA DB, we are manipulating the HA DB file that is stored in disk, this is why we need to ‘stop’ HA while we are touching this DB file, so we don’t have a situation where HA writes to that file at the same time as we do.

@deishelon Ok understood (concurrency with sqlite db and config/.storage/core.restore_state).
Im running on HA OS in a RPI4. Now sure how I can do the “HA off” in this case… Will check if its possible.

SSH Into the RPI, then seems like you can use this CLI: Stop HA with HASSIO to stop/start etc.

If that does not work, can always shutdown RPI, pull SD Card, mount the SD card on your Linux workstation, and assess the files like that.

I wish this proxy concept would be standard integrated into HASS. Long term energy statistics should not be dependent on entities that can change over time. Might there already be a feature request for this we can vote on?

Edit: Found at least one here

Yea, it needs to be more streamlined.
But for now Utility meter in front of every sensor is a way to go.

Thanks for putting this detailed guide together! Figuring out the database structure for LTS is definitely a headache.

For anyone who prefers a UI-based approach and wants to avoid manual SQL queries or database browsers, I actually built a custom HACS component that automates this exact process: HA Merge Sensor History.

It handles exactly what this guide describes: migrating both the raw states and the long-term statistics used by the Energy Dashboard, but directly through the Home Assistant UI.

Quick overview of what it does:

  • Sidebar Panel: Adds a “Merge History” UI where you just select your Source and Destination entities.
  • Safe: It uses atomic database transactions (if anything fails, it fully rolls back) and only imports data older than the destination’s oldest record, so you never get duplicates.
  • Bulk Import: If you’re replacing a whole system (like moving away from Ecowitt) and have dozens of sensors, you can paste a comma/tab-separated list to migrate them all at once.

You can grab it via HACS Custom Repositories (https://github.com/mayerwin/HA-Merge-Sensor-History). Just remember to still make a database backup before doing any migrations!

1 Like

this is awesome
thanks to your post, I’ve found now how to get an entities counter to start with an offset with Homeassistant off :slight_smile:
@ErwinHA , you code is great but it’s not fully covering what @deishelon is doing here as the key things is to have new counter entities to follow up on the imported old data

thanks to both of you

1 Like

Thank you for this! It works really great

1 Like

Can you elaborate what is missing in ErwinHA’s solution? To me it looks like its a complete solution to migrate data to a new entity.

you should focus on the part named " Shenanigans"

basically, your timeserie of data should never decrease or reset for energy dashboard

the only way to achieve that when you import an old part of history data to an ongoing one is
-to play with the core.restore_state in /.homeassistant/.storage if needed
-to use some SQL command to do some math to add the offset to the value if your curent counter has reset
-to delete all the states data with homeassitant stopped
-to check all your history and history_short_term data: you need to check state and sum column are increasing regularly without drop/reset

Again, the challenge is not to import or copy data from one sensor to another one. it’s more to have a working sensor timeserie for Energy dashboard

example
homeassistant_db=# SELECT
start_ts,
to_timestamp(start_ts) AS start,
state,
sum,
min,
max
FROM statistics_short_term
WHERE metadata_id = 2049
ORDER BY start_ts;
start_ts | start | state | sum | min | max
------------±-----------------------±------±------±----±----
1776848100 | 2026-04-22 08:55:00+00 | 1.032 | 1.032 | |
1776848400 | 2026-04-22 09:00:00+00 | 1.032 | 1.032 | |
1776848700 | 2026-04-22 09:05:00+00 | 1.032 | 1.032 | |
1776849000 | 2026-04-22 09:10:00+00 | 1.032 | 1.032 | |
1776849300 | 2026-04-22 09:15:00+00 | 1.032 | 1.032 | |
1776849600 | 2026-04-22 09:20:00+00 | 1.032 | 1.032 | |
1776849900 | 2026-04-22 09:25:00+00 | 1.032 | 1.032 | |
1776850200 | 2026-04-22 09:30:00+00 | 1.032 | 1.032 | |
1776850500 | 2026-04-22 09:35:00+00 | 1.032 | 1.032 | |
1776850800 | 2026-04-22 09:40:00+00 | 1.032 | 1.032 | |
1776851100 | 2026-04-22 09:45:00+00 | 1.032 | 1.032 | |
1776851400 | 2026-04-22 09:50:00+00 | 1.032 | 1.032 | |
1776851700 | 2026-04-22 09:55:00+00 | 1.032 | 1.032 | |
1776852000 | 2026-04-22 10:00:00+00 | 1.032 | 1.032 | |
1776852300 | 2026-04-22 10:05:00+00 | 1.032 | 1.032 | |
1776852600 | 2026-04-22 10:10:00+00 | 1.032 | 1.032 | |
1776852900 | 2026-04-22 10:15:00+00 | 1.032 | 1.032 | |
1776853200 | 2026-04-22 10:20:00+00 | 1.032 | 1.032 | |
1776853500 | 2026-04-22 10:25:00+00 | 1.032 | 1.032 | |

I see. You mean the “State file shenanigans” section which configures the new counter state of the replaced energy counter.

If that’s all what is missing in @erwinha’s otherwise neat solation, I am sure this can be incorporated as well into the tool.

1 Like

@jps-34, have you actually tried @ErwinHA’s tool?

From the tool’s github log it appears that its already doing energy sensor continuity by applying an offset like you suggested as one solution:

  • Cumulative-sum offset for energy sensors (has_sum=True): imported sum
    values are shifted by dest.sum - source.sum at the splice point so
    the imported series joins the destination’s existing series smoothly
    (no jump or drop in the energy dashboard).

well the offset is not enough

you need to update the stored cached and make sure the progression of data value is always increasing, if not your energy dashboard is broken

more important is to take care of value between sum and state, this one is the tricky one.

I’ve solved my issue with a full manual SQL maintenance which took 3 days :slight_smile:

I’m pretty sure we will have a full working git but it will request work as this code would have to be resilient / idempotent and more import to be able to adress many different situations

cheers