Get rid of old entity ID's after Shelly's are already deleted

Recently I deleted a bunch of Shelly’s from the Shelly-integration because the Shelly’s and HASS were moved to a new network with new VLAN’s and subnets, I figured deleting and re-adding the Shelly’s was the easy solution with new subnets and IP-addresses in mind.

When I re-added the first Shelly, the old entity ID’s reappeared, so it seems deleting a Shelly from the integration doesn’t actually delete everything.

After some Googling I found the locked Integration's entities remain in entity registry even after its removal · Issue #22209 · home-assistant/core · GitHub and I’ve already manually tried deleting the orphaned entries (which there were a lot of) from core.device_registry and core.entity_registry (followed by a full restart of the host) but the old ID’s keep popping back up.

Besides doing a clean install of Home Assistant, how can I do a proper cleanup so that the old entity ID’s of deleted devices (in general, not just Shelly’s) are actually gone?

Go to Settings → Devices & Services → Entities List.

Using button the top left filter by “Not provided”.

Delete the entities in bulk by selecting them and using the three dots menu.

@tom_l I wish it was that simple, none of the Shelly entities of the deleted devices (that can be found in core.device_registry and core.entity_registry ) are shown there.

Did already try manually deleting all entries of a specific Shelly from those files, followed by a restart of the VM with Home Assistant: The old entity ID’s (and names) still popped up upon re-adding that Shelly.

Could it be that home-assistant_v2.db is involved as well with the coming back of entities of deleted devices?

I recently had some mqtt entitites I was unable to delete and SPOOK pointed out I still had them in some of my dashboard cards.

Spook does mention such things, eg a hyperlink to the dashboard. Perhaps you’ll give it a try, could be found in HACS.

@justone I came across Spook while Googling (forgot to mention that) and it was able to give me a list of those (and a whole bunch of other) orphaned entities, but when it came to the actual deleting: Delete all orphaned entities not working · Issue #827 · frenck/spook · GitHub , no luck there. Did saw a reference to a script on that page to Entity management - Spook 👻 a scary powerful toolbox for Home Assistant. (ctrl+F “Script to remove entities from database”)

alias: Delete orphaned database entities
sequence:
  - action: homeassistant.list_orphaned_database_entities
    response_variable: orphaned
  - action: recorder.purge_entities
    target:
      entity_id: |
        {{ orphaned.entities }}
    data:
      keep_days: 0
mode: single

After running that “List all orphaned database entities” came up empty, but the entities can still be found in core.entity_registry (and trace.saved_traces but that file looks unrelated to the problem) and home-assistant_v2.db. The only thing that seems to have changed in one of the files in config.storage\ is that the orphaned entities list got cleared out, which didn’t solve the problem.

When you say this … do they appear within the “entities” section or within the “deleted_entities” section in that file?

If in deleted nothing you must care about.

In the deleted_entities section, I already tried manually removing the entries from there (followed by a restart), but HASS seems to be restoring them from somewhere.

Just now I wrote a Bash-script that - based on a list of partial entity names and MAC-addresses (with and without : ) - searches each file in config.storage\ and then deletes the lines that contain a match.

I used a backup copy of the files to test that on, at first glance this crude method seems to work (though a bit slow), but since HASS restored old entitie ID’s and names even after manually deleting lines of a specific Shelly, I’m wondering if there are other locations/files that contain leftover data of entities.

Home-assistant_v2.db sure does, but are there any others?

edit:
@ justone
I just ran my script for a cleanup, did a check if there wasn’t too much deleted (false positive), also deleted Home-assistant_v2.db to start with a clean slate and … still old entity ID’s and names popping up.

edit2: Everything I deleted seems to have been restored in the storage directory and yes I stopped Home Assistant (source /etc/profile.d/homeassistant.sh ; ha core stop) before the editing.

edit3: Huh, instead of issuing the restart command through SSH (like I always do when replacing the Let’s Encrypt certificate) I did this in the VM, now it seems to work?!

Restart in HA does a restart. VM does a reboot.

They’re 2 different things & I’m guessing db changes require a full reboot.

@ShadowFist Woops, I meant stop, tried to do ha core stop using SSH, then clean up the files and then ha core start, while after SCP’ing a new Lets Encrypt certificate I always have a script do ha core restart.

It seems “ha core [command]” doesn’t work through SSH? → 'ha core status' runs from interactive ssh but not from ssh commandline