Unable to remove entities

I’ve just implemented Z2M and MQTT in my HA, I have had to factory reset a TRV a few times and also renamed this and added to HA.

For some reason I have some left over entities that don’t show under the MQTT integration and can’t be removed via the GUI. These are showing up as read only entities.

I run HA Core in docker with a separate backend SQL DB. Do I need to go into the DB and remove these entities or is there another way?