That is something that I did not try.
It might be that ESPhome stores some info in the NVmemory on the ESP and this causes entities not to fully vanish although they are not existing in the config anymore.
I will try with one ESPhome device to completely zero out the device (need to figure how) and then program the same setup.