No update entity for my ESPhome devices

Don’t think so.

The esphome dashboard integration is the communication interface between the add-on and HA which tells about the latest esphome (dashboard) version installed so HA can check if all nodes are up 2 date.

But if you try the search engine some user actually rebuild that function not long ago for non add-on installs…

I tried the search for you and this was the first result: