If you do a full upgrade between major versions of the OS all packages which were put on hold before have to be set to unhold before initiating the full upgrade to avoid problems with packages for older versions which wont get replaced by newer versions coming with the new OS.
You are running the RPI4 from SSD with an USB3 adapter. There is a known problem about cases where HA is getting very slow or even unresponsive especially when accessing the logs, Logbook and History.
What helped me to overcome this issue effectively you can find here and here.