Can't update core: API responds but frontend is not loaded

Trying to update from 2023.4.2 to 2023.12.3.
It looks like the update is completed successfully but then rolls back to previous version.

I think these are the relevant log entries from Supervisor logs:

23-12-27 10:02:32 INFO (MainThread) [supervisor.homeassistant.core] Home Assistant Core state changed to RUNNING
23-12-27 10:02:32 INFO (MainThread) [supervisor.homeassistant.core] Detect a running Home Assistant instance
23-12-27 10:02:32 INFO (MainThread) [supervisor.homeassistant.core] Successfully started Home Assistant 2023.12.3
23-12-27 10:02:32 INFO (SyncWorker_3) [supervisor.docker.manager] Cleanup images: ['ghcr.io/home-assistant/qemux86-64-homeassistant:2023.4.2']
23-12-27 10:02:35 ERROR (MainThread) [supervisor.homeassistant.core] API responds but frontend is not loaded
23-12-27 10:02:35 CRITICAL (MainThread) [supervisor.homeassistant.core] HomeAssistant update failed -> rollback!

This is a new installation on Proxmox with restored configuration from an installation on old phone.