Updating from 2023.11.3 to 2024.01.05 fails and SystemMonitor fails to setup

Installation Home Assistant OS
1

Edit: Gave up for a few weeks before deciding to wipe the drive, and start over, then transfer over all automations/script/etc. Spent half the day sorting it, ran into so many issues trying to get everything back as it was that I gave up, reverted to the last backup. But, somehow, somewhere in that and finding tiny things to change and chasing down small insignificant seeming errors in templates and updating every add-on/integration and removing things I’m not using any more… it worked. Finally updated to 2024.2 today. Just going to accept never knowing exactly what was stopping it before.
Except of course now I keep getting errors about running out of memory. and then it crashes

Having serious trouble with updating HA. Started trying to go from 11.3 to 11.4, gave up and have been trying to get it done now.

Using a Raspberry Pi 3
Currently have:

  • Core - 2023.11.3
  • Supervisor - 2023.12.1
  • Operating System - 11.4
  • Frontend - 20231030.2

First attempt at update, it said installing for a long time, rebooted and was still on 2023.11.3 when it decided it was finished. Only relevant seeming error I could see was:

The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly

Found a similar error online which suggested it was corrupted database and that using MariaDB instead may fix it. So set that up and tried to update again.

This time I got an error about System Monitor

Logger: homeassistant.setup
Source: setup.py:185
First occurred: 14:02:51 (1 occurrences)
Last logged: 14:02:51

Setup failed for systemmonitor: No setup or config entry setup function defined.

Only thing I have about system monitor in my yaml config is

  - platform: systemmonitor
    resources:
      - type: disk_use_percent
        arg: /home
      - type: memory_free
      - type: memory_use_percent
      - type: processor_use
      - type: last_boot
      - type: network_in
        arg: eth0
      - type: network_out
        arg: eth0
      - type: ipv4_address
        arg: eth0

I’ve seen that it’s now apparently available to setup in the UI, but when I try to do that I get a very unhelpful box saying

This device cannot be added from the UI

You can add this device by adding it to your ‘configuration.yaml’. See the documentation for more information.

Unfortunately, documentation now just says to set it up via UI.

I just tried to go back to using normal database, rebooted and now ZHA won’t set up.

Logger: homeassistant.config_entries
Source: config_entries.py:402
First occurred: 14:49:13 (1 occurrences)
Last logged: 14:49:13

Error setting up entry ConBee II, s/n: DE2119661 for zha
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 402, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 144, in async_setup_entry
    async with radio_mgr.connect_zigpy_app() as app:
  File "/usr/local/lib/python3.11/contextlib.py", line 204, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/radio_manager.py", line 167, in connect_zigpy_app
    app_config = self.radio_type.controller.SCHEMA(app_config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
    return base_validate(path, iteritems(data), out)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/voluptuous/schema_builder.py", line 433, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['device']['baudrate']

I’ve tried multiple times with both MariaDB and the original database. I’ve tried removing the yaml systemmonitor config. Nothing fixes it.
I basically seem to be bouncing back and forth between these errors. But HA core refuses to update at all and I always get either the systemmonitor or aha failing to setup, or sqlite3 database not shut down cleanly. Or if I’m really lucky, all three.

Does anyone know how to fix this or what might be wrong? Not sure what other logs I should be looking at that might say why HA Core won’t update