As of Supervisor 2022.8.5 it seems we can now disable auto updates.
I went to Developer Tools > States and selected the update.home_assistant_supervisor_update entity, and changes the auto_update attribute to false.
Hopefully this works, and I can update manually once I confirm the update won’t have any major conflicts with my addons/integrations.