Home Assistant OS wont boot when external drive is connected

A while ago I connected a usb HDD to my Pi4 that runs Hassio. This HDD only has media on it, not Hassio (see below for more details on the setup). I share the HDD with the network using the SambaNas addod. This is working fine as a cheap alternative to a real NAS.

BUT I have one problem: Hassio doesn’t boot when the HDD is connected. The only way to boot into Hassio now is to disconnect the HDD, then boot into Hass.io, then reconnect the HDD, then restart SambaNas. I realy don’t want to have to do all that every time I restart Hassio.

I think the problem might be that the Pi is trying to boot from the HDD instead Hassio boot SSD.
Does anybody have an idea how to fix this? Any help is appreciated.
Please keep in mind that I am very new to Hass, Raspberry Pi’s and Linux in general.

Here’s my setup:
RasberryPi4
Hassio running from usb mounted SSD
Media library on usb mounted HDD
I also have a conbee stick connected.
Because of previous problems I connected ALL usb devices to the USB2 ports using a USB hub. I don’t use the USB3 ports at all.

Choose the boot from sd card file instead

Good luck

Just a note saying that I was looking to boot directly from a USB hdd from my Rapsberry PI 3B+ and I stumbled on this article. I found that at least for Raspberry pi 3B+, you can boot from a USB drive out of the box, no further setup required.