After a presumable error in Homeassistant regarding authentication, I figured out that all my other applications that connect to my Nextcloud CalDAV uses tokens or similar for authentication. After entering the URL I get to authenticate the application using the nextcloud website, instead of passing username/password to the CalDAV server.
Now, in Nextcloud you can use app-passwords, and create a secondary password for your account that single apps can use. But this goes against the use of 2FA in my opinion, and I would like it if you can use the same authentication in the CalDAV Homeassistant addon as you can in (e.g.) DavX⁵