This was caused by me setting up a remote access URL via forwarding a router port.
The redirect-page you see when clicking “Add Integration” still shows the old URL under Your instance URL: http://homeassistant.local:8123 so I think that must be the issue.