Invited HomeKit user cannot get control over HA accessories

Hello all,
I’ve been setting up my HA instance for a while and recently added the HomeKit integration. Setting it up was fine and worked smoothly on my device. The sync across the devices connected to my iCloud account also works reliably. However, inviting my gf to the HomeKit home does not give the expected results.

I attempted to invite my gf using the integrated invite function inside the app. In theory, it seems to work in the sense that the devices show up, but they all have the status “not reachable”. I tried leaving the home and rejoining it, but that did not help either.

Has anyone successfully added two devices using the HomeKit integration?

I appreciate your help!