Google Assistant not listening to other users

He!

My Google Home won’t listen to other users. It wil say that the device is not connected and needs to be added but when i check te Google Assistant app on their device it shows all the switches and smart things, so it’s connected right? Really annoying, because now i’m the only one that can turn on the lights.
We’ve also set voice matching and other settings but doesn’t make a difference.
Does somebody know what to do? Or is somebody having the same issue? Please help me out.

Thanks in advance.

fyi: I’m using Nabu Casa.