I’m not taking credit for it but this works:
Script to send actions to the right Google Home (based on voice commands) - Share your Projects! - Home Assistant Community (home-assistant.io)
What you do is create a routine in GH that runs a script or some how trigger an automation.
The last step in the routine is to start ambient sound on the speaker.
The automation then start with looking at what speaker is playing the ambient sound and that way you know what room it is/what speaker you spoke with.