Google Cast integration and exposing speakers to Google Home

I have moved all of my third-party devices to HA and I now expose them to Google Home from there.

However, I now want to setup the Google Cast integration but I don’t want to end up with two of every speaker in Google Home - one native to the GH app and one exposed via HA.

Is there a way to get around this?

Sorted it with help from a user on Reddit.

As I am running the Google Assistant integration locally, rather than using the paid cloud option, I am able to choose which domains to expose from HA to Google Home, and by doing that I can choose NOT to expose media_player, which works a treat!