I think this is a limitation of the Google API and not Home Assistant.
There are workarounds like this script:
How?