Store and restore scenes with Spotify playing on Google Nest Speakers

I’ve been trying to create a scene (and restore it later again) that includes a Google Nest Speaker (aka Google Home). Creating the scene seems to be working just fine from the frontend. The scene is created and when I check in the scenes.yaml file it seems to have all the correct information stored, including what app the Google Nest Speaker was using (Spotify) and which content was playing. However restoring doesn’t work as expected. I would expect the speaker to play the song/album that was playing when I created the scene, but it won’t. All the other states like volume, etc are correctly restored, but the speaker won’t play the song.

Also note, that when I ask Google to play some white noise and store that in a scene, that it will correctly restore it and play it when I activate the scene. So far only Spotify does not seem to work correctly.

Has anyone gotten this to work before? What do I need to watch out for to make this work? Any advice on how to continue debugging this? There don’t seem to be any messages in the logs when I create or restore the scene.