Send picture and text to display on Google Nest Hub while sending TTS

Background

When sending a TTS, you can only define the message, but if you have a device with a screen, it would be nice if it could display additional information while the text is being played. Especially if there are other sounds, like screaming children, loud espresso machines or drilling neighbours.

Requirements

  • A dummy media player to send the TTS to. As of 2022.4 this media_player needs to work with the TTS service, because otherwise it will throw an error and the script will not continue. I’ve installed the VLC add-on on my system, and use media_player.vlc_telnet created by the VLC Telnet integration for the TTS.

Most recent changes

Version 2.0.0/2.0.1 - 8 April 2022

:star2: Improvements

  • Really simplified this script, no need for a template sensor anymore.

:bug: Bug fixes

  • (2.0.1) Some fixes

Version 1.1.0 - 1 April 2022

:star2: Improvements

  • Simplified wait_template by removing conversion to timestamps
  • Home Assistant 2022.4 checks if an entity actually works with a service call, and throws an error if not, therefor I had to replace the Dummy Media Player with the VLC Media Player.

Version 1.0.0 - 14 March 2022

:sparkles: New feature

  • Initial post of script on Github, posted on the forum here on 17th of March

Setup

Dummy media player

Install the VLC Add on from the Add on store in the Superviser. HA should automatically find a new integration and after adding that you should have media_player.vlc_telnet
In case you don’t have the Superviser, because you use core or docker you could probably use another VLC instance to do the same.

The script itself

google_home_tts_screen:
  alias: "Google Home - Send TTS with picture and information"
  description: Script to send a TTS with picture and text to a player with screen
  icon: mdi:cast-audio
  mode: parallel
  max: 20
  sequence:
    - alias: "Version number"
      variables:
        version: 2.0.1
    - wait_for_trigger:
        - platform: event
          event_type: call_service
          event_data:
            domain: media_player
            service: play_media
            service_data:
              media_content_type: music
              entity_id: "{{ [ dummy_player ] }}"
    - alias: "Send TTS message with picture"
      service: media_player.play_media
      target: 
        entity_id: "{{ target }}"
      data:
        media_content_id: "{{ wait.trigger.event.data.service_data.media_content_id }}"
        media_content_type: "music"
        extra:
          metadata:
            metadataType: 3
            title: "{{ large_text }}"
            artist: "{{ small_text }}"
            images:
              - url: "{{ picture_url }}"

Place it in your scripts.yaml with a file editor (like Visual Studio Code Add-on or File Editor Add-on), not via the GUI.

How to use the script

Example
Let’s say you want to send a message to your Google Nest Hub in the office, to announce dinner is ready, and want to add some additional text and a picture

The script will then be something like this:

dinner_ready:
  alias: "TTS with picture when dinner is ready"
  icon: mdi:food
  sequence:
    - alias: "Call the script with the additional data as script variables"
      service: script.turn_on
      target:
        entity_id: script.google_home_tts_screen
      data:
        variables:
          dummy_player: media_player.vlc_telnet 
          target: media_player.office_hub 
          large_text: DINNER!
          small_text: Quick, before it gets cold
          picture_url: http://10.0.0.5/local/pictures/food.jpg
    - alias: "Send the TTS service call to the dummy player"
      service: tts.google_cloud_say
      target: 
        entity_id: media_player.vlc_telnet
      data:
        message: Dinner is ready!

Variables in service call for the script:

Variable Required Description
dummy_player yes The dummy media_player set up for this script
target yes The target to which the TTS should be sent
message yes The TTS message
large_text yes The text to be displayed large
small_text yes The text to be displayed small
picture_url yes A full url to a picture (so no HA internal URL)

Other scripts

For other related Google Home scripst, see my Github page

Buy me a coffee

If you like this script, please feel free to buy me a coffee (I might spend it on another beverage though).
In case you decide to do so, thanks a lot!

Buy Me A Coffee

8 Likes

Hello, thank you for the script ! I think there is a typo in script.google_home_resume_tts_screen (should be script.google_home_tts_screen I think).

Moreover, after fixing this, I get this error : Google Home - Send TTS with picture and information: Error executing script. Unexpected error for call_service at pos 3: dictionary update sequence element #0 has length 1; 2 is required

Could you please help me ?

Sorry 'bout that, I fixed it now.

1 Like

WoW it work’s perfectly now, thanks !

Do you think it’s possible to achieve the same thing but with a fullscreen picture + TSS (no text needed) ?

Not that I’m aware of, maybe with another metadata type, but I don’t know what can be used there.

Okay ! Thank you :wink:

Hi @TheFes, I like your solution to play the TTS with an image and it certainly looks much better than a black screen. I just missed the option to play an audio jingle before the message, so I added it. So far so good.

But… I’m having an issue with the Google Nest Hub (1st gen) player. Sometimes it plays and displays TTS messages with pictures as it should, but sometimes it doesn’t play anything or just part of a sentence. Unfortunately, it happens randomly :frowning: So playback is not reliable. Also sometimes the beginning of the TTS message is swallowed.

Are you having problems with this player too? Do you have a workaround for this?

As a dummy_player I tried media_player from browser_mod but strangely enough I can use Google Nest Hub itself. It works as well.

What I noticed: the Google Home Nest Hub does not go to idle after playing a media file. It is still in the playing state and my experiments with idle were completely useless.

*EDIT: I completely forgot to mention that I have Google Nest Hub permanently in Photo Frame mode.

I also sometimes have some issues where the picture and text are not sent together with the TTS
If you use the Google Nest Hub itself, both the actual TTS, and the mp3 with picture and text are sent to the same player, that might cause unexpected results.

No clear what you mean with the last part. My player goes to idle after the mp3 (with the TTS message) is played. I also have a 1st gen Nest Hub.

Sometimes after playing a classic full TTS message, the Nest Hub player keeps the status playing and doesn’t automatically go to idle. This happens randomly too, and if you wait for termination with: wait_template: "{{ is_state(media_player, 'idle') }}" it can take forever. Whatever, I can live with that. That was just to explain.

But about the previous post… Here is your modified script with jingle that I mentioned before:

service call:

service: script.tts_announcement_with_jingle_and_cast_image
data:
  message: "Hello world!"
  cast_title: "SunseSome title"
  cast_message: "Some text"
  cast_image: "/local/images/announcement/logo.png"
  jingle_file: "/local/audio/jingle.mp3"

script:

# ==============================================================================
# A script for casting TTS message with audio jingle and  Image
# based on:
# https://github.com/TheFes/HA-configuration/tree/main/include/script/00_general/google_cast
# ==============================================================================
# Parameters:
#   - message: A tts message or template for converting text to TTS
#   - voice: integrated TTS service (e.g: tts.google_cloud_say)
#   - targed: mmedia player to play the TTS with display for casting
#   - jingle: play the sound before the message (defaut = true)
#   - jingle_file: path to custom audio file
#      (defaut path = '/local/audio/announcement/jingle.mp3')
#   - cast_title: A large text for casting
#   - cast_message: A small text for casting
#   - cast_image: URL address to image for casting
#
# Example:
#   - service: script.tts_announcement_with_jingle_and_cast_image
#     data:
#       message: "Hello world!"
#       cast_title: "Big Title"
#       cast_message: "Some text"
#       jingle_file: "/local/audio/crystal-logo.mp3"
# ==============================================================================
tts_announcement_with_jingle_and_cast_image:
  alias: TTS announcement with jingle and cast image
  icon: mdi:bullhorn
  fields:
    message:
      description: A tts message
      example: "Hello world!"
    voice:
      description: Integrated TTS service (voice)
      example: "tts.vlasta"
    target:
      description: Target entity (media player)
      example: media_player.google_nest_hub
    jingle:
      description: Play audio jingle (true/false)
      example: false      
    jingle_file:
      description: An audio jingle file (path to audio file)
      example: "/local/audio/announcement/jingle.mp3"
    cast_title:
      description: A title for casting
      example: "TTS"        
    cast_message:
      description: A message for casting
      example: "Hello world!"         
    cast_image:
      description: An image file (path to image file)
      example: "/local/images/announcement/default.png"
  variables: &variables
    tts_message: "{{ message | default('Hello World!') }}"
    tts_service: "{{ voice | default('tts.vlasta') }}"
    media_player: "{{ target | default('media_player.google_nest_hub') }}"
    jingle_first: "{{ jingle | default(true) }}"
    jingle_file: "{{ jingle_file | default('/local/audio/announcement/jingle.mp3') }}"
    cast_title: "{{ cast_title | default('TTS') }}"
    cast_message: "{{ cast_message | default('Hello World!') }}"
    internal_url: !secret internal_url
    image_url: >-
      {%- if cast_image is not defined -%}
        {{ internal_url ~ '/local/images/announcement/default.png' }}
      {%-else-%}
        {{ internal_url ~ cast_image }}
      {%-endif %}
  sequence:
    - alias: "Call the script with the additional data as script variables"
      service: script.turn_on
      target:
        entity_id: script.tts_announcement_cast
      data:
        variables: *variables
    - alias: "Send the TTS service"
      service: "{{ tts_service }}"
      target: 
        entity_id: "{{ media_player }}"
      data:
        message: "{{ tts_message }}"
    # - wait_template: "{{ is_state(media_player, 'idle') }}"     
    - alias: "Stop casting"
      service: media_player.turn_off
      target: 
        entity_id: "{{ media_player }}"
# Called script to send a TTS to a Google Cast device with a screen (e.g. Google Nest Hub)
# based on:
# https://github.com/TheFes/HA-configuration/tree/main/include/script/00_general/google_cast
tts_announcement_cast:
  alias: "Google Nest Hub TTS Screen"
  description: Called script for casting TTS with jingle and image
  mode: parallel
  max: 10
  sequence:
  - wait_for_trigger:
    - platform: event
      event_type: call_service
      event_data:
        domain: media_player
        service: play_media
        service_data:
          media_content_type: music
          entity_id: "{{ [ media_player ] }}"
  - choose:
    - conditions:
      - condition: template
        value_template: "{{ jingle_first }}"
      sequence:
      - delay: 0.5
      - alias: "jingle"
        service: media_player.play_media
        target:
          entity_id: "{{ [ media_player ] }}"
        data:
          media_content_id: "{{ jingle_file }}"
          media_content_type: "music"
          extra: &extra
            metadata:
              metadataType: 3
              title: "{{ cast_title }}"
              artist: "{{ cast_message }}"
              images:
                - url: "{{ image_url }}"
      - delay: 
          seconds: >
            {{ state_attr(media_player,'media_duration') | int(2) }}
  - alias: "Casting TS message with image"
    service: media_player.play_media
    target:
      entity_id: "{{ [ media_player ] }}"
    data:
      media_content_id: "{{ wait.trigger.event.data.service_data.media_content_id }}"
      media_content_type: "music"
      extra: *extra

You might wonder why it doesn’t always play reliably.

I.just.can’t.make.it.work…

Tried both your ideas, but nope…

Have something happen recently with some updates in HA og Nest hub…

I would really love to have thisnon my nest hub (2nd gen) screen instead of :
“Playing default media reciever”-text on my screen when using TSS…

Any ideas for where i might find something similar to this?
I just need a picture displayed instead of “playing”…

Regards from a guy who have used HA for less than two weeks now…

Not sure if there is something similar.
But if you could say what works, and what doesn’t, I might be able to help.

Does this work with a chromecast?

I never tested it, I don’t have any active Chromecast in my home

Does this script still work?
I tried it on a google nest hub 2 and it seems that the picture and text appears less then a second on the screen. and i get a black screen with the cast icon on it…

Did you only use a very short message, like only “test”?
The picture and text are only shown while the TTS message is being played

Hi,

The message was 'Your coffee is ready"
Can we extend the time it is displayed?

Regards,

Bart

It is metadata for the TTS itself, so not really. Depending on you TTS integration you might be able to include a silent mp3 after your actual message

I wanted to chime in on this since I got something similar working for my Google Nest Hub devices.

Here’s the basics:

  • Trigger an automation when a person leaves a zone
  • Condition the automation to run only on school days
  • Set the media player volume, in preparation for the TTS message.
  • Display a quirky GIF on the Google Nest Hub screen to catch people’s attention
  • Wait for 2 seconds so the GIF has time to play
  • Speak the TTS message
  • Wait for the Google Nest Hub to idle, this allows time for the message to be fully spoken and not get cut off
  • turn the media player off on the Google Nest Hub.

Note, playing the TTS message removes the GIF from the screen, and I’m left with “Playing default media” on the Nest Hub screen. I haven’t found a way to “play” a GIF and also “play” a TTS message at the same time.

Here’s my automation code:

- id: "20230210-33"
  alias: Dad Leaving Kids School
  description: "Dad returning from picking up the kids from school"
  trigger:
    - platform: zone
      entity_id: person.dad
      zone: zone.kids_school
      event: leave
  condition:
    - condition: time
      weekday:
        - mon
        - tue
        - wed
        - thu
        - fri
      after: "07:00:00"
      before: "17:00:00"
  action:
    - service: media_player.volume_set
      data:
        volume_level: 0.8
      target:
        entity_id: media_player.kitchen_display
    - service: media_player.play_media
      data:
        media_content_type: image/jpeg
        media_content_id: https://i.pinimg.com/originals/1b/e8/41/1be84116e72d71bd6785c7050fefd2e3.gif
      target:
        device_id: ab2f5dc8bb8200gh388ecf00ec7eaa7a
      alias: Display GIF
    - delay:
        hours: 0
        minutes: 0
        seconds: 2
        milliseconds: 0
    - service: tts.google_translate_say
      data:
        entity_id: media_player.kitchen_display
        message: Excuse me, the kids are on their way home from school. They should be here in about {{state_attr('sensor.travel_kids_school_to_home', 'duration')|int}} minutes.
    - wait_template: "{{ is_state('media_player.kitchen_display','idle') }}"
      continue_on_timeout: true
    - service: media_player.turn_off
      data: {}
      target:
        device_id: ab2f5dc8bb8200388ecf00ec7eaa7a
    - service: tts.clear_cache
      data: {}
  mode: single

Hope this helps someone out there, or give folks some ideas.

Cheers.

2 Likes

Hello,
I’m not able to find media_player.vlc_telnet
I have installed the VLC add but the only entity I could found is VLC update