Send picture and text to display on Google Nest Hub while sending TTS

Background

When sending a TTS, you can only define the message, but if you have a device with a screen, it would be nice if it could display additional information while the text is being played. Especially if there are other sounds, like screaming children, loud espresso machines or drilling neighbours.

Requirements

  • A dummy media player to send the TTS to. As of 2022.4 this media_player needs to work with the TTS service, because otherwise it will throw an error and the script will not continue. I’ve installed the VLC add-on on my system, and use media_player.vlc_telnet created by the VLC Telnet integration for the TTS.

Most recent changes

Version 2.0.0/2.0.1 - 8 April 2022

:star2: Improvements

  • Really simplified this script, no need for a template sensor anymore.

:bug: Bug fixes

  • (2.0.1) Some fixes

Version 1.1.0 - 1 April 2022

:star2: Improvements

  • Simplified wait_template by removing conversion to timestamps
  • Home Assistant 2022.4 checks if an entity actually works with a service call, and throws an error if not, therefor I had to replace the Dummy Media Player with the VLC Media Player.

Version 1.0.0 - 14 March 2022

:sparkles: New feature

  • Initial post of script on Github, posted on the forum here on 17th of March

Setup

Dummy media player

Install the VLC Add on from the Add on store in the Superviser. HA should automatically find a new integration and after adding that you should have media_player.vlc_telnet
In case you don’t have the Superviser, because you use core or docker you could probably use another VLC instance to do the same.

The script itself

google_home_tts_screen:
  alias: "Google Home - Send TTS with picture and information"
  description: Script to send a TTS with picture and text to a player with screen
  icon: mdi:cast-audio
  mode: parallel
  max: 20
  sequence:
    - alias: "Version number"
      variables:
        version: 2.0.1
    - wait_for_trigger:
        - platform: event
          event_type: call_service
          event_data:
            domain: media_player
            service: play_media
            service_data:
              media_content_type: music
              entity_id: "{{ [ dummy_player ] }}"
    - alias: "Send TTS message with picture"
      service: media_player.play_media
      target: 
        entity_id: "{{ target }}"
      data:
        media_content_id: "{{ wait.trigger.event.data.service_data.media_content_id }}"
        media_content_type: "music"
        extra:
          metadata:
            metadataType: 3
            title: "{{ large_text }}"
            artist: "{{ small_text }}"
            images:
              - url: "{{ picture_url }}"

Place it in your scripts.yaml with a file editor (like Visual Studio Code Add-on or File Editor Add-on), not via the GUI.

How to use the script

Example
Let’s say you want to send a message to your Google Nest Hub in the office, to announce dinner is ready, and want to add some additional text and a picture

The script will then be something like this:

dinner_ready:
  alias: "TTS with picture when dinner is ready"
  icon: mdi:food
  sequence:
    - alias: "Call the script with the additional data as script variables"
      service: script.turn_on
      target:
        entity_id: script.google_home_tts_screen
      data:
        variables:
          dummy_player: media_player.vlc_telnet 
          target: media_player.office_hub 
          large_text: DINNER!
          small_text: Quick, before it gets cold
          picture_url: http://10.0.0.5/local/pictures/food.jpg
    - alias: "Send the TTS service call to the dummy player"
      service: tts.google_cloud_say
      target: 
        entity_id: media_player.vlc_telnet
      data:
        message: Dinner is ready!

Variables in service call for the script:

Variable Required Description
dummy_player yes The dummy media_player set up for this script
target yes The target to which the TTS should be sent
message yes The TTS message
large_text yes The text to be displayed large
small_text yes The text to be displayed small
picture_url yes A full url to a picture (so no HA internal URL)

Other scripts

For other related Google Home scripst, see my Github page

Buy me a coffee

If you like this script, please feel free to buy me a coffee (I might spend it on another beverage though).
In case you decide to do so, thanks a lot!

Buy Me A Coffee

3 Likes

Hello, thank you for the script ! I think there is a typo in script.google_home_resume_tts_screen (should be script.google_home_tts_screen I think).

Moreover, after fixing this, I get this error : Google Home - Send TTS with picture and information: Error executing script. Unexpected error for call_service at pos 3: dictionary update sequence element #0 has length 1; 2 is required

Could you please help me ?

Sorry 'bout that, I fixed it now.

1 Like

WoW it work’s perfectly now, thanks !

Do you think it’s possible to achieve the same thing but with a fullscreen picture + TSS (no text needed) ?

Not that I’m aware of, maybe with another metadata type, but I don’t know what can be used there.

Okay ! Thank you :wink:

Hi @TheFes, I like your solution to play the TTS with an image and it certainly looks much better than a black screen. I just missed the option to play an audio jingle before the message, so I added it. So far so good.

But… I’m having an issue with the Google Nest Hub (1st gen) player. Sometimes it plays and displays TTS messages with pictures as it should, but sometimes it doesn’t play anything or just part of a sentence. Unfortunately, it happens randomly :frowning: So playback is not reliable. Also sometimes the beginning of the TTS message is swallowed.

Are you having problems with this player too? Do you have a workaround for this?

As a dummy_player I tried media_player from browser_mod but strangely enough I can use Google Nest Hub itself. It works as well.

What I noticed: the Google Home Nest Hub does not go to idle after playing a media file. It is still in the playing state and my experiments with idle were completely useless.

*EDIT: I completely forgot to mention that I have Google Nest Hub permanently in Photo Frame mode.

I also sometimes have some issues where the picture and text are not sent together with the TTS
If you use the Google Nest Hub itself, both the actual TTS, and the mp3 with picture and text are sent to the same player, that might cause unexpected results.

No clear what you mean with the last part. My player goes to idle after the mp3 (with the TTS message) is played. I also have a 1st gen Nest Hub.

Sometimes after playing a classic full TTS message, the Nest Hub player keeps the status playing and doesn’t automatically go to idle. This happens randomly too, and if you wait for termination with: wait_template: "{{ is_state(media_player, 'idle') }}" it can take forever. Whatever, I can live with that. That was just to explain.

But about the previous post… Here is your modified script with jingle that I mentioned before:

service call:

service: script.tts_announcement_with_jingle_and_cast_image
data:
  message: "Hello world!"
  cast_title: "SunseSome title"
  cast_message: "Some text"
  cast_image: "/local/images/announcement/logo.png"
  jingle_file: "/local/audio/jingle.mp3"

script:

# ==============================================================================
# A script for casting TTS message with audio jingle and  Image
# based on:
# https://github.com/TheFes/HA-configuration/tree/main/include/script/00_general/google_cast
# ==============================================================================
# Parameters:
#   - message: A tts message or template for converting text to TTS
#   - voice: integrated TTS service (e.g: tts.google_cloud_say)
#   - targed: mmedia player to play the TTS with display for casting
#   - jingle: play the sound before the message (defaut = true)
#   - jingle_file: path to custom audio file
#      (defaut path = '/local/audio/announcement/jingle.mp3')
#   - cast_title: A large text for casting
#   - cast_message: A small text for casting
#   - cast_image: URL address to image for casting
#
# Example:
#   - service: script.tts_announcement_with_jingle_and_cast_image
#     data:
#       message: "Hello world!"
#       cast_title: "Big Title"
#       cast_message: "Some text"
#       jingle_file: "/local/audio/crystal-logo.mp3"
# ==============================================================================
tts_announcement_with_jingle_and_cast_image:
  alias: TTS announcement with jingle and cast image
  icon: mdi:bullhorn
  fields:
    message:
      description: A tts message
      example: "Hello world!"
    voice:
      description: Integrated TTS service (voice)
      example: "tts.vlasta"
    target:
      description: Target entity (media player)
      example: media_player.google_nest_hub
    jingle:
      description: Play audio jingle (true/false)
      example: false      
    jingle_file:
      description: An audio jingle file (path to audio file)
      example: "/local/audio/announcement/jingle.mp3"
    cast_title:
      description: A title for casting
      example: "TTS"        
    cast_message:
      description: A message for casting
      example: "Hello world!"         
    cast_image:
      description: An image file (path to image file)
      example: "/local/images/announcement/default.png"
  variables: &variables
    tts_message: "{{ message | default('Hello World!') }}"
    tts_service: "{{ voice | default('tts.vlasta') }}"
    media_player: "{{ target | default('media_player.google_nest_hub') }}"
    jingle_first: "{{ jingle | default(true) }}"
    jingle_file: "{{ jingle_file | default('/local/audio/announcement/jingle.mp3') }}"
    cast_title: "{{ cast_title | default('TTS') }}"
    cast_message: "{{ cast_message | default('Hello World!') }}"
    internal_url: !secret internal_url
    image_url: >-
      {%- if cast_image is not defined -%}
        {{ internal_url ~ '/local/images/announcement/default.png' }}
      {%-else-%}
        {{ internal_url ~ cast_image }}
      {%-endif %}
  sequence:
    - alias: "Call the script with the additional data as script variables"
      service: script.turn_on
      target:
        entity_id: script.tts_announcement_cast
      data:
        variables: *variables
    - alias: "Send the TTS service"
      service: "{{ tts_service }}"
      target: 
        entity_id: "{{ media_player }}"
      data:
        message: "{{ tts_message }}"
    # - wait_template: "{{ is_state(media_player, 'idle') }}"     
    - alias: "Stop casting"
      service: media_player.turn_off
      target: 
        entity_id: "{{ media_player }}"
# Called script to send a TTS to a Google Cast device with a screen (e.g. Google Nest Hub)
# based on:
# https://github.com/TheFes/HA-configuration/tree/main/include/script/00_general/google_cast
tts_announcement_cast:
  alias: "Google Nest Hub TTS Screen"
  description: Called script for casting TTS with jingle and image
  mode: parallel
  max: 10
  sequence:
  - wait_for_trigger:
    - platform: event
      event_type: call_service
      event_data:
        domain: media_player
        service: play_media
        service_data:
          media_content_type: music
          entity_id: "{{ [ media_player ] }}"
  - choose:
    - conditions:
      - condition: template
        value_template: "{{ jingle_first }}"
      sequence:
      - delay: 0.5
      - alias: "jingle"
        service: media_player.play_media
        target:
          entity_id: "{{ [ media_player ] }}"
        data:
          media_content_id: "{{ jingle_file }}"
          media_content_type: "music"
          extra: &extra
            metadata:
              metadataType: 3
              title: "{{ cast_title }}"
              artist: "{{ cast_message }}"
              images:
                - url: "{{ image_url }}"
      - delay: 
          seconds: >
            {{ state_attr(media_player,'media_duration') | int(2) }}
  - alias: "Casting TS message with image"
    service: media_player.play_media
    target:
      entity_id: "{{ [ media_player ] }}"
    data:
      media_content_id: "{{ wait.trigger.event.data.service_data.media_content_id }}"
      media_content_type: "music"
      extra: *extra

You might wonder why it doesn’t always play reliably.