Issues converting "tts.google_translate_say" to "tts.speak"

Hi,

So I have an automation that will generate a sarcastic comment when the washing machine has finished, and broadcast it over a Nest mini.

This would previously use the “google_generative_ai_conversation.generate_content” action, but given that will be deprecated, I decided to convert the automation to use “ai_task.generate_data” instead. All is good so far.

However when it comes to broadcasting the response on the Nest Mini. It seems to be a bit hit and miss. Sometimes it will work, most of the time it does nothing. And when it does work, sometimes it broadcasts the actual message, but other times it broadcasts the entire response received from the “ai_task.generate_data” service. For example, it would broadcast the entire response below, when all I want it to do is broadcast just the “message:”

conversation_id: 01KA4DXMX487HRD4E5KNA4SGW8
data: |-
  No problem, here's a sarcastic broadcast for your washing machine:

  ```yaml
  service: tts.cloud_say
  data:
    entity_id: media_player.all # Or specific media players, e.g., media_player.google_home
    message: "Breaking news! The washing machine has, against all odds, completed its cycle. Your immediate attention is now required for the critical task of... unpacking it. Don't all rush at once."

Below is a cutdown version of the automation. I'm sure the issue is something to do with the parsing of the response variable, but I can't work it out.

alias: AI_gemini_test
description: “”
triggers:
conditions:
actions:

  • action: ai_task.generate_data
    metadata: {}
    data:
    task_name: test
    instructions: >-
    Broadcast one brief sarcastic message that the washing machine has
    finished and someone needs to unpack it.
    entity_id: ai_task.google_ai_task
    response_variable: generated_content
    enabled: true
  • action: tts.speak
    metadata: {}
    data:
    cache: true
    media_player_entity_id: media_player.study_mini
    message: |
    {{generated_content[‘data’]}}
    target:
    device_id: 7cb73dadcddc981c7fb89a848ac397fb
    mode: single

Does anyone have any idea what I'm doing wrong? Any help would be greatly appreciated. Thanks.

The action is specified incorrectly.
The code should look something like this

action: tts.speak
target:
  entity_id: tts.google_ai_tts
data:
  cache: false
  media_player_entity_id: media_player.study_mini
  options:
    voice: kore
  message: "{{ generated_content.data }}"

Hi mchk. Thanks for responding. I made the change you suggested but it still doesn’t work consistently. If I run the actions then look at the trace and the step details, they differ quite significantly. When it works correctly this is the response I receive:

Response 1

params:
  domain: tts
  service: speak
  service_data:
    cache: false
    media_player_entity_id: media_player.study_mini
    options:
      voice: kore
    message: >-
      Okay, here's a brief sarcastic message you can use for your broadcast:


      "Attention, valued household members! The washing machine has, against all
      odds, completed its heroic task. Someone is undoubtedly *thrilled* to
      empty it right now."
    entity_id:
      - tts.google_ai_tts
  target:
    entity_id:
      - tts.google_ai_tts
running_script: false

However if I run the same actions again these are some examples of the different “message:” I receive (I have only included the message section as all the other sections do not change). I have also been very careful to post the message responses “exactly” as received, including all the formatting/commenting codes.

Response 2

    message: >-
      Okay, here's a playful sarcastic message you can broadcast.


      You can use the `tts.speak` service for this. Replace
      `media_player.your_speaker_name` with the actual entity ID of your smart
      speaker or media player where you want the message to be heard (or
      `media_player.all` to broadcast to all compatible speakers).


      ```yaml

      service: tts.speak

      data:
        media_player_entity_id: media_player.your_speaker_name # Or media_player.all
        message: "Congratulations! The washing machine has, against all odds, successfully completed its mission. Now, if only someone with opposable thumbs and a sense of responsibility could perform the arduous task of unloading it."
      ```

Response 3

    message: |-
      ```yaml
      service: notify.broadcast
      data:
        message: "Breaking news: The washing machine has bravely completed its cycle. Your mission, should you choose to accept it, is to unload it."
      ```

Response 4

      Here's a brief, sarcastic message you can broadcast using your Home
      Assistant Text-to-Speech (TTS) system:


      ```yaml

      service: tts.say

      data:
        entity_id: media_player.your_speaker_name # Or group.all_media_players, media_player.kitchen_display, etc.
        message: "Well, well, well, if it isn't the washing machine, announcing its glorious completion. Someone might want to consider retrieving the clean, fresh laundry before it stages a sit-in."
      ```


      **Before you run this:**


      1.  **Replace `media_player.your_speaker_name`** with the actual entity ID
      of the speaker or media player you want the message to broadcast from. If
      you want it to play on all your speakers, you might have a group set up
      (e.g., `group.all_media_players`) or list multiple individual speakers.

      2.  You can adjust the message to be even more tailored to your
      household's sense of humor

Response 5

    message: >-
      To broadcast a brief, sarcastic message about the washing machine
      finishing, you can use the `tts.speak` service in Home Assistant. This
      will play the message on one or more of your connected media players (like
      Google Homes, Alexas, Sonos, etc.).


      Here's an example service call you can use:


      ```yaml

      service: tts.speak

      data:
        entity_id: all # Or specify a specific media player, e.g., media_player.google_home_mini
        message: "Attention, household! The washing machine has, miraculously, finished its cycle. Your sparkling clean clothes are now eagerly awaiting their rescue from their watery confinement."
      ```


      **How to use this:**


      1.  Go to **Developer Tools** in your Home Assistant interface.

      2.  Navigate to the **Services** tab.

      3.  In the "Service" dropdown, type or select `tts.speak`.

      4.  Copy and paste the YAML code above into the "YAML mode" section.

      5.  **Optional:** Change `entity_id: all` to a specific media player's
      entity ID (e.g., `media_player.kitchen_speaker`) if you only want it to
      play in one location.

      6.  Click **Call Service**.


      The message will then be spoken aloud on your chosen media player(s).

There were a few more different responses I received, but I was conscious this post was getting too large so I omitted them.

The formatting and comments in the message responses varies quite a lot, which explains why sometimes the message responses broadcast to the Nest mini are so different.

Any idea what I am doing wrong? As always, any comments/suggestions would be greatly appreciated.

Change the prompt. There should be a clear instruction to create text, not to perform broadcasting.

And for similar tasks, I recommend using conversation.process instead of ai_task.generate_data.

Agreed it doesn’t have a good string sense of when to use what.

Your system understands the intents just fine. But it doesn’t look like you’ve given it a good ruleset as to when to use what.

Something like:

  • When asked to send messages to voice stations default to using the TTS speak tool for actions you don’t need answers for and broadcast ONLY for urgent messages that need to go to every station. You may send TTS speak messages without further approval while you should confirm before sending a broadcast.

If you get something reeenvling that in your prompt I’m almost positive it will use tts speak almost every time but still have broadcast for when it’s necessary…

Remember the tools and entity_id are only half the story you MUST tell your llm how you want it to behave as well…

So thanks to your help I am making progress here. Having reworded the instruction I now have the message being generated correctly.

However when using the “tts.speak” action and using “Google AI TTS” as the entity (“entity_id: tts.google_ai_tts”) then no audio is played on the Nest mini.

But if I use “Google Translate en com” as the entity (“entity_id: tts.google_translate_en_com”) audio is played on the Nest mini.

So this doesn’t work:

action: tts.speak
metadata: {}
data:
  cache: true
  media_player_entity_id: media_player.study_mini
  message: '{{ generated_content.data }}'
  options:
    voice: kore
target:
  entity_id: tts.google_ai_tts

But this does:

action: tts.speak
metadata: {}
data:
  cache: true
  media_player_entity_id: media_player.study_mini
  message: '{{ generated_content.data }}'
target:
  entity_id: tts.google_translate_en_com

I’ve been playing around with this a lot over the past 24 hours and I’m sure I have had this working using “entity_id: tts.google_ai_tts”.

Is there anything obvious I’m doing wrong? Once again, thanks.

Create automation using the GUI editor. The system allows you to select only existing entities, helping you avoid errors.