'Assist' Voice Assistant as a Service-Call

Feature Requests
1

I would like to call a Voice Assistant ‘Assist’ as a service in an automation. My goal is to pass the variable text from a ‘daily briefing script’ to Bard (via the Google Generative AI Conversation integration) so that I can prompt the generative assistant provide variation, styling, or summary to the daily briefing, then pass the resulting modified text to a TTS engine to play throughout the home. This would be similar to what others were attempting to create with ChatGPT or GPT-4. Is this possible?

2

Specifically, I have a custom ‘daily briefing script’ that greets me, tells me the time, date, and weather, and then gives me a list of home assistant reminders for the day: take out the trash, empty the smart vacuum dustbin, the dog has not been fed today, the windows should be opened, the back door is unlocked, water the plants, there’s a meeting scheduled today, etc. This script produces a block of text which can be sent to a TTS and cast to google home speakers. Automation plays this script when arriving home, etc.

The problem is that this text is vary predictable (literally composed from If, then statements) without significant randomization or variation. Generative AI can easily add styling (i.e. re-write as Iron Man’s Jarvis or a sonnet by Shakespeare) to create the desired unfamiliarity of speech that is entertaining, humorous, and captivating. Thus, I find it desirable to pass locally generated text to Google’s PaLM2 or GPT-4.

Seemingly, I would be able to do this with a service call to the voice assistant ‘Assist’ integration returning the output in a sensor or something similar.

I could see other use cases with automations calling for historical rain data, daily news, local events tonight, or displaying descriptions of currently playing media to a e-ink informational display.

3

An example of how you would use Piper in a call service/automation just to help you out.

alias: Piper TTS
description: ""
trigger: []
condition: []
action:
  - service: tts.speak
    data:
      cache: true
      media_player_entity_id: media_player.your_media_player
      message: "Good Morning"
    target:
      entity_id: tts.piper
mode: single
4

Yes, this is exactly what I do (using the google cloud speak tts service). What I would like to do is send the variable {{message}} to Google Generative AI / Bard for post-processing before sending resulting {{message_AI}} to the TTS service.

alias: Daily Briefing
sequence:
  - variables:
      message: >
        {# Daily Breifing #} {%- macro getGreeting() -%}
          {% if now().strftime('%H')|int < 12 %}
            Good morning.
          {% elif now().strftime('%H')|int >= 12 and now().strftime('%H')|int < 17 %}
            Good afternoon.
          {% else %}
            Good evening.
          {% endif %}

          Today is {{ as_timestamp(now()) | timestamp_custom('%B %d') }}.
          It is {{ now().strftime("%I:%M %p") }}.
        {%- endmacro -%}

        {%- macro getReminders() -%}
           ...
        {%- endmacro -%}

        {# a macro that removes all newline characters, empty spaces, and
        returns formatted text #} {%- macro cleanup(data) -%}
          {%- for item in data.split("\n")  if item | trim != "" -%}
            {{ item | trim }} {% endfor -%}
        {%- endmacro -%}

        {# a macro to call all macros :)  #} {%- macro mother_of_all_macros()
        -%}
          {{ getGreeting() }}
          {{ getReminders() }}
        {%- endmacro -%}

          {# Call the macro  #}
        {{- cleanup(mother_of_all_macros()) -}}
  - alias: TTS for speaker voice command
    service: script.google_home_voice
    data:
      use_resume: true
      action:
        - alias: Send TTS message
          service: tts.google_cloud_say
          data:
            message: "{{ message }}"
      volume: "{{ states('input_number.notification_volume')}}"
mode: single
5

You’ll be able to do this in 2023.7 that will be released next week with the conversation.process service. For an example see Home Assistant Blueprint: Conversation agent Agenda Notification · GitHub

1 Like