Waiting for actions to finish on ESP Satellite

Configuration Voice Assistant
1

I have painstakingly made an automation to play the weather and then the news, dynamically on the calling Satellite. It works for one fatal flaw.

All actions fire at once (with or without the state wait).

Seems awefully limiting for these to be async actions. They all fire and play at once. Net result is theres no weather. The news plays right over the top, even though it’s last to fire. Is there a better way?

I’m on a Pi, using:

https://github.com/OHF-Voice/linux-voice-assistant

alias: Good morning
description: ""
triggers:
  - command:
      - Good morning
      - Morning
    trigger: conversation
conditions: []
actions:
  - variables:
      calling_satellite: >-
        {% set calling_device_id = trigger.device_id %} {% set entities =
        device_entities(calling_device_id) %} {% set satellite = entities |
        select('match', 'assist_satellite\..*') | list %} {% if satellite |
        length > 0 %}
          {{ satellite[0] }}
        {% else %}
          assist_satellite_not_found
        {% endif %}
    alias: Define the Calling Satellite Entity
  - alias: Announce Greeting
    data:
      message: Good morning.
    target:
      entity_id: "{{ calling_satellite }}"
    action: assist_satellite.announce
  - alias: Wait for greeting to finish
    wait_template: "{{ is_state(calling_satellite, 'idle') }}"
    timeout: "00:01:00"
  - alias: Announce Today's Weather
    data:
      message: >
        Today's weather will be: {{
        states('sensor.metservice_weather_today_description') or 'Weather
        information not available' }}
    target:
      entity_id: "{{ calling_satellite }}"
    action: assist_satellite.announce
  - alias: Wait for weather to finish
    wait_template: "{{ is_state(calling_satellite, 'idle') }}"
    timeout: "00:01:00"
  - alias: Pre-Announce the News
    data:
      message: Here's the latest news bulletin.
    target:
      entity_id: "{{ calling_satellite }}"
    action: assist_satellite.announce
  - alias: Get RNZ URL
    response_variable: mp3_url
    action: shell_command.get_rnz_url
  - alias: Wait for news intro to finish
    wait_template: "{{ is_state(calling_satellite, 'idle') }}"
    timeout: "00:01:00"
    continue_on_timeout: true
  - alias: Play the News
    data:
      media_id:
        media_content_id: "{{ mp3_url.stdout }}"
        media_content_type: audio/mpeg
    target:
      entity_id: "{{ calling_satellite }}"
    action: assist_satellite.announce
mode: single
2

You can use device_id as the target for the announcement action.

As for waiting for the speech to end, see here.

3

Thanks for the heads up on device, will save me expanding to entity.

'media_player.'~device_attr(trigger.device_id, 'name')| lower

Looks to be what I want.

I don’t quite understand your link for the timing/async issue. There appears to be no working solution there for my issue, and the suggestion is to do what I’m already doing - waiting for the entity to become idle, which doesn’t work.

4

In my original example I’m using a wait template - they use a wait for trigger. I’m trying that now and it skipped the weather and played the “Here’s the latest news” announcement, then went silent. Trace shows news step was never reached. Probably because device didn’t become idle.

5

I don’t think the Voice PE not being idle is the issue. I had put in a 2 secs delay between the announcements and it still did not work for me.

6

Try using the media player status in the template.
My test script executed a sequence of three announcements (synthesized Piper on the fly) without any problems.

7

@mchk

Are you please able to show yours?

alias: Wait for greeting to finish
wait_template: '{{ is_state(''media_player.pi3_media_player'', ''idle'') }}'
timeout: '00:01:00'

As a test gets stuck on this. Then times out after 60 secs.

Checking the state when some TTS is playing via announce shows it in idle throughout the entire announcement.

image
image1244×252 11 KB

8

OK looks like the TTS action (which requires an entity - but at least I know how to do that)… does change media player state.

image
image1232×256 12.1 KB

9

Nope - it doesn’t reliably go from playing to idle after TTS. Back to the drawing boards. i actually think that’s a Satellite issue and might explain why it’s getting “stuck” a lot in other automations.

I’m also getting TTS being skipped and going straight to the weather.

10

In the template, I use the satellite’s own media player.
In the script, I explicitly specify the device_id {{ is_state('media_player.esp32va01', 'idle') }}, while in the automation, I use the expression {{ is_state('media_player.'~device_attr(trigger.device_id, 'name') | lower, 'idle') }}. The device name must match the ID for the expression to work.
If you are using audio response transmission to a third-party device, I cannot assist with that, as I do not use such a configuration.

11

Thanks I’ll play some more and some good template formats to experiment with. No luck yet.

It’s more complex than it needs to be in my opinion. Different template formats for very similar use-cases that don’t seem to call for it. Different Satellites behaving differently (Wyoming Vs ESP Vs Linux_ESP) and perhaps the most challenging thing for me is a litany of fail-modes…

  • I have ghost TTS and Actions playing and entire Automations (seemingly). While I have deleted TTS cache, and even have a TTS Cache-delete action at the start of the reference automation… I get old prompts playing, I get new prompts that don’t play.
  • My VA tries to answer instead of the automation. Presumably a Whisper STT miss-rec. In this case you have to cancel the current conversation, as repeating yourself in this state won’t trigger an automation intent.
  • I have actions firing in seemingly non-sequential order.
  • ESP Assistant sits in responding mode for minutes on end, with subsequent Auto invocations only partially working, the failing in on certain actions. VA answers general questions in this hung state (with TTS playing on the Satellite), but automations with TTS won’t run until it clears.
  • Satellite media player state remains “playing” half the time too, long after any TTS has completed.
  • Satellite audio device seems to get locked up on occasion, a consequence of useless Linux Audio.

I’m stripping everything back to the most simple single Satellite setup and going to try to solve this TTS / wait for action to complete sequence today and them I’m going to build it up from there. But there’s definitely some bugs in here.

12

OK - some learnings:

{{states("assist_satellite.pi3_assist_satellite")}} seems to cycle through…

idle
listening
processing
responding

With two exceptions being:

  • Where VA expects a followup input prompt where it goes back to listening. E.g. no intent hit for automation, so it tries to clarify (expected).
  • Randomly sits in processing after TTS action in automation is played back (inexplicable). This; I think, explains a lot of the inconsistent behavior waiting for a TTS prompt to “complete”, to move on to the next action.

Media Player state is even more unreliable.

image
image2102×570 42.2 KB

Edit: Further to this I have observed Ollama based VA sometimes tries to handle the query even when it matches the intent for an automation word-for-word. Is there a technique to achieve some level of priority/consistency?

13

I was never able to get this to work