Next iteration of our Voice Assistant is here - Voice chapter 10

[jackjourneyman (Jack)]

But it looks like Voice is going to stay a black box. We can see all sorts of events and errors in the Assist debug log, but we can’t actually use any of them. It’s a bit of a deal breaker.

[codemunkie]

Great work from the team!

Is there any update on voice matching please? All the new stuff you’re adding is great, but voice matching is a must-have for me living in a family where everyone wants to listen to their own music on their own accounts. I can’t move away from Google until Voice/Assist supports this.

Many thanks

[jianyu-li]

Great updates! Hopefully we can get a timer entity when one is set so we can display it on the dashboard

[A011528]

Yes, thanks! I’ve got 8 models currently in “trial” mode—aiming to narrow it down to one, though I might land on three depending on the context. I’m offloading to a 3090 GPU running Whisper and Piper on Proxmox. So far, I’m leaning towards Qwen, Llama, Phi, and Mistral. I’ll check out Friday’s party—I must’ve missed the invite on that.

[dumbdevice]

@jianyu-li

Great updates! Hopefully we can get a timer entity when one is set so we can display it on the dashboard

Yes, we need the whole Timers integration!

WTH there is no Timers integration!

Timers integration is missing! I wish we could set timers with voice or in dashboard. It could be a dedicated icon in side panel. support of multiple timers set timers sound notifications and output speaker icon indicator if timers is enabled pause and cancel timers from the dashboard when you create a timer via voice assistant, the timer should appear in Dashboard immediately. Example: [timers]

---

and Alarm clock integration too

[ShadowFist]

That’s why it helps to read the docs

The UI will almost certainly never show all the possible options available in yaml, especially when it comes to cards. Might as well go direct to the source to see what you can do

[jackjourneyman (Jack)]

@jianyu-li

Great updates! Hopefully we can get a timer entity when one is set so we can display it on the dashboard

You can do that already with a custom sentence/intent.

Custom sentences and intents in home assistant

[pcwii]

So happy to see voice development continue. Can’t wait to try a few things.
Well done team!

[Hedda]

@jackjourneyman

But it looks like Voice is going to stay a black box. We can see all sorts of events and errors in the Assist debug log, but we can’t actually use any of them.

+1 that assist debug logging would be nice to be able to enable for troubleshooting voice and LLM commands.

[mchk]

ESPhome has functionality to transmit events from any stage of VA to Home Assistant. Just configure it to suit your needs.

I used this to get text from stt. But after adding ‘Ask a question,’ it was no longer necessary.

[jackjourneyman (Jack)]

But as I understand it, that’s thanks to ESPhome, not Home Assistant.

Also - and this may come as a shock - not all of us use ESPhome.

[oliof (Harald)]

That page you linked mentions HA as a prerequisite for ESphomes Voice Assistant stuff. And anyone that uses Voice Assistant Preview Edition, uses ESPHome.

[jackjourneyman (Jack)]

But not everyone uses Voice Assistant Preview Edition and Open Voice is more than VPE.

In fact, Assist predates VPE by quite a long way. It has never allowed users to leverage events and errors - and there are plenty of them:

stage: done
run:
  pipeline: 01h0amejr3nf5059gavrx5jm7g
  language: en
  conversation_id: 01JZ86PBWFHMAPA1Q6W0EVMCQN
  runner_data:
    stt_binary_handler_id: null
    timeout: 300
events:
  - type: run-start
    data:
      pipeline: 01h0amejr3nf5059gavrx5jm7g
      language: en
      conversation_id: 01JZ86PBWFHMAPA1Q6W0EVMCQN
      runner_data:
        stt_binary_handler_id: null
        timeout: 300
    timestamp: "2025-07-03T13:12:54.927548+00:00"
  - type: intent-start
    data:
      engine: conversation.home_assistant
      language: en
      intent_input: undo the trittle.
      conversation_id: 01JZ86PBWFHMAPA1Q6W0EVMCQN
      device_id: null
      prefer_local_intents: false
    timestamp: "2025-07-03T13:12:54.927637+00:00"
  - type: intent-end
    data:
      processed_locally: true
      intent_output:
        response:
          speech:
            plain:
              speech: No area named undo
              extra_data: null
          card: {}
          language: en
          response_type: error
          data:
            code: no_valid_targets
        conversation_id: 01JZ86PBWFHMAPA1Q6W0EVMCQN
        continue_conversation: false
    timestamp: "2025-07-03T13:12:55.155924+00:00"
  - type: run-end
    data: null
    timestamp: "2025-07-03T13:12:55.155987+00:00"
intent:
  engine: conversation.home_assistant
  language: en
  intent_input: undo the trittle.
  conversation_id: 01JZ86PBWFHMAPA1Q6W0EVMCQN
  device_id: null
  prefer_local_intents: false
  done: true
  processed_locally: true
  intent_output:
    response:
      speech:
        plain:
          speech: No area named undo
          extra_data: null
      card: {}
      language: en
      response_type: error
      data:
        code: no_valid_targets
    conversation_id: 01JZ86PBWFHMAPA1Q6W0EVMCQN
    continue_conversation: false

This appears to have been an early design decision.

[stevenandres (Steven)]

When it comes to follow-up on missing parameters, it would be cool if we could set some defaults. For example, ‘Ok Nabu, dim the lights in the living room.’

In a separate place, maybe we could define some rules for how it would determine the amount. I’m not a programmer, but perhaps something like this:

defaults:

lights:
  - light.living_room_pot_lights
    - brightness:
      - if (input_boolean.party_mode == on && command == dim) # allow for 'dim' vs 'brighten' type commands
          set to 20%
      - elif (input_select.home_mode == 'day' && light.living_room_pot_light.brightness > 75%)
          change by 25% # allow for 'change by' commands (if dim was requested, it will go down by 25%, if brighten was requested, up by 25%
      - elif (input_select.home_mode == 'day' && light.living_room_pot_lights > 40%)
          set to 35% # allow for 'set to' commands
      - elif ...
      - else: ask # allow for ask command, that will ask for confirmation everytime.

timers:
  - timer.washing_machine    # if a timer is set with the tag washing machine
      0:45:00                # eg. Ok Nabu, set my washing machine timer
  - timer.dryer
      1:00:00
  - timer.kitchen.device # add some defaults on a per device level?
      ask                # eg. When using the kitchen speaker, always ask

[jackjourneyman (Jack)]

Again, this sort of thing can be done with intent scripts.

[NathanCu (Nathan Curtis)]

Agree at least with feentrant scripts and event triggers you can In theory both have a script call out and ask for clarification if said info isn’t in (condition X) and we can now ask a question and receive answers with clarified slots. You could then restart the script with the correct data which would skip the question etc…

Workflow Ala HA voice.