Year of the Voice - Chapter 3: Ready when you are

Blog
1

This year is Home Assistant’s Year of the Voice. It is our goal for 2023 to let users control Home Assistant in their own language. Today we’re presenting our third milestone for the year: Chapter 3.

In Chapter 1, we focused on intents – what the user wants to do. The Home Assistant community has now translated common smart home commands and responses into 50 languages, coming even closer to the 62 languages that Home Assistant supports.

In Chapter 2, we introduced speech-to-text and text-to-speech. This allows users to talk and listen to their smart homes through the browser, ESPHome, and even analog phones.

For Chapter 3, we bring the full power of Assist to the million active Android devices running the Home Assistant Companion app. Got an Android phone, tablet, or watch? Set Assist as your default digital assistant and talk to your Home Assistant with one push!

To watch the video presentation of this blog post, including live demos, check the recording of our live stream.

Native Assist on Android

Starting in Chapter 3, Assist is now natively supported through the Home Assistant Companion App. With this update, the full power of Assist is available to over a million Android devices using Home Assistant. Access the Assist button in the app, and talk to your smart home from your phone:

Assist on Android is built-in to the Android app and uses the speech-to-text from Home Assistant

Have a Home Assistant Cloud subscription? You can now control your smart home via voice from anywhere in the world without giving up your privacy. Multiple Home Assistant servers are supported, so you can just as easily talk to your parent’s house:

Select a voice assistant from any of your configured Home Assistant instances

Default digital assistant

Why open an app if you don’t have to? Set Home Assistant as your default digital assistant app! Now, holding the home or power button will instantly bring up Assist:

Launch Assist from anywhere on Android without having to open the Home Assistant app

It’s even possible to activate Assist directly from the lock screen:

You can find the settings on Android: Settings -> Apps -> Default apps or tap on the tip in the Home Assistant Companion settings.

Native Assist on Wear OS

We didn’t leave out watches! Wear OS devices can now natively use Assist as well:

Text-to-speech responses are spoken directly from the watch, allowing you to listen instead of looking for confirmation.

Like the phones, it is possible to set Home Assistant to be your watch’s default digital assistant. Pressing the crown or holding the button on your watch now brings up Assist!

Native Assist on Wear OS

Sentence triggers

While custom sentences were already part of Assist, they required manually editing YAML. With the new sentence trigger, you can now add custom sentences directly in the UI:

Configure your own sentences to trigger any action in the automation editor

These sentences take priority over the built-in commands, and work anywhere that Assist does. Some basic template syntax is available, such as marking parts of the sentence as optional.

Assist debug tool

The Home Assistant community is hard at work translating common smart home commands. To help contributors, we’ve added an “Assist” developer tool that lets you test out sentences without actually executing the commands:

Test sentence understanding with the Assist developer tool

For each test command or query, you can find out:

  • Which intent would be triggered with its slot values
  • Which entities would be targeted
  • Whether or not each targeted entity matched (queries only)

Piper community voices

Piper is a text-to-speech system created by Nabu Casa, designed to run locally on the Raspberry Pi 4. Since its introduction in Chapter 2, Piper has grown rapidly with adoption in the open source community as well as academia.

The Piper add-on for Home Assistant OS now supports 23 languages and over 70 different voices. This includes voices from 7 brand new voice datasets contributed by the Home Assistant community. These datasets have been collected and donated to the public domain by Nabu Casa for the benefit of text-to-speech research now and in the future.

New ESPHome voice modes

With hardware like the $13 ATOM Echo, it was possible in Chapter 2 to create a push-to-talk voice Assistant using ESPHome.

Using the latest ESPHome, you can push a button to start speaking and let Home Assistant detect the end of the voice command. Continuous mode is also available, allowing multiple conversation turns with Assist similar to an analog phone.

voice_assistant:
  microphone: ...
  speaker: ...
binary_sensor:
  - platform: gpio
    pin: ...
    on_click:
      - if:
          condition: voice_assistant.is_running
          then:
            - voice_assistant.stop:
          else:
            - voice_assistant.start_continuous:

ESPHome Voice Assistant documentation

Silence detection

ESPHome’s new voice modes use Home Assistant to detect the end of each voice command. In your device’s configuration, you can now adjust how much silence is needed with “Finished speaking detection”:

ESPHome Voice Assistant have two config entities to automate it's behavior in Home Assistant

Until next time

That does it for Chapter 3! A big thanks to Joris Pelgröm for his work on the Android app, and to the many Assist contributors. And as always, thank you to Nabu Casa and all of the Home Assistant Cloud subscribers for making this possible.

If you want to help fund our work on voice and further development of Home Assistant, ESPHome, Z-Wave JS and other related projects? Subscribe to Home Assistant Cloud.

Where is wake word?

After three chapters in the Year of Voice, the message from Home Assistant users and creators is loud and clear: we want a wake word for Assist!

In Chapter 4, our focus will be on delivering the foundation for wake word detection. As a part of the Open Home, this foundation will keep in line with our values: privacy, choice, and durability. Stay tuned!

Some links on this page are affiliate links and purchases using these links support the Home Assistant project.

This is a companion discussion topic for the original entry at https://www.home-assistant.io/blog/2023/07/20/year-of-the-voice-chapter-3/
5 Likes
2

Looking forward to trying this?

3

Great works guys! Really excited for Wake Word as well :smiley:

Like the phones, it is possible to set Home Assistant to be your watch’s default digital assistant. Pressing the crown or holding the button on your watch now brings up Assist!

When is this available? I don’t have the option yet to hold button to use assist yet, only Bixby and Google Assistant (on galaxy watch)

4

If your app is up-to-date it should be available. On a Galaxy Watch it is a bit weird, you have to set the hold action to Google Assistant (not Bixby), then when you press and hold you will get a choice between the Google Assistant and Home Assistant.

5

Hey that worked! Thanks dude :grinning_face_with_smiling_eyes:

6

Nice work. Very exciting progress. I only see one default piper voice. Where are the others accessed?

7

This will be available with the Home Assistant 2023.8 release.

2 Likes
8

Something that caught my ear on the YT video was “face detection with ESPhome…” Did I hear that right? How?

9

Hey Guys!
Is there any way to make a “service call for the assist” to start listening?
I want to make an automation which triggered by motion and the action will that the assist will open on the walltablet and start listening.

Thanks for the help :slight_smile:

10

This is possible using the Person Sensor (it’s a hardware sensor)

1 Like
11

That’s not possible out of the box, but could be done with a custom card:

  • Automation fires an event when it wants to open Assist in the frontend
  • Custom card listens for that event and calls action to open Assist
12

Thanks for the great work. Still amazed every day what this plattformen is capable to do and it’s getting better all the time.

Is it possible to activate the assist not through a dashboard button but with a hardware button (zigbee or so)?
I tried to find a service but did not find it.

Thanks.

Merc

13

The post above yours explains exactly your question. :slight_smile:

@balloob Nice chapter you guys opened here, love the constant polishing of all the voice stuff. And I know, I’m in the minority, but for me there are more important things than the “wake word”. Know that from Rhasspy, the false triggering is way to often, especially when watching TV. And that get’s annoying very quickly. So what I want to say is a big thanks, it’s great to be part of this journey! :slight_smile:

5 Likes
14

Will this work with the fdroid version of HA? I’m currently not seeing the option for virtual assistant on my phone. Also running CalyxOS, that may be an issue for it as well

15

I am trying to use the digital assistant with the android compation, but it says that I must have homeassistant in version at least2023.5 (I have 2023.7.1) and have the Assist Pipeline integration enabled (which I do because I have several voice assistants set-up). Is there anyway to make it work?

16

Yes it’ll work, but F-Droid do their own builds which lag behind. Wait until the app is updated to 2023.7.5 or newer, or if you want to use it now use the minimal version we build directly from GitHub.

17

That’s odd. Is this with the watch app or on a phone/tablet/… ? Do other app features such as sensors work correctly?

18

Great progress with this feature.
On my Pixel 5 I set the default digital assistant app to homeassistant but I can only use the keyboard. Speach does not work (no mic button), like it does in the video. What kind of settings did I miss (App version 2023.7.5-full) ? Thanks

19

I’d love to try this but as far as I can tell, German still doesn’t work and the corresponding issue has been ignored since May.

20

Seen this

5 Likes