Year of the Voice - Chapter 4: Wake words

dodger · November 11, 2023, 11:51am

I am still wondering if one can use separate speaker satellites and dedicated mic satellites?
Anyone knows?

will35 · November 11, 2023, 11:56am

Hello,

You can put your wav files in share directory and modify path in assist-microphone configuration

Merc · November 11, 2023, 12:51pm

Thanks will35,
I wondered about that as well. Will give it a go.

EDIT: Worked like a charm. Thanks

tc23 · November 11, 2023, 11:26pm

I am having an issue trying to install piper. I had previously been able to install piper on an RPi3, but now I get the same issue across three different devices with 3 different os (pi4/buster, pi3/bullseye, nuc/ubuntu). here is the error i am receiving. terminate called after throwing an instance of 'nlohmann::json_abi_v3_11_2::detail::parse_error' i have followed the directions, but still get the same error. i have searched the github, but only saw one thread without a resolution. any help would be appreciated

robot33 · November 17, 2023, 12:08am

Hi,
I have installed everything as suggested and I have managed to get my atom echo to work a few times with the ok nabu wake word.
I have updated everything, including esphome but my echo doesn’t respond any longer to the wake word…
I see these errors in the logs which I believe are the cause of this buggy behavior.
Anyone else experiencing this? Not sure where the right place to post this is. I have tried deleting and reinstalling the openwakeword add on but the problem remains

Logger: homeassistant.components.assist_pipeline.pipeline
Source: components/assist_pipeline/pipeline.py:661
Integration: Assist pipeline ([documentation](https://www.home-assistant.io/integrations/assist_pipeline), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+assist_pipeline%22))
First occurred: November 15, 2023 at 8:05:07 PM (43 occurrences)
Last logged: November 16, 2023 at 11:38:07 PM

Unexpected error during wake-word-detection

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 661, in wake_word_detection result = await self.wake_word_entity.async_process_audio_stream( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/wake_word/__init__.py", line 112, in async_process_audio_stream result = await self._async_process_audio_stream(stream, wake_word_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/wyoming/wake_word.py", line 152, in _async_process_audio_stream chunk_info = audio_task.result() ^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/wyoming/wake_word.py", line 82, in next_chunk async for chunk_bytes in stream: File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 745, in _wake_word_audio_stream async for chunk in audio_stream: File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 1156, in process_enhance_audio async for dirty_samples in audio_stream: File "/usr/src/homeassistant/homeassistant/components/esphome/voice_assistant.py", line 155, in _iterate_packets raise RuntimeError("Not running") RuntimeError: Not running

pcwii · November 17, 2023, 12:45am

Did you try this?

robot33 · November 17, 2023, 1:34am

Thanks. I missed that post…

I’m trying it out now.
Edit: so far so good. Thanks!

Any link to the github issue so I can follow along?

pcwii · November 17, 2023, 2:08am

I got the original code from discord. Can’t locate the original message but if I find it I will link it. A quick look on github and I don’t see it there either. Things may have changed but I still use the code I linked you too and it is working for me. Good luck!

pcwii · November 17, 2023, 2:11am

And just after I posted I found the github link that I originally tried the code from.

github.com/esphome/issues

M5Stack Atom Echo no longer detects voice to start wakeword detection after a while

opened 04:03PM - 26 Oct 23 UTC

rccoleman

### The problem There's been quite a bit of discussion about the M5 Atom Echo… no longer responding to the programmed wakeword after "a while" (like overnight for me), but I don't see anything here and it seems related to the ESPHome build on this device. See the tail end of https://github.com/home-assistant/core/issues/101995 for observations and logs from the 2023.10.3 ESPHome build. In summary, the device seems to get stuck in WAITING_FOR_VAD mode until I either shout directly at it at close range, or sometimes even that's not enough. Without it transitioning from that state to recording voice, it won't detect the wakeword. Toggling the "Use wake word" switch in HA always restores normal behavior for me, and the device is otherwise responsive to commands from HA (like Use listen light) even when it's not detecting voice. ### Which version of ESPHome has the issue? 2023.10.3 ### What type of installation are you using? Docker ### Which version of Home Assistant has the issue? 2023.11.0b0 currently, but also 2023.10.x ### What platform are you using? ESP32 ### Board M5Stack Atom Echo ### Component causing the issue Voice Assistant ### Example YAML snippet _No response_ ### Anything in the logs that might be useful for us? _No response_ ### Additional information I'm using the default, unmodified voice assistant configuration for the Atom Echo, as first flashed via <https://www.home-assistant.io/voice_control/thirteen-usd-voice-remote/> and then adopted by ESPHome and clean/build/install with updated ESPHome releases. There are various reports of updates fixing this issue (including from me), but I think it's just that it's not consistent and takes a while to reproduce. I started absolutely clean with the ESPHome 2023.10.3 container image, cleaned build files, and reinstalled the image on both devices yesterday and one was already exhibiting this behavior this morning.

hanogs · November 24, 2023, 5:21pm

anyone getting climate requests working. Z2M trv. Asking “set lounge radiator to 19” response “sorry I didn’t understand that”
using M5Stack Atom Echo

hanogs · November 26, 2023, 1:02pm

Not a fix but was having the same issue with “ok nabu” switched to “hey rhasspy” and it works better.

Merc · November 26, 2023, 4:19pm

Hi all,
I wanted to try the porcupine1 wake word engine and also generated a custom wake word on picovoice.
It did a good job and I downloaded the .ppn file.

Is there a way to use this in the porcupine add-on?

Thanks,

Merc

will35 · November 27, 2023, 11:29am

Hello,

No way to use custom wake word with pocupine1

Merc · November 27, 2023, 12:31pm

Thanks will35.

Had not found that one despite googeling quite a bit.

Cheers,
Merc

Doni49 · November 27, 2023, 8:18pm

I have a bunch of unused Android phones and tablets. They all have HA Companion App installed.

It would be fantastic if the HA app supported wake word detection. Is that on the roadmap?

Endlessvoid · November 28, 2023, 4:20am

Its in the works per mikes earlier comment in this thread:

sparkydave · November 29, 2023, 2:11am

Not sure if I should start a new thread but I have an existing ESP32 that was acting as a Bluetooth Proxy and wanted to add voice detection to it so I added the below code. It doesn’t have any errors (comiles fine and doesn’t give serial log errors while running) but I don’t get any wake word action from it so not sure where to go next…

Code ‘borrowed’ from this forum.

ESPhome code

i2s_audio:
  - id: i2s_in
    i2s_lrclk_pin: GPIO19
    i2s_bclk_pin: GPIO18

microphone:
  - platform: i2s_audio
    adc_type: external
    pdm: false
    id: mic_i2s
    i2s_audio_id: i2s_in
    i2s_din_pin: GPIO23
    
voice_assistant:
  id: va
  microphone: mic_i2s
  noise_suppression_level: 2

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(va).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);

Voice Assist settings:

ESPhome logs

[10:15:40][D][voice_assistant:529]: Event Type: 0
[10:15:40][D][voice_assistant:529]: Event Type: 2
[10:15:40][D][voice_assistant:619]: Assist Pipeline ended
[10:15:40][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[10:15:40][D][voice_assistant:428]: Desired state set to IDLE
[10:15:40][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[10:15:40][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[10:15:40][D][voice_assistant:206]: Requesting start…
[10:15:40][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[10:15:40][D][voice_assistant:443]: Client started, streaming microphone
[10:15:40][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[10:15:40][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[10:15:40][D][voice_assistant:529]: Event Type: 1
[10:15:41][D][voice_assistant:532]: Assist Pipeline running
[10:15:41][D][voice_assistant:529]: Event Type: 9
[10:15:46][D][voice_assistant:529]: Event Type: 0
[10:15:46][D][voice_assistant:529]: Event Type: 2
[10:15:46][D][voice_assistant:619]: Assist Pipeline ended
[10:15:46][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[10:15:46][D][voice_assistant:428]: Desired state set to IDLE
[10:15:46][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[10:15:46][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[10:15:46][D][voice_assistant:206]: Requesting start…
[10:15:46][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[10:15:46][D][voice_assistant:443]: Client started, streaming microphone
[10:15:46][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[10:15:46][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[10:15:46][D][voice_assistant:529]: Event Type: 1
[10:15:46][D][voice_assistant:532]: Assist Pipeline running
[10:15:46][D][voice_assistant:529]: Event Type: 9
[10:15:51][D][voice_assistant:529]: Event Type: 0
[10:15:51][D][voice_assistant:529]: Event Type: 2
[10:15:51][D][voice_assistant:619]: Assist Pipeline ended
[10:15:51][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[10:15:51][D][voice_assistant:428]: Desired state set to IDLE
[10:15:51][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[10:15:51][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[10:15:51][D][voice_assistant:206]: Requesting start…
[10:15:51][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[10:15:51][D][voice_assistant:443]: Client started, streaming microphone
[10:15:51][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[10:15:51][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[10:15:51][D][voice_assistant:529]: Event Type: 1
[10:15:51][D][voice_assistant:532]: Assist Pipeline running
[10:15:51][D][voice_assistant:529]: Event Type: 9
[10:15:56][D][voice_assistant:529]: Event Type: 0
[10:15:56][D][voice_assistant:529]: Event Type: 2
[10:15:56][D][voice_assistant:619]: Assist Pipeline ended
[10:15:56][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[10:15:56][D][voice_assistant:428]: Desired state set to IDLE
[10:15:56][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[10:15:56][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[10:15:56][D][voice_assistant:206]: Requesting start…
[10:15:56][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[10:15:56][D][voice_assistant:443]: Client started, streaming microphone
[10:15:56][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[10:15:56][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[10:15:56][D][voice_assistant:529]: Event Type: 1
[10:15:56][D][voice_assistant:532]: Assist Pipeline running
[10:15:56][D][voice_assistant:529]: Event Type: 9
[10:16:01][D][voice_assistant:529]: Event Type: 0
[10:16:01][D][voice_assistant:529]: Event Type: 2
[10:16:01][D][voice_assistant:619]: Assist Pipeline ended
[10:16:01][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[10:16:01][D][voice_assistant:428]: Desired state set to IDLE
[10:16:01][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[10:16:01][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[10:16:01][D][voice_assistant:206]: Requesting start…
[10:16:01][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[10:16:01][D][voice_assistant:443]: Client started, streaming microphone
[10:16:01][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[10:16:01][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[10:16:01][D][voice_assistant:529]: Event Type: 1
[10:16:01][D][voice_assistant:532]: Assist Pipeline running
[10:16:01][D][voice_assistant:529]: Event Type: 9

Any tips of what to test?

stuartiannaylor · November 29, 2023, 2:19am

Currently the code which is on github uses just a single part of the Esspressif ADF and has the VAD turned to max which is the most restrictive, likely due to false detection.
Also there seems to be little data on initial input volume or AGC and likely an option to trigger and capture the pcm so you can have a quick look in something like Audacity would tell much and having a setting for VAD threshold would likely be a good idea.
I guess a search for Event Type: 2 as must be documented somewhere but likely will have to trawl code.

sparkydave · November 29, 2023, 2:21am

I know of this software but how do you suggest using it in this instance?

stuartiannaylor · November 29, 2023, 2:23am

Dunno as its often an ommision but a option to record to file is a god send for debugging and setup to get optimal volumes and testing.