Year of the Voice - Chapter 4: Wake words

https://wiki.archlinux.org/title/Advanced_Linux_Sound_Architecture
Or portaudio which is a cross platform audio lib.
Currently you can hack it as the input to the KWS is a chunked raw audio stream that you could pipe into aplay on linux so at that point in the code pass it to stdout or save as a file.

It really needs devel I guess as it doesn’t have a setup app or debug currently as far as I know, as not a user.

Seems extremely complicated…

Until its added likely needs a smattering of python, apols.

Do that as is there is a debug

Does anyone know how to build piper-phonemize from the source, as I get a dependancy error when I try to install piper-tts on my NUC. I have taking a look at similar GitHub issues, but no solutions work (like running with python3.9)

I just read through this entire page over an hour of reading and I still have no idea what KWS or KW is in many of your post. Am I supposed to know what this acronym is? Just would like to be able to understand what I’m reading.

KW = keyword
KWS = keyword spotting

Thanks I wonder how many people actually know that. Far from what I was thinking= Kilowatt-something lol

The whole topic is alphabet soup to me.

5 Likes

100% Far too many TLA’s… Three Letter Acronym’s! :rofl:

1 Like

Awesome!!!
Tried to find answer, but w/o success:
How to wake HA Assist by wake word (not by button or widget) on a phone (Android…) with HA server running on Raspberry?
I selected HA Assist as a Digital Assistant in phone’s Settings/Default apps.
(everything else working - TTS, STT, Openwakeword options (also generated own tflite) present)
In case feature not ready: Is there timeline for this “local” (not satellite) waking?

Same problem here.

  File "/esphome/esphome/git.py", line 95, in clone_or_update
    old_sha = run_git_command(["git", "rev-parse", "HEAD"], str(repo_dir))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/git.py", line 32, in run_git_command
    raise cv.Invalid(err_str)
voluptuous.error.Invalid: fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.

I’ve had that for a couple of days now

Sometimes I get this error:

Traceback (most recent call last):
  File "/usr/local/bin/esphome", line 33, in <module>
    sys.exit(load_entry_point('esphome', 'console_scripts', 'esphome')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 1041, in main
    return run_esphome(sys.argv)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 1028, in run_esphome
    rc = POST_CONFIG_ACTIONS[args.command](args, config)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 458, in command_run
    exit_code = write_cpp(config)
                ^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 193, in write_cpp
    return write_cpp_file()
           ^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 211, in write_cpp_file
    writer.write_cpp(code_s)
  File "/esphome/esphome/writer.py", line 344, in write_cpp
    copy_src_tree()
  File "/esphome/esphome/writer.py", line 297, in copy_src_tree
    copy_files()
  File "/esphome/esphome/components/esp32/__init__.py", line 676, in copy_files
    repo_dir, _ = git.clone_or_update(
                  ^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/git.py", line 70, in clone_or_update
    run_git_command(cmd)
  File "/esphome/esphome/git.py", line 31, in run_git_command
    raise cv.Invalid(lines[-1][len("fatal: ") :])
voluptuous.error.Invalid: fetch-pack: invalid index-pack output

The error keeps changing. Today I get this error:

voluptuous.error.Invalid: Cloning into '/data/idf_components/b0ad5728/components/esp-sr'...
error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly: CANCEL (err 8)
error: 7659 bytes of body are still expected
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output
fatal: clone of 'https://github.com/espressif/esp-sr.git' into submodule path '/data/idf_components/b0ad5728/components/esp-sr' failed
Failed to clone 'components/esp-sr'. Retry scheduled
Cloning into '/data/idf_components/b0ad5728/components/esp-sr'...
error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly: CANCEL (err 8)
error: 7264 bytes of body are still expected
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output
fatal: clone of 'https://github.com/espressif/esp-sr.git' into submodule path '/data/idf_components/b0ad5728/components/esp-sr' failed
Failed to clone 'components/esp-sr' a second time, aborting
2 Likes

Can you use Alexa as microphone input in Home Assistant?

By the way I created native language support in HA - 4 years ago. It is still waiting. I added support for Czech and Vietnamese language and it did not require any wake up word. You could talk naturally. But the PR required some changes in HA class design because the architecture was not well made for that (small refactoring). The comment from PR is here: LG WebOS Play Youtube URL using Launch App Params by cyberluke · Pull Request #13588 · home-assistant/core · GitHub …there was some miscommunication and it was not approved, but I still have it on my hard drive. It does not matter, I just want to say my approach for native language support was better.

Does anyone have an I2S microphone yaml file for esphome that works? I keep getting format errors when using the example file in the esphome library.

Tip if you are a developer then also check out the new ”reSpeaker Lite” development boards and the experimental ESPHome fork that Nabu Casa developers are currently working on (which I understand will pushed upstream to ESPHome mainline if and when it is more mature):

What type of microphone?

I use an INMP441 and here is the config:

microphone:
  - platform: i2s_audio
    id: va_mic
    adc_type: external
    i2s_din_pin: GPIO4
    channel: left
    pdm: false
    i2s_audio_id: i2s_in
    bits_per_sample: 32bit

The speaker is a max9814 and I am just adding it to a ESP 32D. Thanks for reaching out and sharing your config.

? Max9814 is a microphone preamp with AGC its external to the ESP32 as the internal ADC is just pretty bad.
Max9814 with an I2S ADC should work better than most as its an analogue preamp and AGC than just quantising small signals into blocky signals with a straight digital mic.

I use them with usb soundcards on the Pi and they do farfield much better as most mics are setup for a broadcast distance of <0.3m

I’m not sure what the difference is yet completely but I hear you. I’ll read up on these two different types you mentioned asap so I can get this one running. I also have a voice assistant unit I’m working on I can you with that is from an everything smart home year of the voice tutorial. It uses a external mic, amp, and an esp32 s3 dev board because it is fast enough to process on the device or something like that.

Max9814 is pure analogue, but a great chip.
I don’t think HA uses the Esspressif IDF which as far as I know only has the vector optimisation of the esp32-s3.
Without that its not much faster than a standard esp32-s3 with DSP/ML can be 10x faster