M5Stack Atom Echo - On-device wake word with MicroWakeWord

asherw83 · July 23, 2024, 12:00pm

On device wake word is working on the M5Stack Atom Echo! I’m pumped.

I started to throw it all together when I came across a PR from someone who did all the hard work. PR Here: Github, ESPHome config file here: m5stack-atom-echo.yaml

My config here:

substitutions:
  name: voiceassistant
  friendly_name: VoiceAssistant
  micro_wake_word_model: alexa

esphome:
  name: ${name}
  friendly_name: ${friendly_name}
  name_add_mac_suffix: true
  min_version: 2024.7.1
  project:
    name: m5stack.atom-echo-wake-word-voice-assistant
    version: "24.7.4.1"

esp32:
  board: m5stack-atom
  framework:
    type: esp-idf

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# Allow Over-The-Air updates
ota:
  - platform: esphome
    password: "xxxxxxxxxxxxxxxxxxxxxxxxx"

# Allow provisioning Wi-Fi via serial
improv_serial:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  ap:

captive_portal:

button:
  - platform: factory_reset
    id: factory_reset_btn
    name: Factory reset

i2s_audio:
  - id: i2s_audio_bus
    i2s_lrclk_pin: GPIO33
    i2s_bclk_pin: GPIO19

microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO23
    adc_type: external
    pdm: true

speaker:
  - platform: i2s_audio
    id: echo_speaker
    i2s_dout_pin: GPIO22 
    dac_type: external
    mode: mono

voice_assistant:
  id: va
  microphone: echo_microphone
  speaker: echo_speaker
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 1.0
  vad_threshold: 3
  on_listening:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Slow Pulse"
  on_stt_vad_end:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Fast Pulse"
  on_tts_start:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: none
  on_end:
    - delay: 100ms
    - voice_assistant.stop:
    - wait_until:
        not:
          voice_assistant.is_running:
    - if:
        condition:
          lambda: return id(wake_word_engine_location).state == "On device";
        then:
          - micro_wake_word.start:
          - script.execute: reset_led
        else:
          - voice_assistant.start_continuous:
          - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led
        red: 100%
        green: 0%
        blue: 0%
        brightness: 100%
        effect: none
    - delay: 2s
    - script.execute: reset_led
  on_client_connected:
    - delay: 5s
    - if:
        condition:
          lambda: return id(wake_word_engine_location).state == "On device";
        then:
          - micro_wake_word.start:
        else:
          - voice_assistant.start_continuous:
          - script.execute: reset_led
  on_client_disconnected:
    - voice_assistant.stop:
    - micro_wake_word.stop:

binary_sensor:
  # button does the following:
  # Restart either microwakeword or voice assistant continuous
  - platform: gpio
    pin:
      number: GPIO39
      inverted: true
    name: Button
    disabled_by_default: true
    entity_category: diagnostic
    id: echo_button
    on_multi_click:
      - timing:
          - ON for at least 50ms
          - OFF for at least 50ms
        then:
          - if:
              condition:
                  lambda: return id(wake_word_engine_location).state == "On device";
              then:
                  - voice_assistant.stop:
                  - micro_wake_word.stop:
                  - delay: 1s
                  - script.execute: reset_led
                  - script.wait: reset_led
                  - micro_wake_word.start:
              else:
                  - if:
                      condition: voice_assistant.is_running
                      then:
                      - voice_assistant.stop:
                      - script.execute: reset_led
                  - voice_assistant.start_continuous: 
      - timing:
          - ON for at least 10s
        then:
          - button.press: factory_reset_btn

light:
  - platform: esp32_rmt_led_strip
    id: led
    name: None
    disabled_by_default: true
    entity_category: config
    pin: GPIO27
    default_transition_length: 0s
    chipset: SK6812
    num_leds: 1
    rgb_order: grb
    rmt_channel: 0
    effects:
      - pulse:
          name: "Slow Pulse"
          transition_length: 250ms
          update_interval: 250ms
          min_brightness: 50%
          max_brightness: 100%
      - pulse:
          name: "Fast Pulse"
          transition_length: 100ms
          update_interval: 100ms
          min_brightness: 50%
          max_brightness: 100%

script:
  - id: reset_led
    then:
      - if:
          condition:
            - lambda: return id(wake_word_engine_location).state == "On device";
            - switch.is_on: use_listen_light
          then:
            - light.turn_on:
                id: led
                red: 100%
                green: 89%
                blue: 71%
                brightness: 60%
                effect: none
          else:
           - if:
              condition:
                - lambda: return id(wake_word_engine_location).state != "On device";
                - switch.is_on: use_listen_light
              then:
                - light.turn_on:
                    id: led
                    red: 0%
                    green: 100%
                    blue: 100%
                    brightness: 60%
                    effect: none
              else:
               - light.turn_off: led

switch:
  - platform: template
    name: Use listen light
    id: use_listen_light
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - script.execute: reset_led
    on_turn_off:
      - script.execute: reset_led

select:
  - id: wake_word_engine_location
    platform: template
    entity_category: config
    name: Wake word engine location
    optimistic: true
    restore_value: true
    options:
      - In Home Assistant
      - On device
    initial_option: On device
    on_value:
      - if:
          condition:
            lambda: return x == "In Home Assistant";
          then:
            - micro_wake_word.stop
            - delay: 500ms
            - lambda: id(va).set_use_wake_word(true);
            - voice_assistant.start_continuous:
      - if:
          condition:
            lambda: return x == "On device";
          then:
            - lambda: id(va).set_use_wake_word(false);
            - voice_assistant.stop
            - delay: 500ms
            - micro_wake_word.start

external_components:
  - source: github://pr#5230
    components:
      - esp_adf
    refresh: 0s
  - source:
      type: git
      url: https://github.com/kahrendt/esphome
      ref:  mww-v2-external-library
    refresh: 0s
    components: [ micro_wake_word ] 

esp_adf:


# https://esphome.io/components/micro_wake_word.html
micro_wake_word:
  vad:
    model: github://esphome/micro-wake-word-models/models/v2/vad.json
  models:
    - model: github://esphome/micro-wake-word-models/models/v2/${micro_wake_word_model}.json
  on_wake_word_detected:
    - voice_assistant.start: 
        wake_word: !lambda return wake_word;

templeton_nash · July 23, 2024, 6:25pm

Hi @asherw83

This works perfectly. Thanks so much!

If I want to use ‘hey jarvis’ as the on device wake word, do you know how I should change this?

substitutions:
  name: bedroom-2-assist-device
  friendly_name: Bedroom 2 Assist Device
  micro_wake_word_model: hey jarvis ???

zyberzero · July 23, 2024, 7:14pm

The code doesn’t use the substition values for models.

You need to change the models in the block:


# https://esphome.io/components/micro_wake_word.html
micro_wake_word:
  vad:
    model: github://esphome/micro-wake-word-models/models/v2/vad.json
  models:
    - model: github://esphome/micro-wake-word-models/models/v2/alexa.json
  on_wake_word_detected:
    - voice_assistant.start: 
        wake_word: !lambda return wake_word;

from alexa.json to hey_jarvis.json and reinstall

zyberzero · July 23, 2024, 7:17pm

Thanks for writing this down! It was very helpful

templeton_nash · July 23, 2024, 7:43pm

Thanks @asherw83 !

sejnub · July 24, 2024, 10:32am

Great, it works.
And there is a lot of space for improvement, acoustically and logically. I see this state as a proof of concept; the everyday usefulness is still there to come.

templeton_nash · July 24, 2024, 10:33am

I think we should wait nd see what hardware Nabu Casa have up their sleeve.

asherw83 · July 25, 2024, 2:24am

Hi @templeton_nash, because the substitute block doesn’t yet support arrays/lists, you’ll have to manually add the ‘hey jarvis’ model.

So if you scroll down to the bottom of the yaml file you’ll find the micro_wake_word: component.

micro_wake_word:
  vad:
    model: github://esphome/micro-wake-word-models/models/v2/vad.json
  models:
    - model: github://esphome/micro-wake-word-models/models/v2/alexa.json
  on_wake_word_detected:
    - voice_assistant.start: 
        wake_word: !lambda return wake_word;

To add another supported wake word model (All current supported wake words can be found here: github/esphome/micro-wake-word-models)

After you add ‘hey jarvis’ your config should look like this:

micro_wake_word:
  vad:
    model: github://esphome/micro-wake-word-models/models/v2/vad.json
  models:
    - model: github://esphome/micro-wake-word-models/models/v2/alexa.json
    - model: github://esphome/micro-wake-word-models/models/v2/hey_jarvis.json
  on_wake_word_detected:
    - voice_assistant.start: 
        wake_word: !lambda return wake_word;

You’ll notice the url looks different as it is the ‘short hand’ version (as mentioned in the esphome doco) which gets translated to the full github url upon compilation.

Merc · July 31, 2024, 3:23pm

HI all,
so with the last GITHUB update of the Atom Echo firmware I managed to get micro wake work running.
Before that I had tweaked my Atom Echo firmware in a way so that it plays a little confirmation sound when the wakeword is recognized. I did not figure that one out on my own but with some code I found on the forum. I can’t remember who was the original author but I am very grateful for the code and definitely do not want to sell this as my product.

I will paste the code and the little click sound I am using below, if people want to use it.
Now I would like to marry the two, i.e. have the confirmation sound played with the firmware that uses on device wakeword but this does not work yet. Maybe someone with more coding experience will quickly figure out what I am doing wrong.

I would appreciate the help!

Here is the code for Atom Echo wakeword with sound feedback:

substitutions:
  name: 
  friendly_name: 
  static_ip: 192.168.1.17
  

esphome:
  name: "${name}"
  friendly_name: "${friendly_name}"
  name_add_mac_suffix: false
  project:
    name: m5stack.atom-echo-voice-assistant
    version: "1.0"
  min_version: 2024.6.0

file:
  - id: ok_sound
    file: wakeword_detected3.raw
  - id: timer_finished_wave_file
    file: https://github.com/esphome/firmware/raw/main/voice-assistant/sounds/timer_finished.wav

esp32:
  board: m5stack-atom
  framework:
    type: esp-idf

logger:
api:
  encryption:
    key: 

ota:
  platform: esphome

dashboard_import:
  package_import_url: github://esphome/firmware/voice-assistant/m5stack-atom-echo.yaml@main

wifi:
  networks:
  - ssid: !secret wifi_ssid
    password: !secret wifi_password

  manual_ip:
    static_ip: ${static_ip}
    gateway: 192.168.1.1
    subnet: 255.255.255.0

button:
  - platform: factory_reset
    id: factory_reset_btn
    name: Factory reset
  - platform: restart
    name: "${friendly_name} Restart"


i2s_audio:
  i2s_lrclk_pin: GPIO33
  i2s_bclk_pin: GPIO19

microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO23
    adc_type: external
    pdm: true

speaker:
  - platform: i2s_audio
    id: echo_speaker
    i2s_dout_pin: GPIO22
    dac_type: external
    mode: mono

voice_assistant:
  id: va
  microphone: echo_microphone
  speaker: echo_speaker
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  vad_threshold: 3
  on_wake_word_detected:
    - microphone.stop_capture:
    - wait_until:
        not:
          microphone.is_capturing:
    - lambda: id(echo_speaker).play(id(ok_sound), sizeof(id(ok_sound)));
  on_listening:
    - delay: 600ms
    - wait_until:
        not:
          speaker.is_playing:
    - microphone.capture:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Slow Pulse"
  on_stt_vad_end:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Fast Pulse"
  on_tts_start:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: none
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          speaker.is_playing:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led
        red: 100%
        green: 0%
        blue: 0%
        brightness: 100%
        effect: none
    - delay: 1s
    - script.execute: reset_led
  on_client_connected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.start_continuous:
          - script.execute: reset_led
  on_client_disconnected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.stop:
          - light.turn_off: led
  on_timer_finished:
    - voice_assistant.stop:
    - switch.turn_on: timer_ringing
    - wait_until:
        not:
          microphone.is_capturing:
    - light.turn_on:
        id: led
        red: 0%
        green: 100%
        blue: 0%
        brightness: 100%
        effect: "Fast Pulse"
    - while:
        condition:
          switch.is_on: timer_ringing
        then:
          - lambda: id(echo_speaker).play(id(timer_finished_wave_file), sizeof(id(timer_finished_wave_file)));
          - delay: 1s
    - wait_until:
        not:
          speaker.is_playing:
    - light.turn_off: led
    - switch.turn_off: timer_ringing
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.start_continuous:
          - script.execute: reset_led

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO39
      inverted: true
    name: Button
    disabled_by_default: true
    entity_category: diagnostic
    id: echo_button
    on_multi_click:
      - timing:
          - ON for at least 50ms
          - OFF for at least 50ms
        then:
          - if:
              condition:
                switch.is_on: timer_ringing
              then:
                - switch.turn_off: timer_ringing
              else:
                - if:
                    condition:
                      switch.is_off: use_wake_word
                    then:
                      - if:
                          condition: voice_assistant.is_running
                          then:
                            - voice_assistant.stop:
                            - script.execute: reset_led
                          else:
                            - voice_assistant.start:
                    else:
                      - voice_assistant.stop
                      - delay: 1s
                      - script.execute: reset_led
                      - script.wait: reset_led
                      - voice_assistant.start_continuous:
      - timing:
          - ON for at least 10s
        then:
          - button.press: factory_reset_btn

light:
  - platform: esp32_rmt_led_strip
    id: led
    name: None
    disabled_by_default: true
    entity_category: config
    pin: GPIO27
    default_transition_length: 0s
    chipset: SK6812
    num_leds: 1
    rgb_order: grb
    rmt_channel: 0
    effects:
      - pulse:
          name: "Slow Pulse"
          transition_length: 250ms
          update_interval: 250ms
          min_brightness: 50%
          max_brightness: 100%
      - pulse:
          name: "Fast Pulse"
          transition_length: 100ms
          update_interval: 100ms
          min_brightness: 50%
          max_brightness: 100%

script:
  - id: reset_led
    then:
      - if:
          condition:
            - switch.is_on: use_wake_word
            - switch.is_on: use_listen_light
          then:
            - light.turn_on:
                id: led
                red: 100%
                green: 89%
                blue: 71%
                brightness: 60%
                effect: none
          else:
            - light.turn_off: led

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(va).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);
      - script.execute: reset_led
  - platform: template
    name: Use listen light
    id: use_listen_light
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - script.execute: reset_led
  - platform: template
    id: timer_ringing
    optimistic: true
    internal: true
    restore_mode: ALWAYS_OFF
    on_turn_on:
      - delay: 15min
      - switch.turn_off: timer_ringing
    on_turn_off:
      - script.execute: reset_led


external_components:
  - source: github://pr#5230
    components:
      - esp_adf
    refresh: 0s
  - source: github://jesserockz/esphome-components
    components: [file]

esp_adf:

I think I cannot attach the soundfile here but would be happy to share that if someone needs a small sound that works on the Atom Echo.

And here is the not working “on device micro wakeword with confirmation sound” firmware that needs fixing:

substitutions:

  name: m5stack-atom-echo

  friendly_name: Atom Echo

  micro_wake_word_model: okay_nabu  # alexa, hey_jarvis, hey_mycroft are also supported

 

esphome:

  name: ${name}

  name_add_mac_suffix: false

  friendly_name: ${friendly_name}

  min_version: 2024.7.1

  project:

    name: m5stack.atom-echo-wake-word-voice-assistant

    version: "24.7.29"

 

esp32:

  board: m5stack-atom

  framework:

    type: esp-idf

 

logger:

api:

  encryption:

    key: 

ota:

  - platform: esphome

    id: ota_esphome

  - platform: http_request

    id: ota_http_request

 

update:

  - platform: http_request

    id: update_http_request

    name: Firmware

    source: https://firmware.esphome.io/wake-word-voice-assistant/m5stack-atom-echo/manifest.json

 

http_request:

 

dashboard_import:

  package_import_url: github://esphome/firmware/wake-word-voice-assistant/m5stack-atom-echo.adopted.yaml@main

 

wifi:

  ssid: !secret wifi_ssid

  password: !secret wifi_password

 

  manual_ip:

    static_ip: 192.168.1.40

    gateway: 192.168.1.1

    subnet: 255.255.255.0

 

improv_serial:

 

captive_portal:

 

button:

  - platform: factory_reset

    id: factory_reset_btn

    name: Factory reset

 

i2s_audio:

  - id: i2s_audio_bus

    i2s_lrclk_pin: GPIO33

    i2s_bclk_pin: GPIO19

 

microphone:

  - platform: i2s_audio

    id: echo_microphone

    i2s_din_pin: GPIO23

    adc_type: external

    pdm: true

 

speaker:

  - platform: i2s_audio

    id: echo_speaker

    i2s_dout_pin: GPIO22

    dac_type: external

    mode: mono

 

voice_assistant:

  id: va

  microphone: echo_microphone

  speaker: echo_speaker

  noise_suppression_level: 2

  auto_gain: 31dBFS

  volume_multiplier: 2.0

  vad_threshold: 3

  on_wake_word_detected:

    - microphone.stop_capture:

    - wait_until:

        not:

          microphone.is_capturing:

    - lambda: id(echo_speaker).play(id(ok_sound), sizeof(id(ok_sound)));

  on_listening:

    - light.turn_on:

        id: led

        blue: 100%

        red: 0%

        green: 0%

        effect: "Slow Pulse"

  on_stt_vad_end:

    - light.turn_on:

        id: led

        blue: 100%

        red: 0%

        green: 0%

        effect: "Fast Pulse"

  on_tts_start:

    - light.turn_on:

        id: led

        blue: 100%

        red: 0%

        green: 0%

        brightness: 100%

        effect: none

  on_end:

    - delay: 100ms

    - voice_assistant.stop:

    - wait_until:

        not:

          voice_assistant.is_running:

    - wait_until:

        not:

          switch.is_on: timer_ringing

    - if:

        condition:

          lambda: return id(wake_word_engine_location).state == "On device";

        then:

          - micro_wake_word.start:

          - script.execute: reset_led

        else:

          - voice_assistant.start_continuous:

          - script.execute: reset_led

  on_error:

    - light.turn_on:

        id: led

        red: 100%

        green: 0%

        blue: 0%

        brightness: 100%

        effect: none

    - delay: 2s

    - script.execute: reset_led

  on_client_connected:

    - delay: 2s  # Give the api server time to settle

    - if:

        condition:

          lambda: return id(wake_word_engine_location).state == "On device";

        then:

          - micro_wake_word.start:

        else:

          - voice_assistant.start_continuous:

          - script.execute: reset_led

  on_client_disconnected:

    - voice_assistant.stop:

    - micro_wake_word.stop:

  on_timer_finished:

    - voice_assistant.stop:

    - micro_wake_word.stop:

    - switch.turn_on: timer_ringing

    - wait_until:

        not:

          microphone.is_capturing:

    - light.turn_on:

        id: led

        red: 0%

        green: 100%

        blue: 0%

        brightness: 100%

        effect: "Fast Pulse"

    - while:

        condition:

          switch.is_on: timer_ringing

        then:

          - lambda: id(echo_speaker).play(id(timer_finished_wave_file), sizeof(id(timer_finished_wave_file)));

          - delay: 1s

    - wait_until:

        not:

          speaker.is_playing:

    - light.turn_off: led

    - switch.turn_off: timer_ringing

    - if:

        condition:

          lambda: return id(wake_word_engine_location).state == "On device";

        then:

          - micro_wake_word.start:

          - script.execute: reset_led

        else:

          - voice_assistant.start_continuous:

          - script.execute: reset_led

 

binary_sensor:

  # button does the following:

  # short click - stop a timer

  # if no timer then restart either microwakeword or voice assistant continuous

  - platform: gpio

    pin:

      number: GPIO39

      inverted: true

    name: Button

    disabled_by_default: true

    entity_category: diagnostic

    id: echo_button

    on_multi_click:

      - timing:

          - ON for at least 50ms

          - OFF for at least 50ms

        then:

          - if:

              condition:

                switch.is_on: timer_ringing

              then:

                - switch.turn_off: timer_ringing

              else:

                - if:

                    condition:

                      lambda: return id(wake_word_engine_location).state == "On device";

                    then:

                      - voice_assistant.stop

                      - micro_wake_word.stop:

                      - delay: 1s

                      - script.execute: reset_led

                      - script.wait: reset_led

                      - micro_wake_word.start:

                    else:

                      - if:

                          condition: voice_assistant.is_running

                          then:

                            - voice_assistant.stop:

                            - script.execute: reset_led

                      - voice_assistant.start_continuous:

      - timing:

          - ON for at least 10s

        then:

          - button.press: factory_reset_btn

 

light:

  - platform: esp32_rmt_led_strip

    id: led

    name: None

    disabled_by_default: true

    entity_category: config

    pin: GPIO27

    default_transition_length: 0s

    chipset: SK6812

    num_leds: 1

    rgb_order: grb

    rmt_channel: 0

    effects:

      - pulse:

          name: "Slow Pulse"

          transition_length: 250ms

          update_interval: 250ms

          min_brightness: 50%

          max_brightness: 100%

      - pulse:

          name: "Fast Pulse"

          transition_length: 100ms

          update_interval: 100ms

          min_brightness: 50%

          max_brightness: 100%

 

script:

  - id: reset_led

    then:

      - if:

          condition:

            - lambda: return id(wake_word_engine_location).state == "On device";

            - switch.is_on: use_listen_light

          then:

            - light.turn_on:

                id: led

                red: 100%

                green: 89%

                blue: 71%

                brightness: 60%

                effect: none

          else:

            - if:

                condition:

                  - lambda: return id(wake_word_engine_location).state != "On device";

                  - switch.is_on: use_listen_light

                then:

                  - light.turn_on:

                      id: led

                      red: 0%

                      green: 100%

                      blue: 100%

                      brightness: 60%

                      effect: none

                else:

                  - light.turn_off: led

 

switch:

  - platform: template

    name: Use listen light

    id: use_listen_light

    optimistic: true

    restore_mode: RESTORE_DEFAULT_ON

    entity_category: config

    on_turn_on:

      - script.execute: reset_led

    on_turn_off:

      - script.execute: reset_led

  - platform: template

    id: timer_ringing

    optimistic: true

    internal: true

    restore_mode: ALWAYS_OFF

    on_turn_on:

      - delay: 15min

      - switch.turn_off: timer_ringing

 

select:

  - platform: template

    entity_category: config

    name: Wake word engine location

    id: wake_word_engine_location

    optimistic: true

    restore_value: true

    options:

      - In Home Assistant

      - On device

    initial_option: On device

    on_value:

      - if:

          condition:

            lambda: return x == "In Home Assistant";

          then:

            - micro_wake_word.stop

            - delay: 500ms

            - lambda: id(va).set_use_wake_word(true);

            - voice_assistant.start_continuous:

      - if:

          condition:

            lambda: return x == "On device";

          then:

            - lambda: id(va).set_use_wake_word(false);

            - voice_assistant.stop

            - delay: 500ms

            - micro_wake_word.start

 

external_components:

  - source: github://pr#5230

    components:

      - esp_adf

    refresh: 0s

  - source: github://jesserockz/esphome-components

    components: [file]

    refresh: 0s

 

esp_adf:

 

file:

  - id: ok_sound

    file: wakeword_detected3.raw

  - id: timer_finished_wave_file

    file: https://github.com/esphome/firmware/raw/main/voice-assistant/sounds/timer_finished.wav

 

micro_wake_word:

  on_wake_word_detected:

    - voice_assistant.start:

        wake_word: !lambda return wake_word;

  vad:

  models:

    - model: ${micro_wake_word_model}

If anybody can spot the error and knows the solution that would be great.

I realised that the Echo’s work very well if they are restarted frequently.

Thanks and regards,

Merc

Merc · August 1, 2024, 8:20am

OK , solved it by moving the

lambda: id(echo_speaker).play(id(ok_sound), sizeof(id(ok_sound)));

part down to the micro wake word section.

Joe68 · August 13, 2024, 9:33pm

Hi, You say you solved it, can you please post the whole working code?

THX

blackie333 · August 14, 2024, 12:58pm

Have tried to adapt this Echo project version for M5Stack AtomU (USB microphone device). I have done this few time before successfully, firmware is linked but it’s size is too big for upload.
Can I do anything, device has 4MB flash but update via OTA is failing?
(I removed .wav file + speaker confirmation code + all openwakeword-related code )

Merc · August 14, 2024, 2:07pm

Clean build files first.
That solved the space issue for me. Since then I have never had it again and I am running 6 Atom Echos here.
Also, make sure the sound file you are using is as small as possible.

Merc · August 14, 2024, 2:21pm

Literally as I wrote.
Just take my yaml which I had pasted in above and move the code I had mentioned to the other section in the code.
Simple cut and paste.

TurfFiber · August 14, 2024, 7:53pm

So what are the benefits of having an on device wake word compared to having it on HA? Doesn’t HA have more horsepower to perform calculations?

Merc · August 15, 2024, 5:14pm

If you run many little satellites the server has to listen all the time. This load is gone when running the wake word on device.
On my Nuc I5 the processor load and with it the temperature came down significantly since switching 6 devices to local.
You can imagine what this means for running HA on a Pi.

ThijsM · August 25, 2024, 6:55pm

Hi @asherw83, Congrats on getting this to work! I’m close but my micro wake word models seem to be too large. How did you create your micro wake word model?

I have used Colab to create a custom model as described here: Create your own wake word - Home Assistant

I use ESPHome to install the model on the Echo, but after install on startup I get the error “Failed to allocate tensors for the streaming model”.

It does work if I use one of the models from here, but my custom models must be larger (the files are about 4x the size as well):

Any suggestions? Thanks!

Vargash · September 6, 2024, 3:03pm

I got the same problem “Failed to allocate tensors for the streaming model” while trying to install a custom wakeword on-device…

Is there any tricks to reduce the size of trained words in order to make them loadable in the Atom Echo?