Send Help (ESP32-S3 N16R8 as Voice Satellite)

ESPHome
,
1

I’m a newbie and have been trying to make a Voice Satellite with ESP32-S3 N16R8 and INMP441 mic. I’ve checked the mic with rp2040, it definitely works. I managed to make a config that repeatedly changes the color of an onboard LED of esp32.

Problems start when I try to make a voice satellite with components set up:

  • esphome
  • esp32
  • logger: (empty)
  • api
  • ota
  • wifi
  • captive_portal: (empty)
  • psram
  • i2s_audio
  • microphone
  • voice_assistant
  • micro_wake_word

Something seems to be missing because no matter what I try in Home Assistant webGUI it doesn’t seem to even try to hear any sound at all or even initiate the mic. Logs show that esp32 has started, connected to wifi and stuff, but nothing related to voice recognition or micro_wake_word related.

Pls send help, I’ve been struggling for 2 weeks already

upd. Maybe I should initiate something on startup with some sort of script? Though documentation for components I’ve added never mentioned it.

2

Give us some more info like your yaml config, your wiring details, what dac if any you are using, how old the guide you are following is and a photo of you wiring. We may then be able to point you in the right direction.

3

My setup is pretty simple: only a ESP32-S3 N16R8 board and INMP441 mic, that’s all. I want to make a bare minimum start working: wait for a wake word (micro_wake_word), then send audio to host with Home Assistant OS (Raspberry Pi 4), process it and turn on a light bulb, for example. No feedback, no talking back, etc.

For now I can’t attach more than one media file, so I’ll describe with text that this setup is assembled partly on a breadboard: esp board is inserted into it, then one end of each wire is also inserted into the breadboard and their other end goes straight into mic.

inmp441-wiring
inmp441-wiring800×800 109 KB

I don’t follow any guide in particular, I try to use documentation provided on https://esphome.io/ and information on Assist - Talk to your smart home with Home Assistant - Home Assistant page (though it lacks any specific info on making a voice assistant with an esp32 that is not a consumer product). I’ve also tried to look into some configurations on Github, but they are mostly too bulky and filled with stuff I don’t yet need or even understand.

I am able to build and send a firmware via OTA, adopt the esp32 in webGUI of Home Assistant, I can even control an onboard led via webGUI, but that’s all.

For now, my config looks like this:

esphome:
  name: voice-satellite-1
  friendly_name: Voice Satellite 1
  on_boot:
    then:
      - script.execute: led_test

esp32:
  variant: ESP32S3
  framework:
    type: esp-idf

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "*******"

ota:
  - platform: esphome
    password: "*******"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Voice-Satellite-1"
    password: "*******"

captive_portal:

light:
  - platform: esp32_rmt_led_strip
    id: onboard_led
    name: "Onboard RGB LED"
    pin: GPIO48
    num_leds: 1
    rgb_order: GRB
    chipset: WS2812

script:
  - id: led_test
    mode: restart
    then:
      - light.turn_on:
          id: onboard_led
          red: 100%
          green: 0%
          blue: 0%
          brightness: 30%
      - delay: 2s
      - light.turn_on:
          id: onboard_led
          red: 50%
          green: 50%
          blue: 0%
          brightness: 30%
      - delay: 2s
      - light.turn_on:
          id: onboard_led
          red: 0%
          green: 100%
          blue: 0%
          brightness: 30%
      - delay: 2s
      - light.turn_on:
          id: onboard_led
          red: 0%
          green: 50%
          blue: 50%
          brightness: 30%
      - delay: 2s
      - light.turn_on:
          id: onboard_led
          red: 0%
          green: 0%
          blue: 100%
          brightness: 30%
      - delay: 2s
      - light.turn_on:
          id: onboard_led
          red: 50%
          green: 0%
          blue: 50%
          brightness: 30%
      - delay: 2s
      - script.execute: led_test



psram:
  mode: octal
  speed: 80MHZ

# I2S settings
i2s_audio:
  - id: i2s_in
    i2s_lrclk_pin: GPIO6 #ws
    i2s_bclk_pin: GPIO5 #sck

# Mic setup
microphone:
  - platform: i2s_audio
    id: inmp441_mic
    adc_type: external
    i2s_din_pin: GPIO7
    i2s_audio_id: i2s_in
    channel: left
    
# Voice Assistant
voice_assistant:
  microphone: inmp441_mic
  use_wake_word: true
  micro_wake_word: micro_wake_word_1
  conversation_timeout: 60s

micro_wake_word:
  id: micro_wake_word_1
  microphone:
    microphone: inmp441_mic
  task_stack_in_psram: true
  models:
    - model: "https://raw.githubusercontent.com/esphome/micro-wake-word-models/refs/heads/main/models/v2/alexa.json"
      probability_cutoff: 0.5
      sliding_window_size: 10

Thank you for your answer!

4

I can’t see anything fundamentally wrong with what you have, but i am no expert on esphome yaml stuff. but everything i have seen for voice assistant is far more complex yaml than yours so my guess is, it needs more.

here is the code i use with my voice boxes, it may help. you will see there is far more to the voice assistant section.

5

Am I getting it right that in your config both voice assistant and micro wake word are activated only when you trigger a switch, thus it doesn’t listen for a wake word all the time? And actions in the API block make it possible to activate and deactivate voice assistant straight from Home Assistant host GUI: you push a button and it starts listening, you push it again and it stops listening?

6

Its all controlled via the HA ui. No buttons on the device I have an enable voice button on the ui does what it suggests, if on it listens for the wake word all the time, if off it does not. It also has a mute button which switches off the mic if needed. This is pretty much a copy of the HA VPE firmware. It has in the code provision for a physical button but I have never fitted one.

7

I see. So it seems I have to script everything that is related to voice commands: make a script to turn on voice assistant when the board starts and connects to host via wifi, turn on micro wake word along with it, then I should make scripts that cover some of intermediate states(?). And if I understand anything, most of it is done right in the “voice_assistant” config block: on_client_connected, on_listening, on_end, etc.

8

I really have no idea, I did not write most of this code I just changed some bits that suited me. It is as said earlier based on the official HA VPE firmware. All I know is it works.

9

Here’s what I think-
Microwakeword needs to be active after the ESP boots;
When mww detects wakeword, it needs to start voice_assistant;
You might need bits_per_sample: 32 bit and sample_rate 16000 in microphone:

micro wake word should run when the esp connects - in api: need

  on_client_connected:
     - delay: 100ms
     - micro_wake_word.start:

micro_wake_word
  needs
  
   on_wake_word_detected:
     - voice_assistant.start:

voice_assistant:   does not need - use_wake_word  and micro_wake_word
 it needs
 an id:    (id: va)
 on_error:   on_idle:   that (re-)start micro_wake_word
  (on_tts_stream_end as well if there is speaker)