Have you enabled to expose to Assist in the Wyoming whisper entity?
Hi AnaCoda,
Thank you for answer. I didn’t have it allowed, so i try, but with the same result. Also tried latest esphome update version 2023.10.1 and disabled tts. I will wait for some time and then try again. 
I’ve got it working on an esp32_devkitc_v4
First it didn’t work with the esphome project m5stack.
I adopted the firmware that came from the web installer and did a few overrides:
substitutions:
name: m5stack-atom-echo-d3187c
friendly_name: M5Stack Atom Echo d3187c
led_brightness: 10%
packages:
m5stack.atom-echo-voice-assistant: github://esphome/firmware/voice-assistant/m5stack-atom-echo.yaml@main
esphome:
name: ${name}
name_add_mac_suffix: false
friendly_name: ${friendly_name}
api:
encryption:
key: GEHEIM
esp32:
board: esp32dev
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
# changes start here, above everything is original
i2s_audio:
i2s_lrclk_pin: GPIO33
i2s_bclk_pin: GPIO18
microphone:
- id: !extend echo_microphone
pdm: false
light:
- id: !extend led
chipset: ws2812
binary_sensor:
- id: !extend echo_button
pin:
number: GPIO14
The binary sensor doesn’t matter… the culprit for me was pin 19 for i2s_bclk_pin which i changed to pin 18 and probably the pdm boolean which i changed to false.
According to this pinout: ESP32 DevKitC v4 high resolution pinout and specs – Renzo Mischianti
pin 18 is the SCK pin on this particular board… not an expert though but it works at leat 
Next i have to add and test the dac and speaker to get the same thing running like stated on the Chapter 4 blog post. Both are waiting in my mailbox.
Hope this helps
Edit:
I have to add that the reception does not work as great as demoed in the videos. Maybe it’s worth waiting for “official” firmware and schematics.
For Reference:
Hardware:
- ESP32: DevKitC V4
- Mic: INMP441
- DAC: MAX98357
Firmware
see above
Pinout:
Microphone
Board → ESP32 (label on ESP)
- VDD → 3.3V
- GND → GND
- SD → GPIO23 (23)
- L/R → GND
- WS → GPIO33 (33)
- SCK → GPIO18 (18)
DAC
Board → ESP32 (label on ESP)
- LRC → GPIO33 (33)
- BCLK → GPIO18 (18)
- DIN → GPIO22 (22)
- GAIN → Unused
- SD → Unused
- GND → GND
- Vin → 3.3V
Today I finally got a setup to work reliably I followed Lewis’ guide.
Removing the button and LED from my original setup and using the pins suggested in the video. My speaker is still a bit crackly but it is working as well as my echo atom.
1 Like
I had problems with the button. It switched constantly on and off so i activated the internal pulldown to stop the gpio from floating.
@Arh which microphone are you using? Do you have good speech recognition from a distance? I use INMP441 and… well impressing enough but not usable. It does not work reliably from a meter distance. If you use the ICS-43434, where did you buy it?
1 of the 2 I am playing with uses an INMP441 the other an SPH0645. Both work well after removing this line
auto_gain: 31dBFS
adding the entry into your config to enable saving the recordings to disk will help you understand what the recordings re like
assist_pipeline:
debug_recording_dir: /share/assist_pipeline
Its still early days with mine as I have only been playing around for an hour or 2 so far.
1 Like
Great, i changed it to the default (0dBFS) and it works way better now. Thanks
I can read the logs of the docker container i run whiser in. I installed it to my homegrown NAS which has way more CPU Power than my HA machine.
Commercial:
Please upvote feature request Assist: "Triggered by" for sentences for having an even better experience
If you guys also experience crackling speech output with a MAX98357 amp you can try to mess around with the gain settings.
- 15dB if a 100K resistor is connected between GAIN and GND
- 12dB if GAIN is connected directly to GND
- 9dB if GAIN is not connected to anything (this is the default)
- 6dB if GAIN is connected directly to Vin
- 3dB if a 100K resistor is connected between GAIN and Vin
My speaker works well with the 6dB setting.
2 Likes
Whoa, it’s great, finally I used config from this guy and it is working. Used classic ESP32. I didn’t try it with ESP32 S3, maybe later. Also it works with Wake word. Just need to do something with a few second delay between voice command and realise, but I will need to add more CPU power to virtual PC.
Secondly it will me teach to correctly speak in English like I’m not native speaker and I don’t want use native language because I thought that it will working worse than English.
Great work, thank you.
Guys, I’m losing my Mind…
I’ve messed around with this for 2 days straight now, but i can’t get anything to work. When i connect the Amp, my ESP does not boot anymore, doesn’t matter if i connect it to 3v3 or 5v. Can’t get any logs out of it as soon as it is connected. If i only connect the Mic, it boots, but I don’t think i was able to trigger a wake word (looked at the ESP Logs aswell as the Binary Sensor “Assist in Use”. I Tried two brand new Amp Boards (These), Two Mics (These), and even tried a new ESP (Here) I also tried switching the Pins around from those used in the Video (13, 25, 26, 27) to those used in the initial post (18, 22, 23, 33), but it’s always the same. Am I doing anything wrong? Are my Amps broken?
Post your yaml.
mine looks like this
esphome:
name: assistant
friendly_name: Assistant
on_boot:
- priority: -100
then:
- wait_until: api.connected
- delay: 1s
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:
esp32:
board: esp32dev
framework:
type: esp-idf
version: recommended
# Enable logging
logger:
# Enable Home Assistant API
api:
encryption:
key: "HKyfJKTonftR4oSfmBmnJzz01HkOwX1SVz23dRzwrUM="
ota:
password: "65c5ad2a5e6322259f99d99e297199c3"
wifi:
ssid: !secret wifi_ssid_4
password: !secret wifi_password_4
manual_ip:
static_ip: 192.168.1.240
gateway: 192.168.1.254
subnet: 255.255.255.0
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: "Esp32-Mic-Speaker"
password: "9vYvAFzzPjuc"
improv_serial:
i2s_audio:
i2s_lrclk_pin: GPIO27
i2s_bclk_pin: GPIO26
microphone:
- platform: i2s_audio
id: echo_microphone
i2s_din_pin: GPIO13
adc_type: external
pdm: false
speaker:
- platform: i2s_audio
id: echo_speaker
i2s_dout_pin: GPIO25
dac_type: external
mode: mono
voice_assistant:
id: va
microphone: echo_microphone
speaker: echo_speaker
noise_suppression_level: 2
#auto_gain: 31dBFS
volume_multiplier: 2.0
vad_threshold: 3
on_listening:
- light.turn_on:
id: led
blue: 100%
red: 0%
green: 0%
brightness: 10%
effect: pulse
on_tts_start:
- light.turn_on:
id: led
blue: 0%
red: 0%
green: 100%
brightness: 100%
effect: pulse
on_end:
- delay: 100ms
- wait_until:
not:
speaker.is_playing:
- script.execute: reset_led
on_error:
- light.turn_on:
id: led
blue: 0%
red: 100%
green: 0%
brightness: 100%
effect: none
- delay: 1s
- script.execute: reset_led
- script.wait: reset_led
- lambda: |-
if (code == "wake-provider-missing" || code == "wake-engine-missing") {
id(use_wake_word).turn_off();
}
binary_sensor:
- platform: gpio
pin:
number: GPIO39
inverted: true
name: Button
disabled_by_default: true
entity_category: diagnostic
id: echo_button
on_click:
- if:
condition:
switch.is_off: use_wake_word
then:
- if:
condition: voice_assistant.is_running
then:
- voice_assistant.stop:
- script.execute: reset_led
else:
- voice_assistant.start:
else:
- voice_assistant.stop
- delay: 1s
- script.execute: reset_led
- script.wait: reset_led
- voice_assistant.start_continuous:
light:
- platform: esp32_rmt_led_strip
id: led
name: None
disabled_by_default: true
entity_category: config
pin: GPIO14
default_transition_length: 0s
chipset: ws2812
num_leds: 1
rgb_order: grb
rmt_channel: 0
effects:
- pulse:
transition_length: 250ms
update_interval: 250ms
script:
- id: reset_led
then:
- if:
condition:
- switch.is_on: use_wake_word
- switch.is_on: use_listen_light
then:
- light.turn_on:
id: led
blue: 100%
red: 100%
green: 0%
brightness: 30%
effect: none
else:
- light.turn_off: led
switch:
- platform: template
name: Use wake word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- lambda: id(va).set_use_wake_word(true);
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
- script.execute: reset_led
on_turn_off:
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
- script.execute: reset_led
- platform: template
name: Use Listen Light
id: use_listen_light
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- script.execute: reset_led
on_turn_off:
- script.execute: reset_led
external_components:
- source: github://pr#5230
components:
- esp_adf
refresh: 0s
esp_adf:
2 Likes
Hello everyone!
I have what I believe is a pebkac error, but I’ve been working on this off and on for weeks and finally decided to ask for help from people smarter then I.
The problem - No sound from INMP441 using ESP23-S3-devkit-C-1. I have used 3 different mics and 2 boards. I have tried several different ESPHome configs (I’ll post the two main ones) and an Arduino config meant to stream the mic (will also post). I’ve also tried different pins on the ESP, and I’ve used the debug_recording_dir but the recordings all have no sound. What I’m hoping for is some way to troubleshoot why I’m not getting any sound from this mic, I’m just past my current knowledge levels and feel a bit lost.
Thank you for reading and any advice, I rely on these forums for almost everything I do. Though usually someone already asked the question I have!
esphome:
name: ha-microphone
friendly_name: HA-Microphone
esp32:
board: esp32-s3-devkitc-1
framework:
type: arduino
# Enable logging
logger:
# Enable Home Assistant API
api:
encryption:
key: ""
ota:
password: ""
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
i2s_audio:
i2s_lrclk_pin: GPIO09 #ws
i2s_bclk_pin: GPIO12 #clk
microphone:
- platform: i2s_audio
i2s_din_pin: GPIO10 #sd
id: adc_mic
adc_type: external
pdm: false
voice_assistant:
microphone: adc_mic
binary_sensor:
- platform: gpio
name: "Voice"
pin:
number: GPIO0
inverted: True
on_press:
- voice_assistant.start:
on_release:
- voice_assistant.stop:
esphome:
name: bedroom-assist
friendly_name: bedroom_assist
esp32:
board: esp32-s3-devkitc-1
framework:
type: arduino
# Enable logging
logger:
# Enable Home Assistant API
api:
encryption:
key: ""
ota:
password: ""
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: "Bedroom-Assist Fallback Hotspot"
password: ""
captive_portal:
##commented out because I read one in one random forum it caused problems for someone. Will try once mic works.
#light:
# - platform: esp32_rmt_led_strip
# rgb_order: GRB
# pin: GPIO48
# num_leds: 30
# rmt_channel: 0
# chipset: ws2812
# name: "Bedroom Assit LED"
i2s_audio:
id: bedroom_i2s_in
i2s_lrclk_pin: GPIO9 #WS / LRC
i2s_bclk_pin: GPIO12 #SCK / BCLK
microphone:
- platform: i2s_audio
id: bedroom_mic
i2s_din_pin: GPIO10 #SD
adc_type: external
pdm: false
channel: right
bits_per_sample: 32bit
i2s_audio_id: bedroom_i2s_in
speaker:
- platform: i2s_audio
id: bedroom_speaker
dac_type: external
i2s_dout_pin: GPIO07 #DIN
mode: mono
i2s_audio_id: bedroom_i2s_in
voice_assistant:
microphone: bedroom_mic
id: bedroom_va
noise_suppression_level: 2
auto_gain: 31dBFS
volume_multiplier: 2.0
# use_wake_word: true
speaker: bedroom_speaker
on_error:
- if:
condition:
switch.is_on: bedroom_use_wake_word
then:
- switch.turn_off: bedroom_use_wake_word
- switch.turn_on: bedroom_use_wake_word
on_client_connected:
- if:
condition:
switch.is_on: bedroom_use_wake_word
then:
- voice_assistant.start_continuous:
on_client_disconnected:
- if:
condition:
switch.is_on: bedroom_use_wake_word
then:
- voice_assistant.stop:
binary_sensor:
- platform: status
name: Bedroom API Connection
id: bedroom_api_connection
filters:
- delayed_on: 1s
on_press:
- if:
condition:
switch.is_on: bedroom_use_wake_word
then:
- voice_assistant.start_continuous:
on_release:
- if:
condition:
switch.is_on: bedroom_use_wake_word
then:
- voice_assistant.stop:
switch:
- platform: template
name: Bedroom Use Wake Word
id: bedroom_use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- lambda: id(bedroom_va).set_use_wake_word(true);
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
on_turn_off:
- voice_assistant.stop
- lambda: id(bedroom_va).set_use_wake_word(false);
/////////////////////////////////////////////////////////////////
/*
Broadcasting Your Voice with ESP32-S3 & INMP441
For More Information: https://youtu.be/qq2FRv0lCPw
Created by Eric N. (ThatProject)
*/
/////////////////////////////////////////////////////////////////
/*
- Device
ESP32-S3 DevKit-C
https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/hw-reference/esp32s3/user-guide-devkitc-1.html
- Required Library
Arduino ESP32: 2.0.9
Arduino Websockets: 0.5.3
https://github.com/gilmaimon/ArduinoWebsockets
*/
#include <driver/i2s.h>
#include <WiFi.h>
#include <ArduinoWebsockets.h>
#define I2S_SD 10
#define I2S_WS 9
#define I2S_SCK 12
#define I2S_PORT I2S_NUM_0
#define bufferCnt 10
#define bufferLen 1024
int16_t sBuffer[bufferLen];
const char* ssid = "";
const char* password = "";
const char* websocket_server_host = "";
const uint16_t websocket_server_port = 8888; // <WEBSOCKET_SERVER_PORT>
using namespace websockets;
WebsocketsClient client;
bool isWebSocketConnected;
void onEventsCallback(WebsocketsEvent event, String data) {
if (event == WebsocketsEvent::ConnectionOpened) {
Serial.println("Connnection Opened");
isWebSocketConnected = true;
} else if (event == WebsocketsEvent::ConnectionClosed) {
Serial.println("Connnection Closed");
isWebSocketConnected = false;
} else if (event == WebsocketsEvent::GotPing) {
Serial.println("Got a Ping!");
} else if (event == WebsocketsEvent::GotPong) {
Serial.println("Got a Pong!");
}
}
void i2s_install() {
// Set up I2S Processor configuration
const i2s_config_t i2s_config = {
.mode = i2s_mode_t(I2S_MODE_MASTER | I2S_MODE_RX),
//.sample_rate = 44100,
.sample_rate = 16000,
.bits_per_sample = i2s_bits_per_sample_t(16),
.channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
.communication_format = i2s_comm_format_t(I2S_COMM_FORMAT_STAND_I2S),
.intr_alloc_flags = 0,
.dma_buf_count = bufferCnt,
.dma_buf_len = bufferLen,
.use_apll = false
};
i2s_driver_install(I2S_PORT, &i2s_config, 0, NULL);
}
void i2s_setpin() {
// Set I2S pin configuration
const i2s_pin_config_t pin_config = {
.bck_io_num = I2S_SCK,
.ws_io_num = I2S_WS,
.data_out_num = -1,
.data_in_num = I2S_SD
};
i2s_set_pin(I2S_PORT, &pin_config);
}
void setup() {
Serial.begin(115200);
connectWiFi();
connectWSServer();
xTaskCreatePinnedToCore(micTask, "micTask", 10000, NULL, 1, NULL, 1);
}
void loop() {
}
void connectWiFi() {
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
}
void connectWSServer() {
client.onEvent(onEventsCallback);
while (!client.connect(websocket_server_host, websocket_server_port, "/")) {
delay(500);
Serial.print(".");
}
Serial.println("Websocket Connected!");
}
void micTask(void* parameter) {
i2s_install();
i2s_setpin();
i2s_start(I2S_PORT);
size_t bytesIn = 0;
while (1) {
esp_err_t result = i2s_read(I2S_PORT, &sBuffer, bufferLen, &bytesIn, portMAX_DELAY);
if (result == ESP_OK && isWebSocketConnected) {
client.sendBinary((const char*)sBuffer, bytesIn);
}
}
}```do you have the LR pin on the mic connected to gnd or vcc ?
Here is my working config for s3-dev kit board You need to set the mic to left to work on these boards:
substitutions:
voice_assist_idle_phase_id: '1'
voice_assist_listening_phase_id: '2'
voice_assist_thinking_phase_id: '3'
voice_assist_replying_phase_id: '4'
voice_assist_not_ready_phase_id: '10'
voice_assist_error_phase_id: '11'
voice_assist_muted_phase_id: '12'
esphome:
name: master-bedroom-voice-assistant
friendly_name: Master Bedroom Voice assistant
platformio_options:
board_build.flash_mode: dio
on_boot:
priority: 600
then:
- script.execute: control_led
- delay: 30s
- if:
condition:
lambda: return id(init_in_progress);
then:
- lambda: id(init_in_progress) = false;
- script.execute: control_led
psram:
mode: octal
speed: 80MHz
esp32:
board: esp32-s3-devkitc-1
variant: esp32s3
framework:
type: esp-idf
version: recommended
components:
- name: esphome_board
source: github://jesserockz/esphome-esp-adf-board@main
refresh: 0s
sdkconfig_options:
CONFIG_ESP32_S3_BOX_BOARD: "y"
CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y"
CONFIG_ESP32S3_DATA_CACHE_64KB: "y"
CONFIG_ESP32S3_DATA_CACHE_LINE_64B: "y"
CONFIG_AUDIO_BOARD_CUSTOM: "y"
# Enable logging
logger:
# Enable Home Assistant API
api:
encryption:
key: "snip"
on_client_connected:
- script.execute: control_led
on_client_disconnected:
- script.execute: control_led
ota:
password: "snip"
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
manual_ip:
static_ip: *
gateway: *
subnet: *
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: "Master-Bedroom-Voice-Assistant"
password: "snip"
esp_adf:
external_components:
- source: github://pr#5230
components:
- esp_adf
refresh: 0s
captive_portal:
light:
- platform: esp32_rmt_led_strip
rgb_order: GRB
pin: GPIO18
num_leds: 3
rmt_channel: 0
chipset: WS2812
name: "Status LED"
id: led
default_transition_length: 0s
effects:
- pulse:
name: "extra_slow_pulse"
transition_length: 800ms
update_interval: 800ms
min_brightness: 0%
max_brightness: 30%
- pulse:
name: "slow_pulse"
transition_length: 250ms
update_interval: 250ms
min_brightness: 50%
max_brightness: 100%
- pulse:
name: "fast_pulse"
transition_length: 100ms
update_interval: 100ms
min_brightness: 50%
max_brightness: 100%
i2s_audio:
- id: i2s_in
i2s_lrclk_pin: GPIO3 ##INMP441-WS
i2s_bclk_pin: GPIO2 ##INMP441-SCK
- id: i2s_out
i2s_lrclk_pin: GPIO6 ##Max98357 - LRC
i2s_bclk_pin: GPIO20 ###Max98357 - BCLK
## l/R pin on ##INMP441 is connected to ground
microphone:
platform: i2s_audio
id: external_microphone
adc_type: external
i2s_audio_id: i2s_in
i2s_din_pin: GPIO4
channel: left
pdm: false
speaker:
platform: i2s_audio
id: external_speaker
dac_type: external
i2s_audio_id: i2s_out
i2s_dout_pin: GPIO8 ###Max98357 - DIN
mode: mono
voice_assistant:
id: va
microphone: external_microphone
speaker: external_speaker
use_wake_word: true
noise_suppression_level: 2
auto_gain: 31dBFS
volume_multiplier: 2.5
on_listening:
- lambda: id(voice_assistant_phase) = ${voice_assist_listening_phase_id};
- script.execute: control_led
on_stt_vad_end:
- lambda: id(voice_assistant_phase) = ${voice_assist_thinking_phase_id};
- script.execute: control_led
on_tts_stream_start:
- lambda: id(voice_assistant_phase) = ${voice_assist_replying_phase_id};
- script.execute: control_led
on_tts_stream_end:
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
- script.execute: control_led
on_error:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- lambda: id(voice_assistant_phase) = ${voice_assist_error_phase_id};
- script.execute: control_led
- delay: 1s
- if:
condition:
switch.is_on: use_wake_word
then:
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
else:
- lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
- script.execute: control_led
on_client_connected:
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
else:
- lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
- script.execute: control_led
on_client_disconnected:
- lambda: id(voice_assistant_phase) = ${voice_assist_not_ready_phase_id};
- script.execute: control_led
switch:
- platform: template
name: Use Wake Word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
on_turn_on:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
- script.execute: control_led
on_turn_off:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- voice_assistant.stop
- lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
- script.execute: control_led
globals:
- id: init_in_progress
type: bool
restore_value: no
initial_value: 'true'
- id: voice_assistant_phase
type: int
restore_value: no
initial_value: ${voice_assist_not_ready_phase_id}
script:
- id: control_led
then:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- if:
condition:
wifi.connected:
then:
- if:
condition:
api.connected:
then:
- lambda: |
switch(id(voice_assistant_phase)) {
case ${voice_assist_listening_phase_id}:
id(led).turn_on().set_rgb(0, 0, 1).set_brightness(1.0).set_effect("none").perform();
break;
case ${voice_assist_thinking_phase_id}:
id(led).turn_on().set_rgb(0, 1, 0).set_effect("slow_pulse").perform();
break;
case ${voice_assist_replying_phase_id}:
id(led).turn_on().set_rgb(0, 0, 1).set_brightness(1.0).set_effect("fast_pulse").perform();
break;
case ${voice_assist_error_phase_id}:
id(led).turn_on().set_rgb(1, 1, 1).set_brightness(.5).set_effect("none").perform();
break;
case ${voice_assist_muted_phase_id}:
id(led).turn_off().perform();
break;
case ${voice_assist_not_ready_phase_id}:
id(led).turn_on().perform();
break;
default:
id(led).turn_on().set_rgb(1, 0, 0).set_brightness(0.2).set_effect("none").perform();
break;
}
else:
- light.turn_off:
id: led
else:
- light.turn_off:
id: led
else:
- light.turn_on:
id: led
blue: 50%
red: 50%
green: 50%
effect: "fast_pulse"
Video of them working:
I have it connected to ground.
You found my pebkac error! I knew it had to use left, I thought I had it on left, but I clearly had it on right. Changed it to left and it is working! I have tweaking to do to get the light working the way I want (your example config is great!) and my speaker currently doesn’t sound very good. Once I have everything working up to par I’ll post my final config to provide one more working example. I don’t have as much time as I’d like so it might take a bit.
Thank you all again!
Anyone ever got this to work on a ESP32-C3 board?
Yaml
esphome:
name: test
friendly_name: TEST
platformio_options:
board_build.flash_mode: dio
board_build.f_cpu: 80000000L
on_boot:
- priority: -100
then:
- wait_until: api.connected
- delay: 1s
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:
esp32:
board: esp32-c3-devkitm-1
variant: esp32c3
framework:
type: esp-idf
# Enable logging cuz board has UART
logger:
baud_rate: 115200
hardware_uart: UART0
level: debug
#Pin25 GND
#Pin26 VCC
#Pin27 aka GPIO05 SD Mic
#Pin23 GPIO07 SCK
#Pin22 GPIO06 WS
debug:
# Enable Home Assistant API
api:
encryption:
key: "3gY/="
ota:
password: ""
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
manual_ip:
static_ip: 192.168.1.192
gateway: 192.168.1.1
subnet: 255.255.255.0
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: "Test Fallback Hotspot"
password: "*"
web_server:
port: 80
captive_portal:
i2s_audio:
i2s_lrclk_pin: GPIO6 ##INMP441-WS
i2s_bclk_pin: GPIO7 ##INMP441-SCK
microphone:
platform: i2s_audio
id: external_microphone
adc_type: external
i2s_din_pin: GPIO5
channel: left
pdm: false
voice_assistant:
microphone: external_microphone
use_wake_word: false
noise_suppression_level: 2
auto_gain: 31dBFS
volume_multiplier: 2.0
id: assist
on_tts_end:
- homeassistant.service:
service: media_player.play_media
data:
media_content_id: !lambda 'return x;'
media_content_type: audio/mpeg
entity_id: media_player.windows
switch:
- platform: template
name: Use wake word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- lambda: id(assist).set_use_wake_word(true);
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
on_turn_off:
- voice_assistant.stop
- lambda: id(assist).set_use_wake_word(false);
Log on the Device
INFO ESPHome 2024.2.1
INFO Reading configuration /config/esphome/test.yaml…
INFO Starting log output from 192.168.1.192 using esphome API
INFO Successfully connected to test @ 192.168.1.192 in 0.109s
INFO Successful handshake with test @ 192.168.1.192 in 0.087s
[16:24:10][I][app:102]: ESPHome version 2024.2.1 compiled on Mar 4 2024, 16:20:37
[16:24:10][C][wifi:577]: WiFi:
[16:24:10][C][wifi:409]: Local MAC: 54:32:04:47:D3:E4
[16:24:10][C][wifi:414]: SSID: [redacted]
[16:24:10][C][wifi:415]: IP Address: 192.168.1.192
[16:24:10][C][wifi:417]: BSSID: [redacted]
[16:24:10][C][wifi:418]: Hostname: ‘test’
[16:24:10][C][wifi:420]: Signal strength: -50 dB ▂▄▆█
[16:24:10][C][wifi:424]: Channel: 12
[16:24:10][C][wifi:425]: Subnet: 255.255.255.0
[16:24:10][C][wifi:426]: Gateway: 192.168.1.1
[16:24:10][C][wifi:427]: DNS1: 0.0.0.0
[16:24:10][C][wifi:428]: DNS2: 0.0.0.0
[16:24:10][C][logger:447]: Logger:
[16:24:10][C][logger:448]: Level: DEBUG
[16:24:10][C][logger:449]: Log Baud Rate: 115200
[16:24:10][C][logger:451]: Hardware UART: UART0
[16:24:10][C][template.switch:068]: Template Switch ‘Use wake word’
[16:24:10][C][template.switch:091]: Restore Mode: restore defaults to ON
[16:24:10][C][template.switch:057]: Optimistic: YES
[16:24:10][C][captive_portal:088]: Captive Portal:
[16:24:10][C][web_server:168]: Web Server:
[16:24:10][C][web_server:169]: Address: 192.168.1.192:80
[16:24:10][C][mdns:115]: mDNS:
[16:24:10][C][mdns:116]: Hostname: test
[16:24:10][C][ota:096]: Over-The-Air Updates:
[16:24:10][C][ota:097]: Address: 192.168.1.192:3232
[16:24:10][C][ota:100]: Using Password.
[16:24:10][C][ota:103]: OTA version: 2.
[16:24:10][W][ota:107]: Last Boot was an unhandled reset, will proceed to safe mode in 8 restarts
[16:24:10][C][api:139]: API Server:
[16:24:10][C][api:140]: Address: 192.168.1.192:6053
[16:24:10][C][api:142]: Using noise encryption: YES
[16:24:10][C][debug:064]: Debug component:
[16:24:10][D][debug:076]: ESPHome version 2024.2.1
[16:24:10][D][debug:080]: Free Heap Size: 161116 bytes
[16:24:10][D][debug:159]: Chip: Model=ESP32-C3, Features=WIFI_BGN,BLE, Cores=1, Revision=4
[16:24:10][D][debug:167]: ESP-IDF Version: 4.4.6
[16:24:10][D][debug:172]: EFuse MAC: 54:32:04:47:D3:E4
[16:24:10][D][debug:272]: Reset Reason: Power On Reset
[16:24:10][D][debug:317]: Wakeup Reason: Unknown
[16:24:17][D][switch:012]: ‘Use wake word’ Turning ON.
[16:24:17][D][switch:055]: ‘Use wake word’: Sending state ON
[16:24:17][D][voice_assistant:414]: State changed from IDLE to START_PIPELINE
[16:24:17][D][voice_assistant:420]: Desired state set to START_MICROPHONE
[16:24:17][D][voice_assistant:118]: microphone not running
[16:24:17][D][voice_assistant:202]: Requesting start…
[16:24:17][D][voice_assistant:414]: State changed from START_PIPELINE to STARTING_PIPELINE
[16:24:17][D][voice_assistant:435]: Client started, streaming microphone
[16:24:17][D][voice_assistant:414]: State changed from STARTING_PIPELINE to START_MICROPHONE
[16:24:17][D][voice_assistant:420]: Desired state set to STREAMING_MICROPHONE
[16:24:17][D][voice_assistant:155]: Starting Microphone
[16:24:17][D][voice_assistant:414]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[16:24:17][D][esp-idf:000]: I (28922) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4
[16:24:17][D][voice_assistant:521]: Event Type: 1
[16:24:17][D][voice_assistant:524]: Assist Pipeline running
[16:24:17][D][voice_assistant:414]: State changed from STARTING_MICROPHONE to STREAMING_MICROPHONE
[16:24:17][D][voice_assistant:521]: Event Type: 9
[16:24:23][D][switch:016]: ‘Use wake word’ Turning OFF.
[16:24:23][D][switch:055]: ‘Use wake word’: Sending state OFF
[16:24:23][D][voice_assistant:514]: Signaling stop…
[16:24:23][D][voice_assistant:414]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE
[16:24:23][D][voice_assistant:420]: Desired state set to IDLE
[16:24:23][D][voice_assistant:521]: Event Type: 0
[16:24:23][E][voice_assistant:651]: Error: no_wake_word - No wake word detected
[16:24:23][D][voice_assistant:514]: Signaling stop…
[16:24:23][D][voice_assistant:414]: State changed from STOP_MICROPHONE to STOP_MICROPHONE
[16:24:23][D][voice_assistant:420]: Desired state set to IDLE
[16:24:23][D][voice_assistant:414]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[16:24:23][D][esp-idf:000]: I (34931) I2S: DMA queue destroyed
[16:24:23][D][voice_assistant:521]: Event Type: 2
[16:24:23][D][voice_assistant:611]: Assist Pipeline ended
[16:24:23][D][voice_assistant:414]: State changed from STOPPING_MICROPHONE to IDLE
I have activate the logging for Assistant in configuration.yaml
assist_pipeline:
debug_recording_dir: /config/stt/assist_pipeline
But the file always ends up 0.0KiB
When i use the Browser/app and use voice assistant there this does not happen!
So i assume the mic just doesn’t work.
I tried different things, i even have a Lolin D32 Board here which has a normal ESP32 chip. didn’t get that to work either.
I see people using esp_adf from pull #5230 but that’s S3 only right?
I don’t have speakers as i want to use my media player for that, so i only need the microphone.
L/R is linked to Ground.
BTW the mic itself is working, as tested with arduino IDE.
this can be caused by many things, if the mic is working in arduino, then everything hardware related will be working, and your yaml is correct for the mic i2s setup so no problem there.
first thing to check is that HA can stream to the device and the device can stream to HA. Check the HA Core log to see if there are any indications in there as to what the issue is Settings>>system>>logs . If the esp and HA are on different VLAN’s then have them both on the same VLAN. In order to stream audio HA needs to have all UDP Ports open, so check that any firewall is not blocking access through ALL udp ports. If running HA in docker, HA has to have it’s own IP address and network needs to be configured to host. Check those things out and let us know how you get on.
I am now at same stage as Jedman with esp32 wroom and INMP441, I can see the audio with DumpDisplay arduino test app using exact same pins but I get no sound in assist_pipeline but if I leave mic DO line unconnected I get garbage in assist_pipeline. I cannot think how this is possible.
It looks a bit like there might be some difference in the i2s setup.
My solution to this conundrum right now is to try build another unit with esp32-s3 and MP34DT01 and try that. If that does not work, I swap INMP441 to esp32-s3 and MP34DT01 to esp32. Hopefully I get one unit that just works™
Regarding high pitch sounds during playback clicks are typically caused by signal integrity issues since these are relatively high speed signals. Especially if the distortion stops if you touch the high speed lines with your hand, move things. You should be able to so solve such issues by using better quality dupont lines with higher load spring mechanisms, soldering the dupont lines without the plastic cases, twisting the high speed lines together with gnd or even adding some grounded tinfoil around the signal wires. Another thing I noticed the INMP441 can pick up alot of noise if the PCB touches a surface that is vibrating. A good solution probably is to use some polyester fiber like in pillows to make sure the MEMS microphone does touch any hard surfaces in casing that you have.
Longer pauses like over 300ms are most likely caused by buffering issues, which is another story altogether.