Malleve
February 28, 2024, 3:33pm
1
Hello everyone,
Unfortunately I have a problem with ESPHome and the voice_assistant component. I can’t see any signs that it is active or working.
Background: I have set up a voice pipeline and would expect to be able to interact with it using the voice_assistant component.
Hardware: INMP441, ESP32-S3
Config:
esphome:
name: jsensor-2
friendly_name: JSensor 2
on_boot:
- priority: -100
then:
- voice_assistant.start_continuous:
esp32:
board: esp32-s3-devkitc-1
framework:
type: arduino
# Enable logging
logger:
baud_rate: 0
# Enable Home Assistant API
api:
encryption:
key: "XXX"
ota:
password: "XXX"
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: "Fallback Hotspot"
password: "XXX"
captive_portal:
i2s_audio:
i2s_lrclk_pin: GPIO12
i2s_bclk_pin: GPIO13
microphone:
- platform: i2s_audio
id: mic
adc_type: external
i2s_din_pin: GPIO10
pdm: False
on_data:
- logger.log:
format: "Received %d bytes"
args: ['x.size()']
voice_assistant:
microphone: mic
use_wake_word: true
id: assist
Console Output:
[16:28:39][I][app:102]: ESPHome version 2024.2.1 compiled on Feb 28 2024, 16:21:29
[16:28:39][C][wifi:577]: WiFi:
[16:28:39][C][wifi:409]: Local MAC: C0:4E:30:17:29:58
[16:28:39][C][wifi:414]: SSID: [redacted]
[16:28:39][C][wifi:415]: IP Address: 10.4.21.225
[16:28:39][C][wifi:417]: BSSID: [redacted]
[16:28:39][C][wifi:418]: Hostname: 'jsensor-2'
[16:28:39][C][wifi:420]: Signal strength: -55 dB ▂▄▆█
[16:28:39][C][wifi:424]: Channel: 6
[16:28:39][C][wifi:425]: Subnet: 255.255.255.0
[16:28:39][C][wifi:426]: Gateway: 10.4.21.1
[16:28:39][C][wifi:427]: DNS1: 10.4.21.1
[16:28:39][C][wifi:428]: DNS2: 0.0.0.0
[16:28:39][C][logger:447]: Logger:
[16:28:39][C][logger:448]: Level: DEBUG
[16:28:39][C][logger:449]: Log Baud Rate: 0
[16:28:39][C][logger:451]: Hardware UART: USB_CDC
[16:28:39][C][captive_portal:088]: Captive Portal:
[16:28:39][C][mdns:115]: mDNS:
[16:28:39][C][mdns:116]: Hostname: jsensor-2
[16:28:39][C][ota:096]: Over-The-Air Updates:
[16:28:39][C][ota:097]: Address: jsensor-2.local:3232
[16:28:39][C][ota:100]: Using Password.
[16:28:39][C][ota:103]: OTA version: 2.
[16:28:39][C][api:139]: API Server:
[16:28:39][C][api:140]: Address: jsensor-2.local:6053
[16:28:39][C][api:142]: Using noise encryption: YES
nothing more and nothing related to voice_assistant
The microphone has already been tested using this script Arduino-DumbDisplay/projects/esp32ddmic/esp32ddmic.ino at 2911832e1661e3e6b53c705a136a8e9b8b0e1da6 · trevorwslee/Arduino-DumbDisplay · GitHub
Within the debug-assistant is also no entry
Does anyone have any ideas as to why this could be, what the error is or how I could debug further?
Arh
February 28, 2024, 3:56pm
2
This is the code I use.
improv_serial:
i2s_audio:
i2s_lrclk_pin: GPIO27
i2s_bclk_pin: GPIO26
microphone:
- platform: i2s_audio
id: echo_microphone
i2s_din_pin: GPIO13
adc_type: external
pdm: false
speaker:
- platform: i2s_audio
id: echo_speaker
i2s_dout_pin: GPIO25
dac_type: external
mode: mono
voice_assistant:
id: va
microphone: echo_microphone
speaker: echo_speaker
noise_suppression_level: 2
#auto_gain: 31dBFS
volume_multiplier: 2.0
vad_threshold: 3
on_listening:
- light.turn_on:
id: led
blue: 100%
red: 0%
green: 0%
brightness: 10%
effect: pulse
on_tts_start:
- light.turn_on:
id: led
blue: 0%
red: 0%
green: 100%
brightness: 100%
effect: pulse
on_end:
- delay: 100ms
- wait_until:
not:
speaker.is_playing:
- script.execute: reset_led
on_error:
- light.turn_on:
id: led
blue: 0%
red: 100%
green: 0%
brightness: 100%
effect: none
- delay: 1s
- script.execute: reset_led
- script.wait: reset_led
- lambda: |-
if (code == "wake-provider-missing" || code == "wake-engine-missing") {
id(use_wake_word).turn_off();
}
binary_sensor:
- platform: gpio
pin:
number: GPIO39
inverted: true
name: Button
disabled_by_default: true
entity_category: diagnostic
id: echo_button
on_click:
- if:
condition:
switch.is_off: use_wake_word
then:
- if:
condition: voice_assistant.is_running
then:
- voice_assistant.stop:
- script.execute: reset_led
else:
- voice_assistant.start:
else:
- voice_assistant.stop
- delay: 1s
- script.execute: reset_led
- script.wait: reset_led
- voice_assistant.start_continuous:
light:
- platform: esp32_rmt_led_strip
id: led
name: None
disabled_by_default: true
entity_category: config
pin: GPIO14
default_transition_length: 0s
chipset: ws2812
num_leds: 1
rgb_order: grb
rmt_channel: 0
effects:
- pulse:
transition_length: 250ms
update_interval: 250ms
script:
- id: reset_led
then:
- if:
condition:
- switch.is_on: use_wake_word
- switch.is_on: use_listen_light
then:
- light.turn_on:
id: led
blue: 100%
red: 100%
green: 0%
brightness: 30%
effect: none
else:
- light.turn_off: led
switch:
- platform: template
name: Use wake word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- lambda: id(va).set_use_wake_word(true);
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
- script.execute: reset_led
on_turn_off:
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
- script.execute: reset_led
- platform: template
name: Use Listen Light
id: use_listen_light
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- script.execute: reset_led
on_turn_off:
- script.execute: reset_led
external_components:
- source: github://pr#5230
components:
- esp_adf
refresh: 0s
esp_adf:
As you can see it is for more involved than your code, but it is mostly driving LEDs and speakers.
I am no expert but i do believe you need this as your framework rather than arduino
framework:
type: esp-idf
version: recommended
Malleve
February 28, 2024, 4:04pm
3
Just tried with this in the config:
esp32:
board: esp32-s3-devkitc-1
framework:
type: esp-idf
version: recommended
unfortunately it does not work and still no output in the logs
if you have the l/r pin on your mic to gnd then you need to define channel: left in your mic config.api:
api:
on_client_connected:
then:
- delay: 1s
- voice_assistant.start_continuous:
- delay: 1s
- lambda: id(va).set_use_wake_word(true);
- delay: 1s