I used the ESPHome yaml from the Seeed website Voice Assistant System for Home Assitant | Seeed Studio Wiki.
I’ll check out the voice-kit GitHub and see how that compares.
1 Like
Tbf, the op can do whatever they want with their thread.
2 Likes
I would argue that both ESPHome voice-kit development and other hardware using ESP32 and XMOS xCORE chips combination for audio AI acceleration is on-topic as without those these “ReSpeaker Lite” kits will in practice never be useful in the future for anyone who is not a ESPHome developer themselves.
The “bigger picture” to make all these eventually become useful to the avérage Home Assistant user they not only need to be fully supported in the upstream ESPHome project, they also need to standardize those devices in both ESPHome and the Home Assistant core, and ESHome developers are now working on several new components related to this, including a new entity component as assist_satellite platform for that which will represent a standard VoIP-based voice satellite for Home Assistant Assist voice control. Check out this architecture discussion (which sounds like it has essentially been approved)
Initial entity component for this new assist_satellite platform been merged to Home Assistant core:
Bigger picture:
- Standardize how voice satellites expose their capabilities (ie, available wake words)
- Standardize how voice satellites are configured (which wake word to listen to)
- Automate based on the state of the satellite’s pipeline
- Control the behavior of a voice satellite from HA during the setup wizard
- Skip wake word and listen for a command (with or without executing it)
- Listen for a specific wake word (without running a pipeline)
- Control a voice satellite from HA using service calls
- Announce text using the TTS portion of the satellite’s pipeline
Also see related patches with new features submitted to both ESPHome and the Home Assistant core:
2 Likes
Hi,
I’ve just received mine and I’m trying to modify the yaml to use micro wake word.
It seems to detect when I say the wake word but it stops streaming the audio just after switching to this state.
Is anyone succeeded on making it work ?
“mine” which ? ReSpeaker Lite, or one of the other similar devices mentioned through this thread ?
It would also make it easier for others to assist if you give a bit more information about your configuration (hardware and software). Eg, is the yaml in the ESPHome configuration for the device ?
1 Like
Have you tried using the experimental ”voice-kit” fork repo project from ESPHome instead of the main esphome repo?
Okay, I’m a bit confused. I already had 2 XIOS S3’s so I just ordered the board. How is the XIAO soldiered to the board? Taking a look at all the pics here and seseds website shows the XIAO sitting on the respeaker lite but I’m not seeing anything soldiered in any pictures.
They also have you flashing 2 different things, one for just the board and another for the i2s. Then on the page with instructions and YAML for ESPHome it says a Sonoff smart switch is required although outside adding it to HA, it’s not referenced anywhere.
It’s using Openeakeword with the default YAML, this is just our laziness by Seeed. I’m pretty sure the DFU driver is for when an ESP32-S3 XIAO isn’t used and using this with a PC. Then the i2s driver is able to be downloaded but the flashing instructions link takes you to a non existent page.
All in all Seeed has done a terrible job with documentaries and broken links on how to flash new firmware is just plain odd.
The frustrating part is they have an example of using a wake work using tensorflow lite but only.for Arduino…
Anyone who has the pre soldiered version could you post a pic of how it’s actually attached to the board? Below is the yaml from Seeed.
esphome:
name: esp32s3
friendly_name: ReSpeakerv3
platformio_options:
board_build.flash_mode: dio
board_build.mcu: esp32s3
# on_boot:
# then:
# - if:
# condition:
# switch.is_on:
# then:
# - voice_assistant.start_continuous:
esp32:
board: esp32-s3-devkitc-1
variant: esp32s3
framework:
type: esp-idf
version: recommended
logger:
level: VERY_VERBOSE
api:
wifi:
ssid: "Wi-Fi name"
password: "Wi-Fi password"
captive_portal:
external_components:
- source: github://QingWind6/ESPHome_XIAO-ESP32S3
i2s_audio_xiao:
i2s_lrclk_pin: GPIO7
i2s_bclk_pin: GPIO8
i2s_mclk_pin: GPIO9
microphone:
- platform: i2s_audio_xiao
id: xiao_mic
adc_type: external
i2s_din_pin: GPIO44
pdm: false
bits_per_sample: 32bit
channel: left
speaker:
- platform: i2s_audio_xiao
id: xiao_speaker
dac_type: external
i2s_dout_pin: GPIO43
mode: stereo
voice_assistant:
microphone: xiao_mic
use_wake_word: true
noise_suppression_level: 0
auto_gain: 0dBFS
volume_multiplier: 1
speaker: xiao_speaker
id: assist
on_listening:
- light.turn_on:
id: led
blue: 100%
red: 0%
green: 0%
effect: "Slow Pulse"
on_stt_vad_end:
- light.turn_on:
id: led
blue: 100%
red: 0%
green: 0%
effect: "Fast Pulse"
on_tts_start:
- light.turn_on:
id: led
blue: 100%
red: 0%
green: 0%
brightness: 100%
effect: none
on_end:
- delay: 100ms
- wait_until:
not:
speaker.is_playing:
- script.execute: reset_led
on_error:
- light.turn_on:
id: led
red: 100%
green: 0%
blue: 0%
brightness: 100%
effect: none
- delay: 1s
- script.execute: reset_led
on_client_connected:
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:
- script.execute: reset_led
on_client_disconnected:
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.stop:
- light.turn_off: led
light:
- platform: esp32_rmt_led_strip
id: led
name: None
disabled_by_default: true
entity_category: config
pin: GPIO1
default_transition_length: 0s
chipset: ws2812
num_leds: 1
rgb_order: grb
rmt_channel: 0
effects:
- pulse:
name: "Slow Pulse"
transition_length: 250ms
update_interval: 250ms
min_brightness: 50%
max_brightness: 100%
- pulse:
name: "Fast Pulse"
transition_length: 100ms
update_interval: 100ms
min_brightness: 50%
max_brightness: 100%
output:
- platform: ledc
id: light_output
pin: GPIO21
inverted: true
script:
- id: reset_led
then:
- if:
condition:
- switch.is_on: use_wake_word
- switch.is_on: use_listen_light
then:
- light.turn_on:
id: led
effect: none
else:
- light.turn_off:
id: led
switch:
- platform: template
name: Use wake word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- lambda: id(assist).set_use_wake_word(true);
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
on_turn_off:
- voice_assistant.stop
- lambda: id(assist).set_use_wake_word(false);
- platform: template
name: Use Listen Light
id: use_listen_light
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- script.execute: reset_led
on_turn_off:
- script.execute: reset_led
You’ll need hot air gun and solder paste to solder the board smd style
Using the normal 7 pin headers that used to come with any esp32 XIAO board I was able to get power to the S3 and ping it when only the respeaker lite board USB C connected. Upon further inspection, I noticed that it appears to be an issue with the code it’s pointed to in the git url. When looking at the repository, nothing has been updated in 6 to 9 months and it’s specifically saying it doesn’t like something in the git url. Even if you don’t have one you can reproduce this using the option to download the bin file… I thought this was a relatively new product? Also, how would the Adafruit QT Py fit if you had to use solder paste and hot air gun? Maybe I am missing something but plug and play with PC. Can’t get the microphones to work at all though on ESP32. Those compile errors below happen regardless if I create a new device and copy/paste seeed’s code in. Another let down from Seeed IMO that will be returned.
lite/src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.o
src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.cpp: In member function 'void esphome::i2s_audio_xiao::I2SAudioMicrophone::start_()':
src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.cpp:63:3: warning: missing initializer for member 'i2s_driver_config_t::fixed_mclk' [-Wmissing-field-initializers]
};
^
src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.cpp:63:3: warning: missing initializer for member 'i2s_driver_config_t::mclk_multiple' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.cpp:63:3: warning: missing initializer for member 'i2s_driver_config_t::chan_mask' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.cpp:63:3: warning: missing initializer for member 'i2s_driver_config_t::total_chan' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.cpp:63:3: warning: missing initializer for member 'i2s_driver_config_t::left_align' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.cpp:63:3: warning: missing initializer for member 'i2s_driver_config_t::big_edin' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.cpp:63:3: warning: missing initializer for member 'i2s_driver_config_t::bit_order_msb' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio_xiao/microphone/i2s_audio_microphone.cpp:63:3: warning: missing initializer for member 'i2s_driver_config_t::skip_msk' [-Wmissing-field-initializers]
Compiling .pioenvs/respeaker-
You asked specifically about the S3 and referenced the kit they sell, I simply gave you the information you requested. Most other platforms will access the respeaker via USB.
You can also break out the pads using short wires, solder wires onto the respeaker pads for the s3 and solder whatever you want onto the other end for testing/debugging. AFAICT this is basically a HW dev kit.
I am waiting on my hot air station, solder paste and flux before I go down the path of making it work. You will notice in previous replies that a lot of the esphome stuff is being worked on to get this product working well.
As for disappointment with docs, from their track record, you should not be surprised. End of the day, we have access to XMOS chips and multiple outfits working towards the same goal.
1 Like
I do appreciate your answer, which is 100 percent the truth. I apologize if any of my frustration seemed aimed towards you, it wasn’t, just frustration at Seeed and time wasted mostly. I do question why they put the holes on the board to fit as one of the mics is directly under the S3, which is also the side with status led.
I know the mic can probably withstand a good bit of heat for the six middle contacts on the XIOS S3/respeaker lite, but obviously I don’t want to damage it as I am not great with paste/hot air. I have only done that twice before and that middle square pad almost right where the second mic is on the other side. Thanks for the tips though as those are methods I’m more comfortable with personally.
You will need to flash different firmware to the ESP32 SoC chip and the XMOS XU316 (DSP) SoC chip. I understand it as you needing to flash the ESPHome first, and then you need to flash the XMOS chip from ESPHome via I2C DFU.
Such tasks are just one of the things that Nabu Casa and ESPHome developers are looking at simplifying for non-developers. Again, the ReSpeaker Lite with ESP32 is a development board and this is all very advanced cutting-edge stuff, but do check out their experimental voice-kit fork of ESPHome if have the skill and time.
Right now they have an open pull request to that experimental fork that will eventuelly have ESPHome automatically get XMOS firmware OTA image from a URL (via example the ESPHome addon) and flash it on boot.
1 Like
Thanks for the explanation, that makes more sense, especially in the long run…
I am glad that they are implementing the OTA/http_request functionality so non technical users who don’t want to learn/troubleshoot YAML can add devices to HA without the ESPHome add on installed and still do OTA updates with the user having zero understanding of yaml configurations. It will all be super user friendly on e it’s done.
It’s great to know how this will probably end up working.and that there are a lot of people making a huge effort to make this work as seamless as possible .
From the link you provided above one of the files is checking the DFU version via I2C to make sure it’s a valid version and will throw an error if it isn’t but not actually updating it although I’m not a developer so I could be wrong but that’s what it appears to be doing in this file and other files in that directory. If they can check the version then updating it would be possible through the ESP32 using I2C. Once again, I know it’s a work in progress. I also didn’t go through every file so it may be doing the DFU firmware update over I2C already.
dfu-util is just a CLI utility to update device firmware via USB so Seeed probably tells you to use it because it exists already and is free. It will be nice to see this functionality in ESPHome eventually . I did find the Arduino sketch to stream audio via MQTT informative and possibly useful to do in HA in specific use cases. The below is from seeeds docs, which is what I was going by.
Flash the I2S firmware
Before we start this chapter, please make sure the ReSpeaker Lite’s firmware is the I2S version.I2S Firmware Download
Check Get Started to flash the firmware.Software Preparation
Install Arduino IDE
Download and Install the stable version of Arduino IDE
Sorry my notification was not set and I forgot to check the thread. I’ve received the respeaker lite with Xiao esp32 s3.
Mic seems to work as it waits for wake word but once wake word is detected, the esp32 crash