"ReSpeaker Lite" - new Seeed Studio Voice Assistant Development Kit hardware combine ESP32 with XMOS XU316 DSP chip for advanced audio processing as a ESPHome-based Home Assistant Assist Satellite voice devkit

Oh yeah, Onju is great.
We had conversation with Tetele on Discord couple days ago, there’s trouble running new components (like PE software) on Onju. Although there’s full-duplex I2S (like in Respeaker Lite), i guess Tetele doesn’t want to use basically experimental support (like we do, with PE repo) and will wait for that components to be merged into ESPHome upstream repo.

Also yeah, on Discord there are many requests to do something for Echo dots. I don’t know if it’s something to be done though - too many hardware revisions there, with different guts…

1 Like

Thanks. I tested mine with Hey Mycroft and it works like before and it recognizes the wake word no matter how far I am away from the voice assistant. So it looks like that Okay Nabu and Hey Jarvis wake words got impacted by one of the recent updates?

There’s two things that got updated:

  • ok Nabu model in VPE firmware
  • microphone implementation in DFU firmware

Both could bring regression. As I said, I don’t see it - but I might be wrong.

Just updated Respeaker Lite ESPHome config to 2025.3.0:

  • LED effects now take brightness from user LED light setting ;
  • user LED doesn’t get turned off after every effect;
  • lowered VAD model probability cut-off to improve wake word detection.

Updated Koala with same probability cut-off, and added more granular volume control.

Also both configs got fix for media player entity ID (which was screwed by PE firmware, now it’s reverted), and also new build will be able to support continuous conversation (which isn’t yet released by HA, but PE is ready for it, and so are we now :slight_smile: ).

If you experience a lot of false-activations on this config, try tuning probability_cutoff up for vad: section. It is 0.5 by default, but i found that if i talk with subtle voice, VAD is cutting me off very often.

4 Likes

Hi everyone, I recently purchased Respeaker lite kit. Everything works, except that I have serious problems with the activation word; specifically ok nabu and ok microft always answer, while Hey jarvis answers once out of 10 if all goes well, sometimes nothing at all. I would like to have if possible and know if there is a solution to this and what could be causing it. Thanks everyone!

You may try to reduce probability_cutoff on that. It may introduce more false-positives, but will decrease false-negatives.

“Hey, Jarvis” wake word is trained on synthetic data set exclusively, so isn’t working great even with slight accents…

I was trying to update the yaml, but for some reason it errors out half way through the installation process with the following error:
I tried to restore the old yaml, but still getting the same error. Has anyone seen this before?

/data/cache/platformio/packages/framework-espidf/components/app_trace/app_trace.c:452:1: internal compiler error: Segmentation fault
  452 | }
      | ^
0x138fee6 internal_error(char const*, ...)
	???:0
In file included from src/esphome/components/i2s_audio/i2s_audio.h:5,
                 from src/esphome.h:44,
                 from src/main.cpp:3:
/data/cache/platformio/packages/framework-espidf/components/driver/deprecated/driver/i2s.h:27:2: warning: #warning "This set of I2S APIs has been deprecated, please include 'driver/i2s_std.h', 'driver/i2s_pdm.h' or 'driver/i2s_tdm.h' instead. if you want to keep using the old APIs and ignore this warning, you can enable 'Suppress leagcy driver deprecated warning' option under 'I2S Configuration' menu in Kconfig" [-Wcpp]
   27 | #warning "This set of I2S APIs has been deprecated, \
      |  ^~~~~~~
xtensa-esp32s3-elf-gcc: internal compiler error: Segmentation fault signal terminated program cc1
Please submit a full bug report, with preprocessed source (by using -freport-bug).
See <https://gcc.gnu.org/bugs/> for instructions.
*** [.pioenvs/respeaker-satellite-parentroom/app_trace/app_trace.c.o] Error 4
In file included from components/esp-tflite-micro/tensorflow/lite/micro/tflite_bridge/flatbuffer_conversions_bridge.h:19,
                 from components/esp-tflite-micro/tensorflow/lite/micro/micro_allocator.h:26,
                 from components/esp-tflite-micro/tensorflow/lite/micro/micro_interpreter.h:26,
                 from src/esphome/components/micro_wake_word/streaming_model.h:10,
                 from src/esphome/components/micro_wake_word/micro_wake_word.h:6,
                 from src/esphome.h:71:
components/esp-tflite-micro/tensorflow/lite/core/api/flatbuffer_conversions.h: In member function 'T* tflite::BuiltinDataAllocator::AllocatePOD()':
components/esp-tflite-micro/tensorflow/lite/core/api/flatbuffer_conversions.h:46:24: warning: 'template<class _Tp> struct std::is_pod' is deprecated: use is_standard_layout && is_trivial instead [-Wdeprecated-declarations]
   46 |     static_assert(std::is_pod<T>::value, "Builtin data structure must be POD.");
      |                        ^~~~~~
In file included from /data/cache/platformio/packages/toolchain-xtensa-esp32s3/xtensa-esp32s3-elf/include/c++/12.2.0/bits/char_traits.h:42,
                 from /data/cache/platformio/packages/toolchain-xtensa-esp32s3/xtensa-esp32s3-elf/include/c++/12.2.0/string:40,
                 from src/esphome/core/hal.h:2,
                 from src/esphome/components/audio_dac/audio_dac.h:4,
                 from src/esphome/components/aic3204/aic3204.h:3,

Segfault error?.. Hmm, can you clean build data?

I just cleaned build data and tried again, but still getting the same segmentation error.

1 Like

It’s in /data/cache/platformio/packages/framework-espidf/components/app_trace/app_trace.c:452:1

It’s Platformio cache. I doubt I can help…
I’ve just built latest config on ESPHome 2025.2.2, everything works.

“Or even go full bananas and use it without dedicated LED strip (top LED should work okay) - just adjust LED diffuser color to your like.”
I have the SeeedStudio ReSpeaker Lite with Soldered on SeeedStudio ESP32S3 if the RGB isn’t working but everything else is what could be wrong? I did solder USR+D3 & MUTE+D2 but I have zero RGB - red mute LED works.
Ideas?

If you use Respeaker with soldered XIAO S3, you will have better luck with this repo: GitHub - formatBCE/Respeaker-Lite-ESPHome-integration. It’s created for that configuration. Koala is too customized.

1 Like

Trying to get ReSpeaker Lite work with repo above but some issues. In yaml error on line 192 : 192 | … turn id(mute_switch_on_sound); 'unknown tag ! (192:69)
'. Clean Build but a number of warnings during compilation but compiles and sensors create in Home Assistant. Buttons seems to work but no response to ’ Hey Nabu ’ at all. Does anyone have a clue how to start debugging ? Is there a very basic setup to try, confirming hardware works as expected ? Any help appreciated.

It’s “ok Nabu” :slight_smile:

Check logs if they have something about detecting wake word.

Also, try pushing the action button and saying command without wake word - what will it do?

And check the firmware version on XMOS board (is it 1.0.9?)

It was indeed not related to the new yaml code. The reason I thought it was, is because the installation process worked fine for my other ESPHome device. After checking the possible reasons for this issue, I found out that it’s a hardware (memorey) related issue. So I was lucky to get it fixed by just shutting down the HASS server and removing the power for some time and then restarted it. I wanted to share this here as it’s somehow related to devices with big yaml just in case someone will have the same issue in the future.

1 Like

Okay thank you!

@formatBCE I see there are 2 options for the ESP32-S3 board an 8MB flash one and a 16MB flash any dissadvantages to usign the 8MB? Here in Canada 16MB stock is limited USA export is expensive. One disadvantage is I see most YAML files all say 16MB. Thanks for the help and amazing job you’ve done contributing to this project! My 3D Printer is sweating because of you.

1 Like

You’re welcome! :slight_smile:

Well, soldered-on XIAO S3 is N8R8, so it has 8MB flash.
As soon as the firmware fits the flash size (well, divided by 2, because this is how ESPHome OTA works…), you’re fine using N8R8.

I also used N8R2 (with 2MB PSRAM), and while it also boots, it’s a bit unstable when playing music or sounding long TTS messages, so I don’t recommend that. But as soon as you have enough PSRAM, you’re good.

Thank’s for reply. I know wakeword is ‘ok nabu’, misspelled from my part. Neither of nabu,mycroft or jarvis gives any respond. I cant see from log what’s the problem, can you ?

INFO ESPHome 2025.3.0
INFO Reading configuration /config/esphome/my-voice-assistant.yaml...
INFO Updating https://github.com/esphome/home-assistant-voice-pe@dev
INFO Updating https://github.com/formatBCE/home-assistant-voice-pe@48kHz_mic_support
INFO Updating https://github.com/formatBCE/Respeaker-Lite-ESPHome-integration@main
WARNING GPIO3 is a strapping PIN and should only be used for I/O with care.
Attaching external pullup/down resistors to strapping pins can cause unexpected failures.
See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins
INFO Starting log output from 192.168.1.94 using esphome API
INFO Successfully connected to respeaker-satellite @ 192.168.1.94 in 0.029s
INFO Successful handshake with respeaker-satellite @ 192.168.1.94 in 0.018s
[17:08:54][I][app:100]: ESPHome version 2025.3.0 compiled on Mar 20 2025, 16:49:13
[17:08:54][I][app:102]: Project formatbce.Respeaker Lite Satellite version 2025.3.0
[17:08:54][C][wifi:600]: WiFi:
[17:08:54][C][wifi:428]:   Local MAC: 98:3D:AE:EE:8A:50
[17:08:54][C][wifi:433]:   SSID: 'wireless'[redacted]
[17:08:54][C][wifi:436]:   IP Address: 192.168.1.94
[17:08:54][C][wifi:439]:   BSSID: 60:22:32:60:2E:52[redacted]
[17:08:54][C][wifi:441]:   Hostname: 'respeaker-satellite'
[17:08:54][C][wifi:443]:   Signal strength: -70 dB ▂▄▆█
[17:08:54][C][wifi:447]:   Channel: 6
[17:08:54][C][wifi:448]:   Subnet: 255.255.255.0
[17:08:54][C][wifi:449]:   Gateway: 192.168.1.1
[17:08:54][C][wifi:450]:   DNS1: 192.168.1.2
[17:08:54][C][wifi:451]:   DNS2: 8.8.8.8
[17:08:54][C][logger:177]: Logger:
[17:08:54][C][logger:178]:   Max Level: DEBUG
[17:08:54][C][logger:179]:   Initial Level: DEBUG
[17:08:54][C][logger:181]:   Log Baud Rate: 115200
[17:08:54][C][logger:182]:   Hardware UART: USB_SERIAL_JTAG
[17:08:54][C][logger:186]:   Level for 'sensor': WARN
[17:08:54][C][i2c.idf:083]: I2C Bus:
[17:08:54][C][i2c.idf:084]:   SDA Pin: GPIO5
[17:08:54][C][i2c.idf:085]:   SCL Pin: GPIO6
[17:08:54][C][i2c.idf:086]:   Frequency: 400000 Hz
[17:08:54][C][i2c.idf:092]:   Recovery: bus successfully recovered
[17:08:54][I][i2c.idf:102]: Results from i2c bus scan:
[17:08:54][I][i2c.idf:108]: Found i2c device at address 0x18
[17:08:54][I][i2c.idf:108]: Found i2c device at address 0x42
[17:08:54][C][switch.gpio:068]: GPIO Switch 'mute_toggle'
[17:08:54][C][switch.gpio:090]:   Restore Mode: always OFF
[17:08:54][C][switch.gpio:031]:   Pin: GPIO4
[17:08:54][C][esp32_rmt_led_strip:250]: ESP32 RMT LED Strip:
[17:08:54][C][esp32_rmt_led_strip:251]:   Pin: 1
[17:08:54][C][esp32_rmt_led_strip:253]:   RMT Symbols: 192
[17:08:54][C][esp32_rmt_led_strip:281]:   RGB Order: GRB
[17:08:54][C][esp32_rmt_led_strip:282]:   Max refresh rate: 0
[17:08:54][C][esp32_rmt_led_strip:283]:   Number of LEDs: 1
[17:08:54][C][template.sensor:022]: Template Sensor 'Next timer'
[17:08:54][C][template.sensor:022]:   Device Class: 'duration'
[17:08:54][C][template.sensor:022]:   State Class: ''
[17:08:54][C][template.sensor:022]:   Unit of Measurement: 's'
[17:08:54][C][template.sensor:022]:   Accuracy Decimals: 1
[17:08:54][C][template.sensor:022]:   Icon: 'mdi:timer'
[17:08:54][C][template.sensor:023]:   Update Interval: never
[17:08:54][C][template.text_sensor:020]: Template Sensor 'Next timer name'
[17:08:54][C][template.text_sensor:020]:   Icon: 'mdi:timer'
[17:08:54][C][template.text_sensor:020]: Template Sensor 'TTS URI'
[17:08:54][C][gpio.binary_sensor:015]: GPIO Binary Sensor 'User button'
[17:08:54][C][gpio.binary_sensor:016]:   Pin: GPIO3
[17:08:54][C][light:092]: Light 'led_internal'
[17:08:54][C][light:094]:   Default Transition Length: 0.0s
[17:08:54][C][light:095]:   Gamma Correct: 2.80
[17:08:54][C][light:092]: Light 'LED Respeaker onboard'
[17:08:54][C][light:094]:   Default Transition Length: 0.0s
[17:08:54][C][light:095]:   Gamma Correct: 2.80
[17:08:54][C][respeaker_lite:042]: Respeaker Lite:
[17:08:54][C][respeaker_lite:043]:   Address: 0x42
[17:08:54][C][respeaker_lite:044]:   Reset Pin: GPIO2
[17:08:54][C][respeaker_lite:046]:   XMOS firmware version: 1.1.0
[17:08:54][C][template.switch:068]: Template Switch 'Speaker mute'
[17:08:54][C][template.switch:070]:   Icon: 'mdi:volume-mute'
[17:08:54][C][template.switch:090]:   Restore Mode: always OFF
[17:08:54][C][template.switch:057]:   Optimistic: YES
[17:08:54][C][template.switch:068]: Template Switch 'Button click sounds'
[17:08:54][C][template.switch:070]:   Icon: 'mdi:bullhorn'
[17:08:54][C][template.switch:090]:   Restore Mode: restore defaults to ON
[17:08:54][C][template.switch:057]:   Optimistic: YES
[17:08:54][C][template.switch:068]: Template Switch 'Mute/unmute sound'
[17:08:54][C][template.switch:070]:   Icon: 'mdi:bullhorn'
[17:08:54][C][template.switch:090]:   Restore Mode: restore defaults to ON
[17:08:54][C][template.switch:057]:   Optimistic: YES
[17:08:54][C][template.switch:068]: Template Switch 'Wake sound'
[17:08:54][C][template.switch:070]:   Icon: 'mdi:bullhorn'
[17:08:54][C][template.switch:090]:   Restore Mode: restore defaults to ON
[17:08:54][C][template.switch:057]:   Optimistic: YES
[17:08:54][C][template.switch:068]: Template Switch 'timer_ringing'
[17:08:54][C][template.switch:090]:   Restore Mode: always OFF
[17:08:54][C][template.switch:057]:   Optimistic: YES
[17:08:54][C][template.switch:068]: Template Switch 'Mic mute'
[17:08:54][C][template.switch:070]:   Icon: 'mdi:microphone-off'
[17:08:54][C][template.switch:090]:   Restore Mode: always OFF
[17:08:54][C][template.switch:057]:   Optimistic: NO
[17:08:54][C][psram:020]: PSRAM:
[17:08:54][C][psram:021]:   Available: YES
[17:08:54][C][psram:028]:   Size: 8192 KB
[17:08:54][C][speaker_mixer:292]: Speaker Mixer:
[17:08:54][C][speaker_mixer:293]:   Number of output channels: 2
[17:08:54][C][speaker_mixer:046]: Mixer Source Speaker
[17:08:54][C][speaker_mixer:047]:   Buffer Duration: 100 ms
[17:08:54][C][speaker_mixer:051]:   Timeout: never
[17:08:54][C][speaker_mixer:046]: Mixer Source Speaker
[17:08:54][C][speaker_mixer:047]:   Buffer Duration: 100 ms
[17:08:54][C][speaker_mixer:051]:   Timeout: never
[17:08:54][C][aic3204:112]: AIC3204:
[17:08:54][C][aic3204:113]:   Address: 0x18
[17:08:54][C][factory_reset.button:011]: Factory Reset Button 'Factory Reset'
[17:08:54][C][factory_reset.button:011]:   Icon: 'mdi:restart-alert'
[17:08:54][C][restart.button:017]: Restart Button 'Restart'
[17:08:55][C][restart.button:017]:   Icon: 'mdi:restart'
[17:08:55][C][mdns:116]: mDNS:
[17:08:55][C][mdns:117]:   Hostname: respeaker-satellite
[17:08:55][C][esphome.ota:073]: Over-The-Air updates:
[17:08:55][C][esphome.ota:074]:   Address: respeaker-satellite.local:3232
[17:08:55][C][esphome.ota:075]:   Version: 2
[17:08:55][C][esphome.ota:078]:   Password configured
[17:08:55][C][safe_mode:018]: Safe Mode:
[17:08:55][C][safe_mode:019]:   Boot considered successful after 60 seconds
[17:08:55][C][safe_mode:021]:   Invoke after 10 boot attempts
[17:08:55][C][safe_mode:022]:   Remain in safe mode for 300 seconds
[17:08:55][C][api:140]: API Server:
[17:08:55][C][api:141]:   Address: respeaker-satellite.local:6053
[17:08:55][C][api:145]:   Using noise encryption: NO
[17:08:55][C][micro_wake_word:072]: microWakeWord:
[17:08:55][C][micro_wake_word:073]:   models:
[17:08:55][C][micro_wake_word:015]:     - Wake Word: Okay Nabu
[17:08:55][C][micro_wake_word:016]:       Probability cutoff: 0.85
[17:08:55][C][micro_wake_word:017]:       Sliding window size: 5
[17:08:55][C][micro_wake_word:015]:     - Wake Word: Hey Jarvis
[17:08:55][C][micro_wake_word:016]:       Probability cutoff: 0.97
[17:08:55][C][micro_wake_word:017]:       Sliding window size: 5
[17:08:55][C][micro_wake_word:015]:     - Wake Word: Hey Mycroft
[17:08:55][C][micro_wake_word:016]:       Probability cutoff: 0.95
[17:08:55][C][micro_wake_word:017]:       Sliding window size: 5
[17:08:55][C][micro_wake_word:015]:     - Wake Word: Stop
[17:08:55][C][micro_wake_word:016]:       Probability cutoff: 0.50
[17:08:55][C][micro_wake_word:017]:       Sliding window size: 5
[17:08:55][C][micro_wake_word:021]:     - VAD Model
[17:08:55][C][micro_wake_word:022]:       Probability cutoff: 0.05
[17:08:55][C][micro_wake_word:023]:       Sliding window size: 5
[17:08:55][C][debug:021]: Debug component:
[17:08:55][D][debug:035]: ESPHome version 2025.3.0
[17:08:55][D][debug:039]: Free Heap Size: 113000 bytes
[17:08:55][D][debug:284]: Chip: Model=ESP32-S3, Features=WIFI_BGN,BLE, Cores=2, Revision=2
[17:08:55][D][debug:299]: Framework: ESP-IDF
[17:08:55][D][debug:306]: ESP-IDF Version: 5.1.5
[17:08:55][D][debug:311]: EFuse MAC: 98:3D:AE:EE:8A:50
[17:08:55][D][debug:204]: Reset Reason: Software reset via esp_restart
[17:08:55][D][debug:359]: Wakeup Reason: Unknown
[17:08:55][C][debug:033]: Partition table:
[17:08:55][C][debug:034]:   Name         Type Subtype  Address    Size      
[17:08:55][C][debug:038]:   otadata      1    0        0x00009000 0x00002000
[17:08:55][C][debug:038]:   phy_init     1    1        0x0000B000 0x00001000
[17:08:55][C][debug:038]:   app0         0    16       0x00010000 0x003C0000
[17:08:55][C][debug:038]:   app1         0    17       0x003D0000 0x003C0000
[17:08:55][C][debug:038]:   nvs          1    2        0x00790000 0x0006D000
[17:09:03][I][safe_mode:041]: Boot seems successful; resetting boot loop counter
[17:09:03][D][esp32.preferences:114]: Saving 1 preferences to flash...
[17:09:03][D][esp32.preferences:142]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed

Btw error in yaml that I mentioned is it more of ‘cosmetic’ nature or does it matter ?

What hardware are you using?