"ReSpeaker Lite" - new Seeed Studio Voice Assistant Development Kit hardware combine ESP32 with XMOS XU316 DSP chip for advanced audio processing as a ESPHome-based Home Assistant Assist Satellite voice devkit

KevinA · October 26, 2024, 5:51pm

We configure the aic3204 DAC in ESPHome code for a 48 kHz sample rate. Since the Respeaker board runs at 16 kHz, the various filters on the DAC are most likely introducing noise (we ran into a similar issue when we switched from 16 kHz to 48 kHz…).

We’ll have to make the aic3204 component work for different sample rates to avoid this. It’s definitely possible, especially if we limit it to just the 16 kHz or 48 kHz cases. Configuring the DAC correctly is a huge pain, as there are 4 settings that depend on each other and the sample rate, so adjusting it to make it work for any sample rate would be quite tedious.

AndyCap · October 26, 2024, 5:59pm

Hi Kevin,

Thanks for the info.

Has this changed recently then?

Also why would this only effect the headphone out and not the speaker out?

KevinA · October 26, 2024, 6:14pm

The audio DAC code hasn’t changed recently, the aic3204 component has been configured for 48 kHz from the start.

There may be a different issue, as it is odd there is a difference between the internal speaker and the headphone out port. When we changed to 48 kHz but didn’t update the DAC configuration, we found that the audio through our headphone port was very flat and not of great quality. So I’m not 100% sure its the cause in your situation, but since it works fine with Seeed’s code and not with ours, its most likely due to the DAC misconfiguration.

AndyCap · October 26, 2024, 7:20pm

Thanks for all the info.

What I am unsure of though is why when I built it all before the headphone out was fine, and now it isn’t

There is always the chance I’m messing something up, it has been known to happen before.

It would be good to see if anyone else is seeing this from the headphone out.

formatBCE · October 26, 2024, 8:26pm

That might be because before I didn’t explicitly use sic3204 in Nabu media player, so probably that was going through the Respeaker software resampler?
Try commenting out audio_dac: code in YAML and reinstall.

Also, Seeed right now works on 48kHz firmware, I convinced them that S3 is powerful enough to support it.

AndyCap · October 27, 2024, 5:30am

I will have a little play today but I have been thinking about it and I think you guys are right, it must be to do with the AIC3204 setup.

For my test code (esp-idf) all I am doing is setting up the i2s config, and a few i2c functions (Headphone amp, speaker amp and digital volume). I am not setting up the AIC3204 in any other way.

So I’m guessing the Xmos firmware must be setting up the AIC3204 via i2c to its preferred state otherwise my code would not work.

I’m a bit confused why this is different now though as there have been no recent changes, maybe some timing issue and they are stepping over each other at the same time via i2c and the setup is getting corrupted?

AndyCap · October 27, 2024, 6:59am

Removing the aic3204 and audio_dac from the YAML doesn’t change anything, there goes that idea!

Tested with 1.0.7 and 1.0.9

formatBCE · October 27, 2024, 8:38am

Okay then, it must be some glitch? I have no idea .

AndyCap · October 27, 2024, 6:35pm

Yep, very strange. Is your headphone out ok?

formatBCE · October 27, 2024, 7:35pm

I tried several times, I can’t hear noise… Sound is clear.
Tried with headphones and line-out to active speaker. Quality isn’t good, but it’s 16kHz all in all…

AndyCap · October 27, 2024, 7:51pm

Thanks, could you by any chance post your esp32 firmware file so I can flash it here.

formatBCE · October 27, 2024, 7:54pm

It won’t work, right? It has my WiFi and stuff.

You can compile same from my repo. And make sure you have 1.0.9 DFU (check it thrice, there was a lot of false troubles because it didn’t actually flash, especially on Windows…

AndyCap · October 27, 2024, 7:57pm

Good point

I am using your repo, I definitely have 1.0.9, I’m just wondering if it is a build issue here.

I will remove all platformios, uninstall esphome and re-install it. Maybe that will help!

formatBCE · October 27, 2024, 8:19pm

Is that noise you’re talking about very noticable? Can you make a video with it? So I could compare? Probably I’m just too old or deaf to hear that?

AndyCap · October 28, 2024, 7:52am

It’s very noticeable.

I’ll get a recording for you in a bit…

AndyCap · October 28, 2024, 8:29am

Here are two recordings of me asking for the time, so noise floor first, then broadband noise, then ‘silence’ with audio corruption and answer. Volume is at 80% but the volume doesn’t have any effect on the noise, just the audio playing. So with volume at 0% you would get exactly the same noise but no voice.

First the output recorded at -10dBV: Dropbox

Then the file normalised: Dropbox

AndyCap · October 28, 2024, 9:09am

So I deleted all my esp home stuff and .platformio from my user folder.

I created a new venv, installed wheel, setuptools and ESPHome.

I cloned your repo and added a secrets.yaml. I get exactly the same noise problems

I couldn’t post the entire log, so I deleted the compile lines

INFO ESPHome 2024.10.2
INFO Reading configuration esphome-respeaker-lite.yaml...
INFO Cloning https://github.com/esphome/voice-kit@dev
INFO Fetching dev
INFO Cloning https://github.com/esphome/esphome.git@pull/7605/head
INFO Fetching pull/7605/head
INFO Cloning https://github.com/formatBCE/Respeaker-Lite-ESPHome-integration@main
INFO Fetching main
WARNING GPIO3 is a strapping PIN and should only be used for I/O with care.
Attaching external pullup/down resistors to strapping pins can cause unexpected failures.
See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins
INFO Generating C++ source...
INFO Core config, version or integrations changed, cleaning build files...
INFO Cloning https://github.com/espressif/[email protected]
INFO Fetching v1.3.1
INFO Compiling app...
Processing respeaker-satellite (board: esp32-s3-devkitc-1; framework: espidf; platform: platformio/[email protected])
---------------------------------------------------------------------------------------------------------------------------------------------------
Library Manager: Installing esphome/esp-audio-libs @ 1.0.0
INFO Installing esphome/esp-audio-libs @ 1.0.0
Unpacking  [####################################]  100%
Library Manager: [email protected] has been installed!
INFO [email protected] has been installed!
Library Manager: Installing kahrendt/ESPMicroSpeechFeatures @ 1.1.0
INFO Installing kahrendt/ESPMicroSpeechFeatures @ 1.1.0
Unpacking  [####################################]  100%
Library Manager: [email protected] has been installed!
INFO [email protected] has been installed!
HARDWARE: ESP32S3 240MHz, 320KB RAM, 8MB Flash
 - framework-espidf @ 3.40408.0 (4.4.8) 
 - tool-cmake @ 3.16.4 
 - tool-ninja @ 1.9.0 
 - toolchain-esp32ulp @ 2.35.0-20220830 
 - toolchain-riscv32-esp @ 8.4.0+2021r2-patch5 
 - toolchain-xtensa-esp32s3 @ 8.4.0+2021r2-patch5
Reading CMake configuration...
Generating assembly for certificate bundle...
Dependency Graph
|-- esp-audio-libs @ 1.0.0
|-- ESPMicroSpeechFeatures @ 1.1.0
Compiling .pioenvs/respeaker-satellite/src/esphome/components/api/api_connection.o
...
...
Archiving .pioenvs/respeaker-satellite/lib9c1/libESPMicroSpeechFeatures.a
Indexing .pioenvs/respeaker-satellite/lib9c1/libESPMicroSpeechFeatures.a
Archiving .pioenvs/respeaker-satellite/libc4c/libesp-audio-libs.a
Indexing .pioenvs/respeaker-satellite/libc4c/libesp-audio-libs.a
Linking .pioenvs/respeaker-satellite/firmware.elf
RAM:   [=         ]  10.7% (used 35116 bytes from 327680 bytes)
Flash: [====      ]  35.3% (used 1387449 bytes from 3932160 bytes)
Building .pioenvs/respeaker-satellite/firmware.bin
Creating esp32s3 image...
Successfully created esp32s3 image.
esp32_create_combined_bin([".pioenvs/respeaker-satellite/firmware.bin"], [".pioenvs/respeaker-satellite/firmware.elf"])
Wrote 0x162d20 bytes to file /Volumes/X82/Development/esp/esphome/Respeaker-Lite-ESPHome-integration/.esphome/build/respeaker-satellite/.pioenvs/respeaker-satellite/firmware.factory.bin, ready to flash to offset 0x0
esp32_copy_ota_bin([".pioenvs/respeaker-satellite/firmware.bin"], [".pioenvs/respeaker-satellite/firmware.elf"])
========================================================== [SUCCESS] Took 126.20 seconds ==========================================================
INFO Successfully compiled program.
Found multiple options for uploading, please choose one:
  [1] /dev/cu.usbmodem1442301 (USB JTAG/serial debug unit)
  [2] Over The Air (respeaker-satellite.local)
(number): 1
esptool.py v4.7.0
Serial port /dev/cu.usbmodem1442301
Connecting...
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
MAC: 64:e8:33:7f:81:94
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 8MB
Flash will be erased from 0x00010000 to 0x00162fff...
Flash will be erased from 0x00000000 to 0x00005fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x00009000 to 0x0000afff...
Compressed 1387808 bytes to 903995...
Wrote 1387808 bytes (903995 compressed) at 0x00010000 in 9.4 seconds (effective 1177.8 kbit/s)...
Hash of data verified.
Compressed 20848 bytes to 13063...
Wrote 20848 bytes (13063 compressed) at 0x00000000 in 0.3 seconds (effective 481.4 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 134...
Wrote 3072 bytes (134 compressed) at 0x00008000 in 0.0 seconds (effective 514.2 kbit/s)...
Hash of data verified.
Compressed 8192 bytes to 31...
Wrote 8192 bytes (31 compressed) at 0x00009000 in 0.1 seconds (effective 781.9 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
INFO Successfully uploaded program.
INFO Starting log output from /dev/cu.usbmodem1442301 with baud rate 115200
[08:59:13]I (107) esp_image: segment 0: paddr=00010020 vaddr=3c0e0020 size=612c0h (398016) map
[08:59:13]I (187) esp_image: segment 1: paddr=000712e8 vaddr=3fc9ca70 size=03fc4h ( 16324) load
[08:59:13]I (191) esp_image: segment 2: paddr=000752b4 vaddr=40378000 size=0ad64h ( 44388) load
[08:59:13]I (203) esp_image: segment 3: paddr=00080020 vaddr=42000020 size=d8fd0h (888784) map
[08:59:13]I (363) esp_image: segment 4: paddr=00158ff8 vaddr=40382d64 size=09d04h ( 40196) load
[08:59:13]I (380) boot: Loaded app from partition at offset 0x10000
[08:59:13]I (409) boot: Set actual ota_seq=1 in otadata[0]
[08:59:13]I (409) boot: Disabling RNG early entropy source...
[08:59:13]I (409) cpu_start: Multicore app
[08:59:13]I (413) opi psram: vendor id : 0x0d (AP)
[08:59:13]I (417) opi psram: dev id    : 0x02 (generation 3)
[08:59:13]I (422) opi psram: density   : 0x03 (64 Mbit)
[08:59:13]I (427) opi psram: good-die  : 0x01 (Pass)
[08:59:13]I (432) opi psram: Latency   : 0x01 (Fixed)
[08:59:13]I (437) opi psram: VCC       : 0x01 (3V)
[08:59:13]I (442) opi psram: SRF       : 0x01 (Fast Refresh)
[08:59:13]I (447) opi psram: BurstType : 0x01 (Hybrid Wrap)
[08:59:13]I (452) opi psram: BurstLen  : 0x01 (32 Byte)
[08:59:13]I (457) opi psram: Readlatency  : 0x02 (10 cycles@Fixed)
[08:59:13]I (463) opi psram: DriveStrength: 0x00 (1/1)
[08:59:13]I (469) MSPI Timing: PSRAM timing tuning index: 4
[08:59:13]I (474) spiram: Found 64MBit SPI RAM device
[08:59:13]I (479) spiram: SPI RAM mode: sram 80m
[08:59:13]I (483) spiram: PSRAM initialized, cache is in normal (1-core) mode.
[08:59:13]I (490) cpu_start: Pro cpu up.
[08:59:13]I (494) cpu_start: Starting app cpu, entry point is 0x4037a9e0
[08:59:13]I (0) cpu_start: App cpu up.
[08:59:14]I (792) spiram: SPI SRAM memory test OK
[08:59:14]I (801) cpu_start: Pro cpu start user code
[08:59:14]I (801) cpu_start: cpu freq: 240000000
[08:59:14]I (801) cpu_start: Application information:
[08:59:14]I (801) cpu_start: Project name:     respeaker-satellite
[08:59:14]I (802) cpu_start: App version:      2024.10.2
[08:59:14]I (802) cpu_start: Compile time:     Oct 28 2024 08:57:14
[08:59:14]I (802) cpu_start: ELF file SHA256:  47215019686bf34d...
[08:59:14]I (803) cpu_start: ESP-IDF:          4.4.8
[08:59:14]I (803) cpu_start: Min chip rev:     v0.0
[08:59:14]I (803) cpu_start: Max chip rev:     v0.99 
[08:59:14]I (803) cpu_start: Chip rev:         v0.2
[08:59:14]I (803) heap_init: Initializing. RAM available for dynamic allocation:
[08:59:14]I (804) heap_init: At 3FCA53A0 len 00044370 (272 KiB): D/IRAM
[08:59:14]I (804) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DIRAM
[08:59:14]I (804) heap_init: At 600FE000 len 00002000 (8 KiB): RTCRAM
[08:59:14]I (805) spiram: Adding pool of 8192K of external SPI memory to heap allocator
[08:59:14]I (805) spi_flash: detected chip: gd
[08:59:14]I (806) spi_flash: flash io: dio
[08:59:14]I (807) sleep: Configure to isolate all GPIO pins in sleep state
[08:59:14]I (807) sleep: Enable automatic switching of GPIO sleep configuration
[08:59:14]I (808) cpu_start: Starting scheduler on PRO CPU.
[08:59:14]I (0) cpu_start: Starting scheduler on APP CPU.
[08:59:14][I][logger:156]: Log initialized
[08:59:14][C][safe_mode:079]: There have been 0 suspected unsuccessful boot attempts
[08:59:14][D][esp32.preferences:114]: Saving 1 preferences to flash...
[08:59:14][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed
[08:59:14][I][app:029]: Running through setup()...
[08:59:14][C][i2c.idf:017]: Setting up I2C bus...
[08:59:14][I][i2c.idf:252]: Performing I2C bus recovery
[08:59:14][D][esp-idf:000]: I (134) gpio: GPIO[6]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0 
[08:59:14]
[08:59:14][D][esp-idf:000]: I (135) gpio: GPIO[5]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0 
[08:59:14]
[08:59:14][C][switch.gpio:011]: Setting up GPIO Switch 'mute_toggle'...
[08:59:14][D][switch:016]: 'mute_toggle' Turning OFF.
[08:59:14][D][switch:055]: 'mute_toggle': Sending state OFF
[08:59:14][D][esp-idf:000]: I (165) gpio: GPIO[4]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
[08:59:14]
[08:59:14][D][switch:016]: 'mute_toggle' Turning OFF.
[08:59:14][C][esp32_rmt_led_strip:021]: Setting up ESP32 LED Strip...
[08:59:14][D][esp-idf:000]: I (168) gpio: GPIO[3]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
[08:59:14]
[08:59:14][D][binary_sensor:034]: 'User button': Sending initial state ON
[08:59:14][C][light:035]: Setting up light 'respeaker-satellite'...
[08:59:14][D][light:036]: 'respeaker-satellite' Setting:
[08:59:14][D][light:041]:   Color mode: RGB
[08:59:14][D][template.switch:046]:   Restored state OFF
[08:59:14][D][switch:016]: 'speaker_mute_switch' Turning OFF.
[08:59:14][D][switch:055]: 'speaker_mute_switch': Sending state OFF
[08:59:14][D][template.switch:046]:   Restored state OFF
[08:59:14][D][switch:016]: 'Mic mute' Turning OFF.
[08:59:14][D][switch:012]: 'mute_toggle' Turning ON.
[08:59:14][D][switch:055]: 'mute_toggle': Sending state ON
[08:59:14][D][template.switch:046]:   Restored state OFF
[08:59:14][D][switch:016]: 'timer_ringing' Turning OFF.
[08:59:14][D][switch:055]: 'timer_ringing': Sending state OFF
[08:59:14][D][light:036]: 'respeaker-satellite' Setting:
[08:59:14][D][light:047]:   State: ON
[08:59:14][D][light:051]:   Brightness: 30%
[08:59:14][D][light:059]:   Red: 100%, Green: 100%, Blue: 0%
[08:59:14][D][light:109]:   Effect: 'Slow Pulse'
[08:59:14][D][light:036]: 'respeaker-satellite' Setting:
[08:59:14][D][light:051]:   Brightness: 30%
[08:59:14][D][light:059]:   Red: 100%, Green: 100%, Blue: 0%
[08:59:14][I][respeaker_lite:013]: Setting up RespeakerLite...
[08:59:14][D][esp-idf:000]: I (183) gpio: GPIO[2]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
[08:59:14]
[08:59:14][C][i2s_audio:028]: Setting up I2S Audio...
[08:59:14][C][i2s_audio:028]: Setting up I2S Audio...
[08:59:14][C][i2s_audio.microphone:075]: Setting up I2S Audio Microphone...
[08:59:14][D][ring_buffer:034]: Created ring buffer with size 2048
[08:59:14][D][ring_buffer:034]: Created ring buffer with size 2048
[08:59:14][C][wifi:048]: Setting up WiFi...
[08:59:14][D][esp-idf:000]: I (189) pp: pp rom version: e7ae62f
[08:59:14]
[08:59:14][D][esp-idf:000]: I (190) net80211: net80211 rom version: e7ae62f
[08:59:14]
[08:59:14][D][esp-idf:000]: I (192) wifi:
[08:59:14][D][esp-idf:000]: wifi driver task: 3fcb3118, prio:23, stack:6656, core=0
[08:59:14][D][esp-idf:000]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1195) system_api: Base MAC address is not set
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1195) system_api: read default base MAC address from EFUSE
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1198) wifi:
[08:59:14][D][esp-idf:000][wifi]: wifi firmware version: ff661c3
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1199) wifi:
[08:59:14][D][esp-idf:000][wifi]: wifi certification version: v7.0
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1200) wifi:
[08:59:14][D][esp-idf:000][wifi]: config NVS flash: enabled
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1202) wifi:
[08:59:14][D][esp-idf:000][wifi]: config nano formating: disabled
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1203) wifi:
[08:59:14][D][esp-idf:000][wifi]: Init data frame dynamic rx buffer num: 512
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1205) wifi:
[08:59:14][D][esp-idf:000][wifi]: Init static rx mgmt buffer num: 5
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1206) wifi:
[08:59:14][D][esp-idf:000][wifi]: Init management short buffer num: 32
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1207) wifi:
[08:59:14][D][esp-idf:000][wifi]: Init static tx buffer num: 8
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1209) wifi:
[08:59:14][D][esp-idf:000][wifi]: Init tx cache buffer num: 32
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1210) wifi:
[08:59:14][D][esp-idf:000][wifi]: Init static tx FG buffer num: 2
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1212) wifi:
[08:59:14][D][esp-idf:000][wifi]: Init static rx buffer size: 1600
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1213) wifi:
[08:59:14][D][esp-idf:000][wifi]: Init static rx buffer num: 16
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1214) wifi:
[08:59:14][D][esp-idf:000][wifi]: Init dynamic rx buffer num: 512
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1216) wifi_init: rx ba win: 32
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1216) wifi_init: tcpip mbox: 512
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1217) wifi_init: udp mbox: 6
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1218) wifi_init: tcp mbox: 512
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1218) wifi_init: tcp tx win: 65535
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1219) wifi_init: tcp rx win: 65535
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1219) wifi_init: tcp mss: 1436
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1220) wifi_init: WiFi/LWIP prefer SPIRAM
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1220) wifi_init: WiFi IRAM OP enabled
[08:59:14]
[08:59:14][D][esp-idf:000]: I (1221) wifi_init: WiFi RX IRAM OP enabled
[08:59:14]
[08:59:14][C][wifi:061]: Starting WiFi...
[08:59:14][C][wifi:062]:   Local MAC: 64:E8:33:7F:81:94
[08:59:14][D][esp-idf:000][wifi]: I (1223) phy_init: phy_version 670,b7bc9b9,Apr 30 2024,10:54:13
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1258) wifi:
[08:59:14][D][esp-idf:000][wifi]: mode : sta (64:e8:33:7f:81:94)
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1260) wifi:
[08:59:14][D][esp-idf:000][wifi]: enable tsf
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: I (1262) wifi:
[08:59:14][D][esp-idf:000][wifi]: Set ps type: 1
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][D][wifi:482]: Starting scan...
[08:59:14][D][esp-idf:000][wifi]: I (1264) wifi:
[08:59:14][D][esp-idf:000][wifi]: Set ps type: 1
[08:59:14]
[08:59:14][D][esp-idf:000][wifi]: 
[08:59:14]
[08:59:14][W][component:157]: Component wifi set Warning flag: scanning for networks
[08:59:14][D][switch:055]: 'Mic mute': Sending state OFF
[08:59:14][D][switch:016]: 'mute_toggle' Turning OFF.
[08:59:14][D][switch:055]: 'mute_toggle': Sending state OFF
[08:59:17][D][wifi:497]: Found networks:
[08:59:17][I][wifi:541]: - 'WIFI UP' (3C:89:94:24:6F:72) ▂▄▆█
[08:59:17][D][wifi:542]:     Channel: 11
[08:59:17][D][wifi:543]:     RSSI: -58 dB
[08:59:17][I][wifi:541]: - 'WIFI UP' (38:A6:CE:07:BE:CE) ▂▄▆█
[08:59:17][D][wifi:542]:     Channel: 11
[08:59:17][D][wifi:543]:     RSSI: -64 dB
[08:59:17][I][wifi:541]: - 'WIFI UP' (20:47:ED:A5:C7:1E) ▂▄▆█
[08:59:17][D][wifi:542]:     Channel: 11
[08:59:17][D][wifi:543]:     RSSI: -71 dB
[08:59:17][D][wifi:546]: - 'WIFI' (90:84:0D:D7:A2:31) ▂▄▆█
[08:59:17][D][wifi:546]: - '' (7A:82:8C:9F:12:27) ▂▄▆█
[08:59:17][D][wifi:546]: - 'EE WiFi' (7A:82:8C:9F:12:24) ▂▄▆█
[08:59:17][D][wifi:546]: - 'BT-TRAG7J' (18:82:8C:9F:12:23) ▂▄▆█
[08:59:17][I][wifi:313]: WiFi Connecting to 'WIFI UP'...
[08:59:17][D][esp-idf:000][wifi]: I (4075) wifi:
[08:59:17][D][esp-idf:000][wifi]: new:<11,0>, old:<1,0>, ap:<255,255>, sta:<11,0>, prof:1
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:17][I][respeaker_lite:032]: Reading firmware version
[08:59:17][I][respeaker_lite:049]: Firmware version: 1.0.9
[08:59:17][D][text_sensor:064]: 'XMOS firmware version': Sending state '1.0.9'
[08:59:17][I][respeaker_lite:076]: Mic mute state: 0
[08:59:17][D][binary_sensor:036]: 'mute_state': Sending state OFF
[08:59:17][D][esp-idf:000][wifi]: I (4446) wifi:
[08:59:17][D][esp-idf:000][wifi]: state: init -> auth (b0)
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:17][D][esp-idf:000][wifi]: I (4450) wifi:
[08:59:17][D][esp-idf:000][wifi]: state: auth -> assoc (0)
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:17][D][esp-idf:000][wifi]: I (4460) wifi:
[08:59:17][D][esp-idf:000][wifi]: state: assoc -> run (10)
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:17][D][esp-idf:000][wifi]: I (4474) wifi:
[08:59:17][D][esp-idf:000][wifi]: <ba-add>idx:0 (ifx:0, 3c:89:94:24:6f:72), tid:0, ssn:0, winSize:64
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:17][D][esp-idf:000][wifi]: I (4520) wifi:
[08:59:17][D][esp-idf:000][wifi]: connected with WIFI UP, aid = 3, channel 11, BW20, bssid = 3c:89:94:24:6f:72
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:17][D][esp-idf:000][wifi]: I (4522) wifi:
[08:59:17][D][esp-idf:000][wifi]: security: WPA2-PSK, phy: bgn, rssi: -56
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:17][D][esp-idf:000][wifi]: I (4523) wifi:
[08:59:17][D][esp-idf:000][wifi]: pm start, type: 1
[08:59:17]
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:17][D][esp-idf:000][wifi]: I (4524) wifi:
[08:59:17][D][esp-idf:000][wifi]: set rx beacon pti, rx_bcn_pti: 0, bcn_timeout: 0, mt_pti: 25000, mt_time: 10000
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:17][D][esp-idf:000][wifi]: I (4586) wifi:
[08:59:17][D][esp-idf:000][wifi]: AP's beacon interval = 102400 us, DTIM period = 1
[08:59:17][D][esp-idf:000][wifi]: 
[08:59:17]
[08:59:18][D][esp-idf:000][sys_evt]: I (5527) esp_netif_handlers: sta ip: 192.168.0.8, mask: 255.255.255.0, gw: 192.168.0.1
[08:59:18]
[08:59:18][C][api:026]: Setting up Home Assistant API server...
[08:59:18][C][micro_wake_word:083]: Setting up microWakeWord...
[08:59:18][C][micro_wake_word:109]: Micro Wake Word initialized
[08:59:18][C][i2s_audio.speaker:091]: Setting up I2S Audio Speaker...
[08:59:18][I][nabu_media_player:111]: Set up nabu media player
[08:59:18][I][app:062]: setup() finished successfully!
[08:59:18][D][light:036]: 'respeaker-satellite' Setting:
[08:59:18][D][light:051]:   Brightness: 30%
[08:59:18][D][light:059]:   Red: 100%, Green: 100%, Blue: 0%
[08:59:18][W][component:170]: Component wifi cleared Warning flag
[08:59:18][W][component:157]: Component api set Warning flag: unspecified
[08:59:18][I][app:100]: ESPHome version 2024.10.2 compiled on Oct 28 2024, 08:57:14
[08:59:18][C][wifi:600]: WiFi:
[08:59:18][C][wifi:428]:   Local MAC: 64:E8:33:7F:81:94
[08:59:18][C][wifi:433]:   SSID: 'WIFI UP'
[08:59:18][C][wifi:436]:   IP Address: 192.168.0.8
[08:59:18][C][wifi:440]:   BSSID: 3C:89:94:24:6F:72
[08:59:18][C][wifi:441]:   Hostname: 'respeaker-satellite'
[08:59:18][C][wifi:443]:   Signal strength: -58 dB ▂▄▆█
[08:59:18][C][wifi:447]:   Channel: 11
[08:59:18][C][wifi:448]:   Subnet: 255.255.255.0
[08:59:18][C][wifi:449]:   Gateway: 192.168.0.1
[08:59:18][C][wifi:450]:   DNS1: 192.168.0.1
[08:59:18][C][wifi:451]:   DNS2: 0.0.0.0
[08:59:18][C][logger:185]: Logger:
[08:59:18][C][logger:186]:   Level: DEBUG
[08:59:18][C][logger:188]:   Log Baud Rate: 115200
[08:59:18][C][logger:189]:   Hardware UART: USB_SERIAL_JTAG
[08:59:18][C][i2c.idf:075]: I2C Bus:
[08:59:18][C][i2c.idf:076]:   SDA Pin: GPIO5
[08:59:18][C][i2c.idf:077]:   SCL Pin: GPIO6
[08:59:18][C][i2c.idf:078]:   Frequency: 50000 Hz
[08:59:18][C][i2c.idf:084]:   Recovery: bus successfully recovered
[08:59:18][I][i2c.idf:094]: Results from i2c bus scan:
[08:59:18][I][i2c.idf:100]: Found i2c device at address 0x18
[08:59:18][I][i2c.idf:100]: Found i2c device at address 0x42
[08:59:18][C][switch.gpio:068]: GPIO Switch 'mute_toggle'
[08:59:18][C][switch.gpio:091]:   Restore Mode: always OFF
[08:59:18][C][switch.gpio:031]:   Pin: GPIO4
[08:59:18][C][esp32_rmt_led_strip:187]: ESP32 RMT LED Strip:
[08:59:18][C][esp32_rmt_led_strip:188]:   Pin: 1
[08:59:18][C][esp32_rmt_led_strip:189]:   Channel: 0
[08:59:18][C][esp32_rmt_led_strip:214]:   RGB Order: GRB
[08:59:18][C][esp32_rmt_led_strip:215]:   Max refresh rate: 0
[08:59:18][C][esp32_rmt_led_strip:216]:   Number of LEDs: 1
[08:59:18][C][gpio.binary_sensor:015]: GPIO Binary Sensor 'User button'
[08:59:18][C][gpio.binary_sensor:016]:   Pin: GPIO3
[08:59:18][C][light:103]: Light 'respeaker-satellite'
[08:59:18][C][light:105]:   Default Transition Length: 0.0s
[08:59:18][C][light:106]:   Gamma Correct: 2.80
[08:59:18][C][template.switch:068]: Template Switch 'speaker_mute_switch'
[08:59:18][C][template.switch:091]:   Restore Mode: always OFF
[08:59:18][C][template.switch:057]:   Optimistic: YES
[08:59:18][C][template.switch:068]: Template Switch 'Mic mute'
[08:59:18][C][template.switch:070]:   Icon: 'mdi:microphone-off'
[08:59:18][C][template.switch:091]:   Restore Mode: always OFF
[08:59:18][C][template.switch:057]:   Optimistic: NO
[08:59:18][C][template.switch:068]: Template Switch 'timer_ringing'
[08:59:18][C][template.switch:091]:   Restore Mode: always OFF
[08:59:18][C][template.switch:057]:   Optimistic: YES
[08:59:18][C][psram:020]: PSRAM:
[08:59:18][C][psram:021]:   Available: YES
[08:59:18][C][psram:024]:   Size: 8191 KB
[08:59:18][C][safe_mode.button:024]: Safe Mode Button 'Safe Mode Boot'
[08:59:18][C][safe_mode.button:024]:   Icon: 'mdi:restart-alert'
[08:59:19][C][factory_reset.button:011]: Factory Reset Button 'Factory reset'
[08:59:19][C][factory_reset.button:011]:   Icon: 'mdi:restart-alert'
[08:59:19][C][restart.button:017]: Restart Button 'Restart'
[08:59:19][C][restart.button:017]:   Icon: 'mdi:restart'
[08:59:19][C][captive_portal:089]: Captive Portal:
[08:59:19][C][mdns:116]: mDNS:
[08:59:19][C][mdns:117]:   Hostname: respeaker-satellite
[08:59:19][C][esphome.ota:073]: Over-The-Air updates:
[08:59:19][C][esphome.ota:074]:   Address: respeaker-satellite.local:3232
[08:59:19][C][esphome.ota:075]:   Version: 2
[08:59:19][C][esphome.ota:078]:   Password configured
[08:59:19][C][safe_mode:018]: Safe Mode:
[08:59:19][C][safe_mode:020]:   Boot considered successful after 60 seconds
[08:59:19][C][safe_mode:021]:   Invoke after 10 boot attempts
[08:59:19][C][safe_mode:023]:   Remain in safe mode for 300 seconds
[08:59:19][C][api:140]: API Server:
[08:59:19][C][api:141]:   Address: respeaker-satellite.local:6053
[08:59:19][C][api:145]:   Using noise encryption: NO
[08:59:19][C][micro_wake_word:072]: microWakeWord:
[08:59:19][C][micro_wake_word:073]:   models:
[08:59:19][C][micro_wake_word:015]:     - Wake Word: Okay Nabu
[08:59:19][C][micro_wake_word:016]:       Probability cutoff: 0.97
[08:59:19][C][micro_wake_word:017]:       Sliding window size: 5
[08:59:19][C][micro_wake_word:015]:     - Wake Word: Stop
[08:59:19][C][micro_wake_word:016]:       Probability cutoff: 0.50
[08:59:19][C][micro_wake_word:017]:       Sliding window size: 5
[08:59:19][C][micro_wake_word:021]:     - VAD Model
[08:59:19][C][micro_wake_word:022]:       Probability cutoff: 0.50
[08:59:19][C][micro_wake_word:023]:       Sliding window size: 5
[08:59:23][D][api:103]: Accepted 192.168.0.29
[08:59:23][W][component:170]: Component api cleared Warning flag
[08:59:23][D][api.connection:1446]: Home Assistant 2024.10.4 (192.168.0.29): Connected successfully
[08:59:23][D][light:036]: 'respeaker-satellite' Setting:
[08:59:23][D][light:051]:   Brightness: 30%
[08:59:23][D][light:059]:   Red: 100%, Green: 100%, Blue: 0%
[08:59:23][D][micro_wake_word:379]: Starting wake word detection
[08:59:23][D][light:036]: 'respeaker-satellite' Setting:
[08:59:23][D][light:047]:   State: OFF
[08:59:23][D][light:109]:   Effect: 'None'
[08:59:23][D][esp-idf:000][microphone_task]: I (10234) I2S: DMA Malloc info, datalen=blocksize=1280, dma_buf_count=4
[08:59:23]
[08:59:23][D][i2s_audio.microphone:377]: Starting I2S Audio Microphne
[08:59:23][D][i2s_audio.microphone:381]: Started I2S Audio Microphone
[08:59:23][D][micro_wake_word:418]: State changed from IDLE to DETECTING_WAKE_WORD
[08:59:29][D][micro_wake_word:357]: Detected 'Okay Nabu' with sliding average probability is 0.98 and max probability is 1.00
[08:59:29][D][voice_assistant:516]: State changed from IDLE to START_MICROPHONE
[08:59:29][D][voice_assistant:522]: Desired state set to START_PIPELINE
[08:59:29][D][voice_assistant:225]: Starting Microphone
[08:59:29][D][ring_buffer:034]: Created ring buffer with size 16384
[08:59:29][D][voice_assistant:516]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[08:59:29][D][voice_assistant:516]: State changed from STARTING_MICROPHONE to START_PIPELINE
[08:59:29][D][voice_assistant:280]: Requesting start...
[08:59:29][D][voice_assistant:516]: State changed from START_PIPELINE to STARTING_PIPELINE
[08:59:29][D][voice_assistant:537]: Client started, streaming microphone
[08:59:29][D][voice_assistant:516]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[08:59:29][D][voice_assistant:522]: Desired state set to STREAMING_MICROPHONE
[08:59:29][D][voice_assistant:639]: Event Type: 1
[08:59:29][D][voice_assistant:642]: Assist Pipeline running
[08:59:29][D][voice_assistant:639]: Event Type: 3
[08:59:29][D][voice_assistant:653]: STT started
[08:59:29][D][light:036]: 'respeaker-satellite' Setting:
[08:59:29][D][light:047]:   State: ON
[08:59:29][D][light:051]:   Brightness: 60%
[08:59:29][D][light:059]:   Red: 100%, Green: 20%, Blue: 100%
[08:59:29][D][light:109]:   Effect: 'Slow Pulse'
[08:59:44][D][voice_assistant:639]: Event Type: 12
[08:59:44][D][voice_assistant:806]: STT by VAD end
[08:59:44][D][voice_assistant:516]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE
[08:59:44][D][voice_assistant:522]: Desired state set to AWAITING_RESPONSE
[08:59:44][D][voice_assistant:516]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[08:59:44][D][light:036]: 'respeaker-satellite' Setting:
[08:59:44][D][light:051]:   Brightness: 60%
[08:59:44][D][light:059]:   Red: 100%, Green: 20%, Blue: 100%
[08:59:44][D][light:109]:   Effect: 'Fast Pulse'
[08:59:44][D][voice_assistant:516]: State changed from STOPPING_MICROPHONE to AWAITING_RESPONSE
[08:59:44][D][voice_assistant:516]: State changed from AWAITING_RESPONSE to AWAITING_RESPONSE
[08:59:54][D][voice_assistant:639]: Event Type: 4
[08:59:54][D][voice_assistant:667]: Speech recognised as: " 1.  1.  2."
[08:59:54][D][voice_assistant:639]: Event Type: 5
[08:59:54][D][voice_assistant:672]: Intent started
[08:59:54][D][voice_assistant:639]: Event Type: 6
[08:59:54][D][voice_assistant:639]: Event Type: 7
[08:59:54][D][voice_assistant:695]: Response: "Sorry, I couldn't understand that"
[08:59:54][D][light:036]: 'respeaker-satellite' Setting:
[08:59:54][D][light:051]:   Brightness: 60%
[08:59:54][D][light:059]:   Red: 20%, Green: 100%, Blue: 100%
[08:59:54][D][light:109]:   Effect: 'Slow Pulse'
[08:59:54][D][voice_assistant:639]: Event Type: 8
[08:59:54][D][voice_assistant:717]: Response URL: "http://192.168.0.29:8123/api/tts_proxy/dae2cdcb27a1d1c3b07ba2c7db91480f9d4bfd8f_en-gb_1c54da98d5_tts.piper.flac"
[08:59:54][D][voice_assistant:516]: State changed from AWAITING_RESPONSE to STREAMING_RESPONSE
[08:59:54][D][voice_assistant:522]: Desired state set to STREAMING_RESPONSE
[08:59:54][D][media_player:080]: 'Media Player' - Setting
[08:59:54][D][media_player:087]:   Media URL: http://192.168.0.29:8123/api/tts_proxy/dae2cdcb27a1d1c3b07ba2c7db91480f9d4bfd8f_en-gb_1c54da98d5_tts.piper.flac
[08:59:54][D][media_player:093]:  Announcement: yes
[08:59:54][D][voice_assistant:639]: Event Type: 2
[08:59:54][D][voice_assistant:731]: Assist Pipeline ended
[08:59:54][D][ring_buffer:034]: Created ring buffer with size 48000
[08:59:54][D][ring_buffer:034]: Created ring buffer with size 48000
[08:59:54][D][ring_buffer:034]: Created ring buffer with size 65536
[08:59:54][D][ring_buffer:034]: Created ring buffer with size 65536
[08:59:54][D][nabu_media_player.pipeline:174]: Reading FLAC file type
[08:59:55][D][nabu_media_player.pipeline:186]: Decoded audio has 1 channels, 16000 Hz sample rate, and 16 bits per sample
[08:59:55][D][nabu_media_player.pipeline:211]: Converting mono channel audio to stereo channel audio
[08:59:55][D][ring_buffer:034][speaker_task]: Created ring buffer with size 16384
[08:59:55][D][esp-idf:000][speaker_task]: I (42081) I2S: DMA Malloc info, datalen=blocksize=2048, dma_buf_count=4
[08:59:55]
[08:59:55][D][i2s_audio.speaker:118]: Starting Speaker
[08:59:55][D][i2s_audio.speaker:123]: Started Speaker
[08:59:57][D][voice_assistant:516]: State changed from STREAMING_RESPONSE to IDLE
[08:59:57][D][voice_assistant:522]: Desired state set to IDLE
[08:59:57][D][light:036]: 'respeaker-satellite' Setting:
[08:59:57][D][light:047]:   State: OFF
[08:59:57][D][light:109]:   Effect: 'None'
[08:59:58][D][esp-idf:000][speaker_task]: I (44841) I2S: DMA queue destroyed
[08:59:58]
[08:59:58][D][i2s_audio.speaker:130]: Stopping Speaker
[08:59:58][D][i2s_audio.speaker:136]: Stopped Speaker
[09:00:06][D][esp32.preferences:114]: Saving 1 preferences to flash...
[09:00:06][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed

ginandbacon · October 28, 2024, 5:41pm

My workaround for muting my soundbar since the assist binary sensor was deprecated. Actually works better because you can un mute on the responding phase. You have to create some template sensors. Here are all four, then create an automation. Need to get fancy and create a template sensor to store the current volume level, lower the volume to a specific level that makes voice commands fire, then restore to the value in the template sensor. Just posting it here in case it helps anyone else. It still doesn’t work great when the TV is loud

respeaker_thinking_phase:
      value_template: >
        {% if is_state('assist_satellite.respeakerv3_assist_satellite', 'processintg') %}
         thinking
        {% else %} 
            no
        {% endif %}      
        
    respeaker_replying_phase:
      value_template: >
        {% if is_state('assist_satellite.respeakerv3_assist_satellite', 'responding') %}
         replying
        {% else %} 
            no
        {% endif %}   
        
    respeaker_idle_phase:
      value_template: >
        {% if is_state('assist_satellite.respeakerv3_assist_satellite', 'idle') %}
           idle
        {% else %} 
            no
        {% endif %}

formatBCE · October 28, 2024, 8:22pm

I’ve updated YAML. I didn’t realize that audio_dac: section wasn’t there…
@AndyCap can you try with that addition?

Apart from that, added sounds for mute on/off, and wake sound (switchable on/off in UI).

AndyCap · October 29, 2024, 7:07am

Thanks, I still get the same noise with that version unfortunately.

I like the sound it plays now when it recognises the wake word

Everything working perfectly through the speaker though.

I rendered a Sine wave to a wav file and loaded it up into HA and used the media player to get an idea of the noise:

Here is what is happening to the sine wave:

And a close up:

Here is the noise I get after the wake word:

And a close up:

The distortion is all positive, if you look at the last image you can see this easily.

I’m also beginning to think this might be dependent on what the cpu is doing, some interference of some kind. It’s very weird.

I might just stick to using the speaker