not on the s3box i haven’t yet … but have on other devices. after adding, did you do a ‘clean build’
and yes Rob is correct 
Hi John 
This is what I added for my first gen ESP32 S3 Box:
micro_wake_word:
model: 'github://kahrendt/microwakeword/models/hey_jarvis.json'
on_wake_word_detected:
then:
- voice_assistant.start:
It works but I’m not sure if it’s just using the remote (HA server) wake word detection or on-board…
if you watch the ESPHome logs, you will see if it is using it. it should be
I’ll have a look when I get home tonight.
@robgough1970 After more troubleshooting…
Even just adding this to a working yaml using the standard package produces the same error:
micro_wake_word:
model: okay_nabu
This also produces the same error:
micro_wake_word:
model: github://esphome/micro-wake-word-models/models/okay_nabu.json
Commenting out those two lines results in a compile without error. And yes, I am running it on a new Box3 purchased in December.
Did you get any of those error 1’s on fake_micro_context.o? How did you get past them? I’m guessing there’s a library ref missing from the docs.
I had a peak at the code and I doubt just that code would disable the streaming wake word detection that is in the box3 yaml repo. In there it starts voice_assistant.start_continuous. I suspect that would need to be replaced with micro_wake_word.start plus other places where it is start/stopped by the mute switch. Not 100% on this though, however there would definitely need to be a micro_wake_word.start somewhere.
I forked the s3 firmware here to experiment with these factors but am not able to test yet due to the compile errors I’m having.
I don’t think so because the Voice Assistan part ever changes. We are still starting the same VA, just via another method.
micro_wake_word:
model: 'github://kahrendt/microwakeword/models/hey_jarvis.json'
on_wake_word_detected:
then:
- voice_assistant.start:
…already has that covered.
EDIT: actually, I think we just need to invoke the micro wake word start when the device boots. Then the above covers the rest.
So I added the below under the VA section:
on_client_connected:
- if:
condition:
switch.is_off: mute
then:
- wait_until:
not: ble.enabled
- micro_wake_word.start # <-----ADDED THIS LINE ONLY FOR WAKE WORD. YET TO TEST
- voice_assistant.start_continuous:
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
It passed the validation test, now to flash it. (I’m still at work so doing this remotely)
UPDATE: Compiled and flashed. Looks promising…
[15:12:30][C][micro_wake_word:057]: microWakeWord:
[15:12:30][C][micro_wake_word:058]: Wake Word: hey jarvis
[15:12:30][C][micro_wake_word:059]: Probability cutoff: 0.500
[15:12:30][C][micro_wake_word:060]: Sliding window size: 10
[15:12:33][D][esp32_ble:255]: Disabling BLE...
[15:12:33][D][esp-idf:000]: W (12532) BT_APPL: bta_dm_disable BTA_DISABLE_DELAY set to 200 ms
[15:12:33][W][component:214]: Component esp32_ble took a long time for an operation (0.21 s).
[15:12:33][W][component:215]: Components should block for at most 20-30ms.
…however I think I have the micro_wake_word.start in the wrong location. Going to try starting it at after api connection with a delay.
the only errors that I have run into is when compiling using cli on 2024.2.x I get a can’t find ‘hey_jarvis.tflite’ error. but works fine compiling from ESPHome add-on.
I was previously using it as an external component direct from kahrendts repo and it compiled from cli and ESPHome add-on
My latest config to start the wake word service 5 seconds after the API connects seems to have made a difference based on the logs. Can’t wait to get home and run some real tests!
[15:21:46][C][micro_wake_word:057]: microWakeWord:
[15:21:46][C][micro_wake_word:058]: Wake Word: hey jarvis
[15:21:46][C][micro_wake_word:059]: Probability cutoff: 0.500
[15:21:46][C][micro_wake_word:060]: Sliding window size: 10
[15:21:48][D][esp32_ble:255]: Disabling BLE...
[15:21:48][D][esp-idf:000]: W (12626) BT_APPL: bta_dm_disable BTA_DISABLE_DELAY set to 200 ms
[15:21:49][W][component:214]: Component esp32_ble took a long time for an operation (0.21 s).
[15:21:49][W][component:215]: Components should block for at most 20-30ms.
[15:21:50][D][micro_wake_word:170]: State changed from IDLE to START_MICROPHONE
[15:21:50][D][micro_wake_word:108]: Starting Microphone
[15:21:50][D][micro_wake_word:170]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[15:21:50][D][esp-idf:000]: I (14078) I2S: DMA Malloc info, datalen=blocksize=512, dma_buf_count=8
[15:21:50][D][esp-idf:000]: I (14083) I2S: I2S0, MCLK output by GPIO2
[15:21:50][D][esp-idf:000]: I (14088) ESP32_S3_BOX: I2S0, MCLK output by GPIO0
[15:21:50][D][esp-idf:000]: I (14092) AUDIO_PIPELINE: link el->rb, el:0x3d85a774, tag:i2s, rb:0x3d85ab88
[15:21:50][D][esp-idf:000]: I (14098) AUDIO_PIPELINE: link el->rb, el:0x3d85a8e8, tag:filter, rb:0x3d85cbc8
[15:21:50][D][esp-idf:000]: I (14105) AUDIO_ELEMENT: [i2s-0x3d85a774] Element task created
[15:21:50][D][esp-idf:000]: I (14109) AUDIO_THREAD: The filter task allocate stack on external memory
[15:21:50][D][esp-idf:000]: I (14116) AUDIO_ELEMENT: [filter-0x3d85a8e8] Element task created
[15:21:50][D][esp-idf:000]: I (14121) AUDIO_ELEMENT: [raw-0x3d85aa18] Element task created
[15:21:50][D][esp-idf:000]: I (14125) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8102111 Bytes, Inter:101592 Bytes, Dram:101592 Bytes
[15:21:50][D][esp-idf:000]: I (14131) AUDIO_ELEMENT: [i2s] AEL_MSG_CMD_RESUME,state:1
[15:21:50][D][esp-idf:000]: I (14137) AUDIO_ELEMENT: [filter] AEL_MSG_CMD_RESUME,state:1
[15:21:50][D][esp-idf:000]: I (14141) RSP_FILTER: sample rate of source data : 16000, channel of source data : 2, sample rate of destination data : 16000, channel of destination data : 1
[15:21:50][D][esp-idf:000]: I (14146) AUDIO_PIPELINE: Pipeline started
[15:21:50][D][esp_adf.microphone:273]: Microphone started
[15:21:50][D][micro_wake_word:170]: State changed from STARTING_MICROPHONE to DETECTING_WAKE_WORD
[15:21:50][W][micro_wake_word:150]: Wake word is already running
[15:26:38][I][ota:117]: Boot seems successful, resetting boot loop counter.
[15:26:38][D][esp32.preferences:114]: Saving 1 preferences to flash...
[15:26:38][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed
did you try a ‘clean build files’ or even manually deleting the build directory ?
You definitely have stopped the normal ESPhome add-on and started the Beta one…? (rude question I know, but I’ve seen some craziness in this forum!)
I deleted the esphome addon and added the beta (2024.2.0b2).
I have to do a clean build files every time, otherwise it errors out after about 3 compile lines right at the start.
Deleting the build directory is not something I had considered. Where to I find that? I’m running the current HA OS on a green.
I haven’t been doing any of that, I simple stop the normal add-on and start the beta one. Then open the beta UI.
Perhaps post your full yaml
2024.2.0 has been released. I have installed it and removed the beta, but still same error.
yaml is pretty basic:
substitutions:
name: esp32-s3-box-3-5ad0fc
friendly_name: Jarvis
packages:
esphome.voice-assistant: github://esphome/firmware/voice-assistant/esp32-s3-box-3.yaml@main
esphome:
name: ${name}
name_add_mac_suffix: false
friendly_name: ${friendly_name}
api:
encryption:
key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
micro_wake_word:
model: github://esphome/micro-wake-word-models/models/okay_nabu.json
Archiving .pioenvs/esp32-s3-box-3-5ad0fc/esp-idf/esp-sr/libesp-sr.a
Compiling .pioenvs/esp32-s3-box-3-5ad0fc/components/esp-tflite-micro/tensorflow/lite/micro/fake_micro_context.o
Compiling .pioenvs/esp32-s3-box-3-5ad0fc/components/esp-tflite-micro/tensorflow/lite/micro/flatbuffer_utils.o
Compiling .pioenvs/esp32-s3-box-3-5ad0fc/components/esp-tflite-micro/tensorflow/lite/micro/memory_helpers.o
Archiving .pioenvs/esp32-s3-box-3-5ad0fc/esp-idf/esp-dsp/libesp-dsp.a
Compiling .pioenvs/esp32-s3-box-3-5ad0fc/components/esp-tflite-micro/tensorflow/lite/micro/micro_allocation_info.o
xtensa-esp32s3-elf-g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
*** [.pioenvs/esp32-s3-box-3-5ad0fc/components/esp-tflite-micro/tensorflow/lite/micro/fake_micro_context.o] Error 1
========================= [FAILED] Took 586.13 seconds =========================
Is there a debug log somewhere to see more detail? It seems Error 1 is pretty generic.
I don’t have my ESPhome pulling the code from GitHub, I have it all locally in the yaml file, plus mine is a first gen (not Box3) but I don’t see why it should be failing.
Not that I’m aware of. That is a pretty useless error message.
I recently bought the S3 box as well and was trying to get it flashed with the example proyect. I was not able to flash it on my HA instance (RP 2GB), but I came up with the idea of spinning op a docker container with EspHome.
docker run -d --name=esphome -e TZ=Etc/UTC -e ESPHOME_DASHBOARD_USE_PING=false -p 6052:6052 ghcr.io/esphome/esphome
I copied the encryption key from my HA instance and everything needed into the config. I first choose Install > Manual > Modern binary and after that I choose Install > Computer. This way I was able to get it flashed and connected to HA
Ok, I got some good help from Jessie and others on the Discord server:
jesserockz
Killed means the os killed it due to ram
What are you compiling on?-jaym-
On a green
This is a bummer. I just moved my instance from a VM on an M1 laptop to give better separation of service and keep availability high. Is there any way to give more RAM/Swap to the docker instance?jesserockz
A green should be alright, but it also depends on what else you have running on there in terms of add-ons etc
-jaym-
Yes, I do have a few add ons running. 16 as in the screenshot. I’ll have to find out a way to see how much each is consuming.
-jaym-
Thanks, I’ve installed the ‘glances’ addon to get the ram usage of the addons. Stopped the TeslaMate addon and it’s dependancies (Grafana, MQTT, PostGres) which were using a lot of RAM. Also stopped openWakeWord, SQLite, and the compile is proceeding past the point where it has always gotten stuck.
I finally tested the local wake word on my ESP Box… doesn’t work at all. Zero detection / activation. So I’ve definitely got something wrong. It would be nice to see a full yaml config of how to set this up rather than just bits and pieces of code here and there throughout the ESPhome docs.
EDIT: The code on Github for these ESP Boxes has been updated to include the on-board wake word.
With the introduction if the new MicroWakeWord on the S3 box itself, it shouldn’t tax the CPU of the Raspberry Pi correct?
Yep, that’s right.