Correct. It had been working for a while but i changed my network mode for some other reason not associated to voice. Then i read host mode only and took a peek.
Here’s the korvo-1 (not V1. 1) with continued conversation. This allows it to listen for multiple commands. The slider is for how many seconds pass after hearing the last command then it stops. I need to snag the URL of the site I got the yaml for. You have to use a subscription for the wake word model or it doesn’t work as it’s used in scripts and other stuff.
micro_wake_word:
model: ${micro_wake_word_model}
substitutions:
name: "korvo"
friendly_name: korvo
voice_assist_idle_phase_id: "1"
voice_assist_listening_phase_id: "2"
voice_assist_thinking_phase_id: "3"
voice_assist_replying_phase_id: "4"
voice_assist_not_ready_phase_id: "10"
voice_assist_error_phase_id: "11"
voice_assist_muted_phase_id: "12"
micro_wake_word_model: hey_jarvis
esphome:
name: ${name}
friendly_name: ${friendly_name}
name_add_mac_suffix: true
platformio_options:
board_build.flash_mode: dio
upload_speed: 460800
project:
name: esphome.voice-assistant
version: "1.0"
min_version: 2023.11.5
on_boot:
- priority: 600
then:
- light.turn_on:
id: led_ring
red: 0%
blue: 0%
green: 100%
brightness: 100%
effect: random
- if:
condition:
lambda: return id(init_in_progress);
then:
- lambda: id(init_in_progress) = false;
esp32:
board: esp32-s3-devkitc-1
flash_size: 16MB
framework:
type: esp-idf
sdkconfig_options:
CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y"
CONFIG_ESP32S3_DATA_CACHE_64KB: "y"
CONFIG_ESP32S3_DATA_CACHE_LINE_64B: "y"
CONFIG_AUDIO_BOARD_CUSTOM: "y"
CONFIG_ESP32_S3_KORVO1_BOARD: "y"
components:
- name: esp32_s3_korvo1_board
#source: github://espressif/components/hardware_driver@main
source: github://abmantis/esphome_custom_audio_boards@main
refresh: 0s
psram:
mode: octal
speed: 80MHz
external_components:
- source: github://pr#5230
components: esp_adf
ota:
logger:
api:
encryption:
key: YOU_API_KEY
on_client_connected:
then:
- if:
condition:
switch.is_off: mute
then:
- delay: 20ms
- ble.disable:
- light.turn_on:
id: led_ring
blue: 0%
red: 0%
green: 100%
brightness: 50%
effect: led12
on_client_disconnected:
then:
- ble.enable
- light.turn_on:
id: led_ring
blue: 0%
red: 100%
green: 100%
brightness: 50%
effect: connecting
dashboard_import:
package_import_url: github://esphome/firmware/voice-assistant/esp32-s3-korvo1.yaml@main
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
use_address: 192.168.0.48
ap:
on_connect:
then:
- delay: 20ms # Gives time for improv results to be transmitted
- ble.disable:
- delay: 5s
on_disconnect:
then:
- ble.enable:
improv_serial:
esp32_improv:
authorizer: none
button:
- platform: factory_reset
id: factory_reset_btn
name: Factory reset
esp_adf:
board: esp32s3korvo1
microphone:
- platform: esp_adf
id: korvo_mic
speaker:
- platform: esp_adf
id: korvo_speaker
micro_wake_word:
model: ${micro_wake_word_model}
on_wake_word_detected:
- if:
condition:
switch.is_on: continued_conversation
then:
- lambda: id(va).set_use_wake_word(false);
- voice_assistant.start_continuous: # wake_word: !lambda return wake_word; not yet supported for continuous
else:
- lambda: id(va).set_use_wake_word(false);
- voice_assistant.start:
wake_word: !lambda return wake_word;
- light.turn_on:
id: led_ring
red: 30%
green: 30%
blue: 70%
brightness: 100%
effect: wakeword
voice_assistant:
id: va
microphone: korvo_mic
speaker: korvo_speaker
noise_suppression_level: 2
auto_gain: 31dBFS
volume_multiplier: 3.0
vad_threshold: 3
on_listening:
- lambda: id(voice_assistant_phase) = ${voice_assist_listening_phase_id};
- light.turn_on:
id: led_ring
blue: 100%
red: 0%
green: 0%
brightness: 100%
effect: led12
# - script.execute: reset_led
- if:
condition:
- switch.is_on: continued_conversation
then:
- script.execute: stt_timeout_to_idle
- lambda: id(va).set_use_wake_word(false);
on_stt_vad_end:
- lambda: id(voice_assistant_phase) = ${voice_assist_thinking_phase_id};
- script.execute: reset_led
- if:
condition:
- switch.is_on: continued_conversation
then:
- script.execute: stt_timeout_to_idle
- light.turn_on:
id: led_ring
blue: 0%
red: 50%
green: 50%
brightness: 100%
effect: working
# - script.execute: reset_led
on_tts_stream_start:
- lambda: id(voice_assistant_phase) = ${voice_assist_replying_phase_id};
- light.turn_on:
id: led_ring
blue: 50%
red: 50%
green: 0%
brightness: 100%
effect: pulse
#- script.execute: reset_led
on_tts_stream_end:
- if:
condition:
- switch.is_on: continued_conversation
then:
- lambda: id(voice_assistant_phase) = ${voice_assist_listening_phase_id};
- script.execute: stt_timeout_to_idle
else:
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
- light.turn_on:
id: led_ring
blue: 0%
red: 0%
green: 100%
brightness: 20%
effect: connecting
# - script.execute: reset_led
on_end:
- if:
condition:
and:
- switch.is_off: mute
- lambda: return id(wake_word_engine_location).state == "On device";
- switch.is_off: continued_conversation
then:
- script.execute: return_to_idle
on_error:
- if:
condition:
and:
- lambda: return !id(init_in_progress);
- lambda: return !(code == "stt-no-text-recognized");
then:
- lambda: id(voice_assistant_phase) = ${voice_assist_error_phase_id};
- script.execute: reset_led
- delay: 1s
- script.stop: stt_timeout_to_idle
- if:
condition:
switch.is_off: mute
then:
# - logger.log: "(on_client_connected) Returning to idle by script"
- script.execute: return_to_idle
else:
- lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
- script.execute: reset_led
on_client_connected:
- if:
condition:
switch.is_off: mute
then:
- wait_until:
not: ble.enabled
- script.execute: return_to_idle
else:
- lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
- lambda: id(init_in_progress) = false;
- script.execute: reset_led
on_client_disconnected:
- if:
condition:
lambda: return id(wake_word_engine_location).state == "In Home Assistant";
then:
- lambda: id(va).set_use_wake_word(false);
- voice_assistant.stop:
- if:
condition:
lambda: return id(wake_word_engine_location).state == "On device";
then:
- micro_wake_word.stop
- lambda: id(voice_assistant_phase) = ${voice_assist_not_ready_phase_id};
script:
- id: stt_timeout_to_idle
mode: restart # restart the timer if called before timeout
then:
if:
condition:
switch.is_on: continued_conversation
then:
- delay: !lambda "return id(continued_timeout).state * 1000;"
- if:
condition:
lambda: return (id(voice_assistant_phase) == ${voice_assist_replying_phase_id});
then:
- wait_until:
condition:
lambda: return !(id(voice_assistant_phase) == ${voice_assist_replying_phase_id});
# normally this would complete and move to next phase with on_tts_stream_end,
# but sometimes this is missed so put a time limit on the wait
timeout: 10s
- delay: 1s # Give time for the stream to end and the phase to be switched back to listening and this timeout to be reset
- script.execute: return_to_idle
- id: return_to_idle
then:
- if:
condition:
lambda: return id(wake_word_engine_location).state == "On device";
then:
- script.stop: stt_timeout_to_idle
- if:
condition:
voice_assistant.is_running
then:
- lambda: id(va).set_use_wake_word(false);
- voice_assistant.stop
- wait_until:
condition:
not:
voice_assistant.is_running
timeout: 5s
- if:
condition:
not:
micro_wake_word.is_running
then:
- micro_wake_word.start:
- if:
condition:
lambda: return id(wake_word_engine_location).state == "In Home Assistant";
then:
- if:
condition:
micro_wake_word.is_running
then:
- micro_wake_word.stop:
- wait_until:
condition:
not:
micro_wake_word.is_running
timeout: 5s
- wait_until:
condition:
not:
voice_assistant.is_running
timeout: 5s
- lambda: id(va).set_use_wake_word(true);
- if:
condition:
- not:
voice_assistant.is_running
then:
- voice_assistant.start_continuous:
- lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
- id: reset_led
then:
- if:
condition:
switch.is_off: mute
then:
- light.turn_on:
id: led_ring
blue: 100%
red: 100%
green: 0%
brightness: 100%
effect: connecting
else:
- light.turn_off: led_ring
switch:
- platform: gpio
id: pa_ctrl
pin: GPIO38
name: "${friendly_name} Speaker Mute"
restore_mode: ALWAYS_ON
- platform: template
name: Mute
id: mute
optimistic: true
restore_mode: RESTORE_DEFAULT_OFF
entity_category: config
on_turn_off:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- script.execute: return_to_idle
on_turn_on:
- if:
condition:
lambda: return !id(init_in_progress);
then:
- lambda: id(va).set_use_wake_word(false);
- if:
condition:
- voice_assistant.is_running
then:
- voice_assistant.stop
- if:
condition:
- micro_wake_word.is_running
then:
- micro_wake_word.stop
- lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
- script.execute: reset_led
- platform: template
name: Continued conversation
id: continued_conversation
optimistic: true
restore_mode: RESTORE_DEFAULT_OFF
entity_category: config
on_turn_off:
- script.execute: return_to_idle
on_turn_on:
- script.execute: return_to_idle
- platform: restart
name: "korvo restart"
number:
- platform: template
entity_category: config
name: Continued timeout
id: continued_timeout
icon: mdi:clock
optimistic: true
restore_value: true
initial_value: 8
min_value: 1
step: 1
max_value: 20
unit_of_measurement: s
mode: slider
select:
- platform: template
entity_category: config
name: Wake word engine location
id: wake_word_engine_location
optimistic: true
restore_value: true
options:
- In Home Assistant
- On device
initial_option: On device
on_value:
- wait_until:
lambda: return id(voice_assistant_phase) == ${voice_assist_muted_phase_id} || id(voice_assistant_phase) == ${voice_assist_idle_phase_id};
- script.execute: return_to_idle
globals:
- id: init_in_progress
type: bool
restore_value: false
initial_value: "true"
- id: voice_assistant_phase
type: int
restore_value: false
initial_value: ${voice_assist_not_ready_phase_id}
light:
- platform: esp32_rmt_led_strip
id: led_ring
is_rgbw: true
rgb_order: GRB
pin: GPIO19
num_leds: 12
rmt_channel: 0
chipset: WS2812
name: "${friendly_name} Light"
default_transition_length: 1s
effects:
- addressable_scan:
name: "led12"
move_interval: 10ms
scan_width: 12
- pulse:
name: "pulse"
transition_length: 0.5s
update_interval: 0.5s
- addressable_twinkle:
name: "working"
twinkle_probability: 5%
progress_interval: 4ms
- addressable_color_wipe:
name: "wakeword"
colors:
- red: 0%
green: 0%
blue: 100%
num_leds: 12
add_led_interval: 20ms
reverse: false
- addressable_color_wipe:
name: "connecting"
colors:
- red: 40%
green: 30%
blue: 30%
num_leds: 12
add_led_interval: 50ms
reverse: true
binary_sensor:
- platform: template
name: "${friendly_name} Volume Up"
id: btn_volume_up
- platform: template
name: "${friendly_name} Volume Down"
id: btn_volume_down
- platform: template
name: "${friendly_name} Set"
id: btn_set
on_multi_click:
- timing:
- ON for at least 10s
then:
- button.press: factory_reset_btn
- platform: template
name: "${friendly_name} Play"
id: btn_play
- platform: template
name: "${friendly_name} Mode"
id: btn_mode
on_press:
- voice_assistant.start_continuous:
- platform: template
name: "${friendly_name} Record"
id: btn_record
on_press:
- voice_assistant.start:
#- lambda: id(va).set_use_wake_word(true);
sensor:
- id: button_adc
platform: adc
internal: true
pin: 8
attenuation: 11db
update_interval: 15ms
filters:
- median:
window_size: 5
send_every: 5
send_first_at: 1
- delta: 0.1
on_value_range:
- below: 0.55
then:
- binary_sensor.template.publish:
id: btn_volume_up
state: ON
- above: 0.65
below: 0.92
then:
- binary_sensor.template.publish:
id: btn_volume_down
state: ON
- above: 1.02
below: 1.33
then:
- binary_sensor.template.publish:
id: btn_set
state: ON
- above: 1.43
below: 1.77
then:
- binary_sensor.template.publish:
id: btn_play
state: ON
- above: 1.87
below: 2.15
then:
- binary_sensor.template.publish:
id: btn_mode
state: ON
- above: 1.01
below: 2.56
then:
- binary_sensor.template.publish:
id: btn_record
state: ON
- above: 2.3
then:
- binary_sensor.template.publish:
id: btn_volume_up
state: OFF
- binary_sensor.template.publish:
id: btn_volume_down
state: OFF
- binary_sensor.template.publish:
id: btn_set
state: OFF
- binary_sensor.template.publish:
id: btn_play
state: OFF
- binary_sensor.template.publish:
id: btn_mode
state: OFF
- binary_sensor.template.publish:
id: btn_record
state: OFF
Have you added the media_player itself? It appears ID is a required field also according to the docs but I have not tried to use it as a media player either
All media_player actions can be used without specifying an id if you have only one media_player in your configuration YAML
So continued conversation is both awesome and amazing depending on the situation. It works flawlessly with no audio source near it like a TV or music. If you do, the Korvo picks up that audio and kind of gets stuck in a loop of “sorry, I didn’t understand that” because it picks up audio from TV/Music and will just start saying that over and over. There was an enhancement request for full support but the ESPHome team has been busy fixing issues and bug requests always get fixed before engagement requests for obvious reasons. That and most the examples are for the S3 box variants which have a screen so you have to remove all that stuff. If they added something like a “stop listening” command, which should be somewhat easy to do, it would be way better because it would fix the external audio loop issue. It is nice to be able to tell it multiple commands without having to say the trigger word every single time.
One small issue is I e had it lock up twice but only after applying the latest core update yesterday, before that it didn’t freeze up so I’m not sure if I got lucky and didn’t have any lock ups before the update or if the core update caused the issue.
Enhancement request
ESPHome code that I merged into my working config file.
I believe you can say “Cancel” to stop the wake word from detecting. I have a box-3 now and tested that PR out yesterday and it worked, but as you said, if you’re in a loud environment it doesnt work so well. I ended up reverting back for the box-3 to the default but modified the idle screen to display a clock which i think is really handy.
substitutions:
name: esp32-s3-box-3-6405bc
friendly_name: Kitchen ESP Box
micro_wake_word_model: alexa
packages:
esphome.voice-assistant: github://esphome/firmware/wake-word-voice-assistant/esp32-s3-box-3.yaml
esphome:
name: ${name}
name_add_mac_suffix: false
friendly_name: ${friendly_name}
api:
encryption:
key: [redacted]
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
voice_assistant:
on_tts_end:
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.the_kitchen
media_content_id: !lambda 'return x;'
media_content_type: music
announce: "true"
font:
- file:
type: gfonts
family: Orbitron
weight: 500
glyphs: "0123456789:"
id: font_large
# bpp: 4
size: 95
text_sensor:
- id: text_time
platform: homeassistant
entity_id: sensor.local_clock
on_value:
then:
- component.update: s3_box_lcd
display:
- id: !extend s3_box_lcd
pages:
- id: !extend idle_page
lambda: |-
it.fill(id(loading_color));
it.printf(160, 120, id(font_large), id(listening_color), TextAlign::CENTER, "%s", id(text_time).state.c_str());
This requires a template string sensor in HA which is super simple to create:
{{ now().timestamp() | timestamp_custom('%-I:%M') }}
Now that i have those all working, im gonna start focusing on my korvo 1.1s. Hopefully i can get that all working.
How do you broadcast it to your sonos without the amp etc?
Are you talking about piping the TTS out to a different speaker? I do that through HA.
Voice Assistant/ESPHome > HA media_player > speaker.
@Alextrical did you ever print a case for your korvo 1.1? If so, mind sharing updates and/or print files?
I’m still using that older model posted. Printed on a bluish PETG CF combo. Took apart an old Alexa. It just fits but no way to route the 3.5mm jack anywhere and didn’t feel like doing that. That ribbon cable freaks me out sometimes. Literally the worst place for it on the korvo-1 (I don’t there’s a huge hardware payout difference between the 2, outside the -1 has 2 micro USB ports, one for power and one for UART). That Alexa had a surprisingly well built speaker that I’ll be snagging. I should know how to cut a hole out for the 3.5mm audio jack in a slicer but haven’t yet.
I thought you had to allow the ESPHome device to make HA service calls. If you go to devices, then ESPHome where the list shows up and each device has a “configure” option, you get the below
ESPHome devices can make service calls to any Home Assistant service. This functionality is not enabled by default for newly configured device, but can be turned on the options flow on a per device basis.
Yup, has to be done but then you could use one of the 2 below services to output audio, or should be able to.
e.io/components/api.html#homeassistant-service-action
Yep, that will allow the communication between esphome and HA but you also need to configure the on_tts_end to pipe the wav to a media player which is what i was talking about.
Okay, that makes total sense now. I was just a bit confused. I don’t think you used to have to specifically allow a device to make HA service calls from ESPHome. I think that might have been added at some point but if/when it was I’m not sure.
Did anyone else have any issues with the latest core update? All my voice assistants, regardless of microwakeword just stopped working. This included Korvo-1, USB speakerphone hooked directly to HA server using Assist Microphone and Openeakeword, and Android phone which I have to long press the power button so no wake word involved. They all reply but nothing happens. Like turning a light on, it says “okay” but it doesn’t trigger actions.
I always take a full backup before a core upgrade so restring is the solution for now. Lastly, has any tried the below board? It seems like a Korvo V1.1 clone but doesn’t use a ribbon cable. There is a male pin connector on the bottom board and a female connector on the bottom of the mic array board. While it only has 4MB ROM, that’s still more than enough for me. It does have 8MB of external PSRAM. Seems like a bargain for the price.
ESP32-LyraTD-MSC Development Board ESP32 WiFi Bluetooth-compatible Audio Module ESP32-WROVER-E 4MB Flash 8MB P-SRAM
https://a.aliexpress.com/_mK98axG
My devices are all working currently although im just using 2 box-s3’s. I havent plugged my korvo 1.1’s in recently as i was hoping for things to harden and for someone smarter than me to figure out the secret sauce. I’ve heard a few folks talking about the lyra boards and have had some success with them. The pin seems like such a better design than the ribbon cables for sure.
Edit: ok so i tested my boxes after updating HA and EspHome and what im seeing happening is the wakeword not stopping after executing an action. Its just stuck listening until I can ask it again to cancel. Then it goes back to idle.
Hmm. Original issue was some old deprecated stuff for the recorder in my configuration.yaml and then everything started working although some things have been flaky. Might roll back. Just an FYI, the I2/tts doesn’t work with esp_adf which to my knowledge is required for microwakeword. When trying to do the below (per the docs) and from the HA forums to actually send audio to an external speaker. Once I got the config correct it instantly went red. While hovering over it it said “i2s_audio doesn’t work with esp_adf” or something similar and per the docs you have to install the i2s_audio component now to play on an external speaker for audio.
Today’s release of Music Assistant 2.0 has me even more stoked for this project! It would be awesome if the device ends up being supported as a player by Music Assistant.
Yep you’re right. I meant openWakeWord vs microWakeWord. My bad.
Hi, I’ve recently completed this:
It needs some minor tweaks but will be being uploaded here: GitHub - joey-90/ESP32-S3-Korvo-1---Voice-Assistant: Yaml configuration for ESP Home using an Espressif ESP32-S3-KORVO-1 board. shortly.
Looks very nice, looking forward to seeing construction details.
where did you get that case? did you design it?