Far field satellite with an Elegant 3d printed enclosures

Correct. It had been working for a while but i changed my network mode for some other reason not associated to voice. Then i read host mode only and took a peek.

1 Like

Here’s the korvo-1 (not V1. 1) with continued conversation. This allows it to listen for multiple commands. The slider is for how many seconds pass after hearing the last command then it stops. I need to snag the URL of the site I got the yaml for. You have to use a subscription for the wake word model or it doesn’t work as it’s used in scripts and other stuff.

micro_wake_word:
model: ${micro_wake_word_model}

substitutions:
  name: "korvo"
  friendly_name: korvo

  voice_assist_idle_phase_id: "1"
  voice_assist_listening_phase_id: "2"
  voice_assist_thinking_phase_id: "3"
  voice_assist_replying_phase_id: "4"
  voice_assist_not_ready_phase_id: "10"
  voice_assist_error_phase_id: "11"
  voice_assist_muted_phase_id: "12"

  micro_wake_word_model: hey_jarvis

esphome:
  name: ${name}
  friendly_name: ${friendly_name}
  name_add_mac_suffix: true
  platformio_options:
    board_build.flash_mode: dio
    upload_speed: 460800
  project:
    name: esphome.voice-assistant
    version: "1.0"
  min_version: 2023.11.5
  on_boot:
    - priority: 600
      then:
        - light.turn_on:
            id: led_ring
            red: 0%
            blue: 0%
            green: 100%
            brightness: 100%
            effect: random
        - if:
            condition:
              lambda: return id(init_in_progress);
            then:
              - lambda: id(init_in_progress) = false;

esp32:
  board: esp32-s3-devkitc-1
  flash_size: 16MB
  framework:
    type: esp-idf
    sdkconfig_options:
      CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y"
      CONFIG_ESP32S3_DATA_CACHE_64KB: "y"
      CONFIG_ESP32S3_DATA_CACHE_LINE_64B: "y"
      CONFIG_AUDIO_BOARD_CUSTOM: "y"
      CONFIG_ESP32_S3_KORVO1_BOARD: "y"
    components:
      - name: esp32_s3_korvo1_board
        #source: github://espressif/components/hardware_driver@main
        source: github://abmantis/esphome_custom_audio_boards@main
        refresh: 0s

psram:
  mode: octal
  speed: 80MHz

external_components:
  - source: github://pr#5230
    components: esp_adf

ota:
logger:
api:
  encryption:
     key: YOU_API_KEY
  on_client_connected:
    then:
      - if:
          condition:
            switch.is_off: mute
          then:
            - delay: 20ms
            - ble.disable:
      - light.turn_on:
          id: led_ring
          blue: 0%
          red: 0%
          green: 100%
          brightness: 50%
          effect: led12              
  on_client_disconnected:
    then:
      - ble.enable
      - light.turn_on:
          id: led_ring
          blue: 0%
          red: 100%
          green: 100%
          brightness: 50%
          effect: connecting

dashboard_import:
  package_import_url: github://esphome/firmware/voice-assistant/esp32-s3-korvo1.yaml@main

wifi:
  ssid: !secret wifi_ssid 
  password: !secret wifi_password
  use_address: 192.168.0.48
  ap:
  on_connect:
    then:
      - delay: 20ms # Gives time for improv results to be transmitted
      - ble.disable:
      - delay: 5s
  on_disconnect:
    then:
      - ble.enable:

improv_serial:

esp32_improv:
  authorizer: none

button:
  - platform: factory_reset
    id: factory_reset_btn
    name: Factory reset

esp_adf:
  board: esp32s3korvo1

microphone:
  - platform: esp_adf
    id: korvo_mic

speaker:
  - platform: esp_adf
    id: korvo_speaker

micro_wake_word:
  model: ${micro_wake_word_model}
  on_wake_word_detected:
    - if:
        condition:
          switch.is_on: continued_conversation
        then:
          - lambda: id(va).set_use_wake_word(false);
          - voice_assistant.start_continuous: # wake_word: !lambda return wake_word; not yet supported for continuous
        else:
          - lambda: id(va).set_use_wake_word(false);
          - voice_assistant.start:
              wake_word: !lambda return wake_word;
    - light.turn_on:
        id: led_ring      
        red: 30%
        green: 30%
        blue: 70%
        brightness: 100%
        effect: wakeword
  
voice_assistant:
  id: va
  microphone: korvo_mic
  speaker: korvo_speaker
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 3.0
  vad_threshold: 3
  on_listening:     
    - lambda: id(voice_assistant_phase) = ${voice_assist_listening_phase_id};    
    - light.turn_on:
        id: led_ring
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: led12
  #   - script.execute: reset_led    
    - if:
        condition:
          - switch.is_on: continued_conversation
        then:
          - script.execute: stt_timeout_to_idle
          - lambda: id(va).set_use_wake_word(false);  
  on_stt_vad_end:
    - lambda: id(voice_assistant_phase) = ${voice_assist_thinking_phase_id};
    - script.execute: reset_led
    - if:
        condition:
          - switch.is_on: continued_conversation
        then:
          - script.execute: stt_timeout_to_idle    
    - light.turn_on:
        id: led_ring
        blue: 0%
        red: 50%
        green: 50%
        brightness: 100%        
        effect: working
  #  - script.execute: reset_led    
  on_tts_stream_start:
    - lambda: id(voice_assistant_phase) = ${voice_assist_replying_phase_id};
    - light.turn_on:
        id: led_ring
        blue: 50%
        red: 50%
        green: 0%
        brightness: 100%        
        effect: pulse
    #- script.execute: reset_led    
  on_tts_stream_end:
    - if:
        condition:
          - switch.is_on: continued_conversation
        then:
          - lambda: id(voice_assistant_phase) = ${voice_assist_listening_phase_id};
          - script.execute: stt_timeout_to_idle
        else:
          - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
    - light.turn_on:
        id: led_ring
        blue: 0%
        red: 0%
        green: 100%
        brightness: 20%        
        effect: connecting
   # - script.execute: reset_led    
  on_end:
    - if:
        condition:
          and:
            - switch.is_off: mute
            - lambda: return id(wake_word_engine_location).state == "On device";
            - switch.is_off: continued_conversation
        then:
          - script.execute: return_to_idle
  on_error:
    - if:
        condition:
          and:
            - lambda: return !id(init_in_progress);
            - lambda: return !(code == "stt-no-text-recognized");
        then:
          - lambda: id(voice_assistant_phase) = ${voice_assist_error_phase_id};
          - script.execute: reset_led
          - delay: 1s
          - script.stop: stt_timeout_to_idle
          - if:
              condition:
                switch.is_off: mute
              then:
                # - logger.log: "(on_client_connected) Returning to idle by script"
                - script.execute: return_to_idle
              else:
                - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
          - script.execute: reset_led
  on_client_connected:
    - if:
        condition:
          switch.is_off: mute
        then:
          - wait_until:
              not: ble.enabled
          - script.execute: return_to_idle
        else:
          - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
    - lambda: id(init_in_progress) = false;
    - script.execute: reset_led
  on_client_disconnected:
    - if:
        condition:
          lambda: return id(wake_word_engine_location).state == "In Home Assistant";
        then:
          - lambda: id(va).set_use_wake_word(false);
          - voice_assistant.stop:
    - if:
        condition:
          lambda: return id(wake_word_engine_location).state == "On device";
        then:
          - micro_wake_word.stop
    - lambda: id(voice_assistant_phase) = ${voice_assist_not_ready_phase_id};
    
    
script:
  - id: stt_timeout_to_idle
    mode: restart     # restart the timer if called before timeout
    then:
      if:
        condition:
          switch.is_on: continued_conversation
        then:
          - delay: !lambda "return id(continued_timeout).state * 1000;"
          - if:
              condition:
                lambda: return (id(voice_assistant_phase) == ${voice_assist_replying_phase_id});
              then:
                - wait_until:
                    condition:
                      lambda: return !(id(voice_assistant_phase) == ${voice_assist_replying_phase_id});
                      # normally this would complete and move to next phase with on_tts_stream_end,
                      # but sometimes this is missed so put a time limit on the wait
                    timeout: 10s
                - delay: 1s  # Give time for the stream to end and the phase to be switched back to listening and this timeout to be reset
          - script.execute: return_to_idle
          
  - id: return_to_idle
    then:
      - if:
          condition:
            lambda: return id(wake_word_engine_location).state == "On device";
          then:
            - script.stop: stt_timeout_to_idle
            - if:
                condition:
                  voice_assistant.is_running
                then:
                  - lambda: id(va).set_use_wake_word(false);
                  - voice_assistant.stop
                  - wait_until:
                      condition:
                        not:
                          voice_assistant.is_running
                      timeout: 5s
            - if:
                condition:
                  not:
                    micro_wake_word.is_running
                then:
                  - micro_wake_word.start:
      - if:
          condition:
            lambda: return id(wake_word_engine_location).state == "In Home Assistant";
          then:
            - if:
                condition:
                  micro_wake_word.is_running
                then:
                  - micro_wake_word.stop:
                  - wait_until:
                      condition:
                        not:
                          micro_wake_word.is_running
                      timeout: 5s
            - wait_until:
                condition:
                  not:
                    voice_assistant.is_running
                timeout: 5s
            - lambda: id(va).set_use_wake_word(true);
            - if:
                condition:
                  - not:
                      voice_assistant.is_running
                then:
                  - voice_assistant.start_continuous:
      - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};      
      
  - id: reset_led
    then:
      - if:
          condition:
            switch.is_off: mute
          then:
            - light.turn_on:
                id: led_ring
                blue: 100%
                red: 100%
                green: 0%
                brightness: 100%
                effect: connecting
          else:
            - light.turn_off: led_ring

switch:
  - platform: gpio
    id: pa_ctrl
    pin: GPIO38
    name: "${friendly_name} Speaker Mute"
    restore_mode: ALWAYS_ON

  - platform: template
    name: Mute
    id: mute
    optimistic: true
    restore_mode: RESTORE_DEFAULT_OFF
    entity_category: config
    on_turn_off:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - script.execute: return_to_idle
    on_turn_on:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - lambda: id(va).set_use_wake_word(false);
            - if:
                condition:
                  - voice_assistant.is_running
                then:
                  - voice_assistant.stop
            - if:
                condition:
                  - micro_wake_word.is_running
                then:
                  - micro_wake_word.stop
            - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
            - script.execute: reset_led
            
  - platform: template
    name: Continued conversation
    id: continued_conversation
    optimistic: true
    restore_mode: RESTORE_DEFAULT_OFF
    entity_category: config
    on_turn_off:
      - script.execute: return_to_idle
    on_turn_on:
      - script.execute: return_to_idle
      
  - platform: restart
    name: "korvo restart"        

number:
  - platform: template
    entity_category: config
    name: Continued timeout
    id: continued_timeout
    icon: mdi:clock
    optimistic: true
    restore_value: true
    initial_value: 8
    min_value: 1
    step: 1
    max_value: 20
    unit_of_measurement: s
    mode: slider
 

select:
  - platform: template
    entity_category: config
    name: Wake word engine location
    id: wake_word_engine_location
    optimistic: true
    restore_value: true
    options:
      - In Home Assistant
      - On device
    initial_option: On device
    on_value:
      - wait_until:
          lambda: return id(voice_assistant_phase) == ${voice_assist_muted_phase_id} || id(voice_assistant_phase) == ${voice_assist_idle_phase_id};
      - script.execute: return_to_idle

globals:
  - id: init_in_progress
    type: bool
    restore_value: false
    initial_value: "true"
  - id: voice_assistant_phase
    type: int
    restore_value: false
    initial_value: ${voice_assist_not_ready_phase_id}

light:
  - platform: esp32_rmt_led_strip
    id: led_ring
    is_rgbw: true
    rgb_order: GRB    
    pin: GPIO19
    num_leds: 12
    rmt_channel: 0
    chipset: WS2812
    name: "${friendly_name} Light"
    default_transition_length: 1s
    effects:
      - addressable_scan:
          name: "led12"
          move_interval: 10ms
          scan_width: 12
      - pulse:
          name: "pulse"
          transition_length: 0.5s
          update_interval: 0.5s
      - addressable_twinkle:
          name: "working"
          twinkle_probability: 5%
          progress_interval: 4ms
      - addressable_color_wipe:
          name: "wakeword"
          colors:
            - red: 0%
              green: 0%
              blue: 100%
              num_leds: 12
          add_led_interval: 20ms
          reverse: false
      - addressable_color_wipe:
          name: "connecting"
          colors:
            - red: 40%
              green: 30%
              blue: 30%
              num_leds: 12
          add_led_interval: 50ms
          reverse: true

binary_sensor:
  - platform: template
    name: "${friendly_name} Volume Up"
    id: btn_volume_up
  - platform: template
    name: "${friendly_name} Volume Down"
    id: btn_volume_down
  - platform: template
    name: "${friendly_name} Set"
    id: btn_set
    on_multi_click:
      - timing:
          - ON for at least 10s
        then:
          - button.press: factory_reset_btn    
  - platform: template
    name: "${friendly_name} Play"
    id: btn_play
  - platform: template
    name: "${friendly_name} Mode"
    id: btn_mode
    on_press:
      - voice_assistant.start_continuous:
  - platform: template
    name: "${friendly_name} Record" 
    id: btn_record
    on_press:
      - voice_assistant.start:
      #- lambda: id(va).set_use_wake_word(true);
          
sensor:
  - id: button_adc
    platform: adc
    internal: true
    pin: 8
    attenuation: 11db
    update_interval: 15ms
    filters:
      - median:
          window_size: 5
          send_every: 5
          send_first_at: 1
      - delta: 0.1
    on_value_range:
      - below: 0.55
        then:
          - binary_sensor.template.publish:
              id: btn_volume_up
              state: ON
      - above: 0.65
        below: 0.92
        then:
          - binary_sensor.template.publish:
              id: btn_volume_down
              state: ON
      - above: 1.02
        below: 1.33
        then:
          - binary_sensor.template.publish:
              id: btn_set
              state: ON
      - above: 1.43
        below: 1.77
        then:
          - binary_sensor.template.publish:
              id: btn_play
              state: ON
      - above: 1.87
        below: 2.15
        then:
          - binary_sensor.template.publish:
              id: btn_mode
              state: ON
      - above: 1.01
        below: 2.56
        then:
          - binary_sensor.template.publish:
              id: btn_record
              state: ON
      - above: 2.3
        then:
          - binary_sensor.template.publish:
              id: btn_volume_up
              state: OFF
          - binary_sensor.template.publish:
              id: btn_volume_down
              state: OFF
          - binary_sensor.template.publish:
              id: btn_set
              state: OFF
          - binary_sensor.template.publish:
              id: btn_play
              state: OFF
          - binary_sensor.template.publish:
              id: btn_mode
              state: OFF
          - binary_sensor.template.publish:
              id: btn_record
              state: OFF

Have you added the media_player itself? It appears ID is a required field also according to the docs but I have not tried to use it as a media player either

All media_player actions can be used without specifying an id if you have only one media_player in your configuration YAML

So continued conversation is both awesome and amazing depending on the situation. It works flawlessly with no audio source near it like a TV or music. If you do, the Korvo picks up that audio and kind of gets stuck in a loop of “sorry, I didn’t understand that” because it picks up audio from TV/Music and will just start saying that over and over. There was an enhancement request for full support but the ESPHome team has been busy fixing issues and bug requests always get fixed before engagement requests for obvious reasons. That and most the examples are for the S3 box variants which have a screen so you have to remove all that stuff. If they added something like a “stop listening” command, which should be somewhat easy to do, it would be way better because it would fix the external audio loop issue. It is nice to be able to tell it multiple commands without having to say the trigger word every single time.

One small issue is I e had it lock up twice but only after applying the latest core update yesterday, before that it didn’t freeze up so I’m not sure if I got lucky and didn’t have any lock ups before the update or if the core update caused the issue.

Enhancement request

ESPHome code that I merged into my working config file.

I believe you can say “Cancel” to stop the wake word from detecting. I have a box-3 now and tested that PR out yesterday and it worked, but as you said, if you’re in a loud environment it doesnt work so well. I ended up reverting back for the box-3 to the default but modified the idle screen to display a clock which i think is really handy.

substitutions:
  name: esp32-s3-box-3-6405bc
  friendly_name: Kitchen ESP Box
  micro_wake_word_model: alexa
packages:
  esphome.voice-assistant: github://esphome/firmware/wake-word-voice-assistant/esp32-s3-box-3.yaml
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
api:
  encryption:
    key: [redacted]
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

voice_assistant:
  on_tts_end:
    - homeassistant.service:
        service: media_player.play_media
        data:
          entity_id: media_player.the_kitchen
          media_content_id: !lambda 'return x;'
          media_content_type: music
          announce: "true"

font:
  - file: 
      type: gfonts
      family: Orbitron
      weight: 500
    glyphs: "0123456789:"
    id: font_large
    # bpp: 4
    size: 95

text_sensor:
  - id: text_time
    platform: homeassistant
    entity_id: sensor.local_clock
    on_value:
      then:
        - component.update: s3_box_lcd

display:
  - id: !extend s3_box_lcd
    pages:
      - id: !extend idle_page
        lambda: |-
          it.fill(id(loading_color));
          it.printf(160, 120, id(font_large), id(listening_color), TextAlign::CENTER, "%s", id(text_time).state.c_str()); 

This requires a template string sensor in HA which is super simple to create:

{{ now().timestamp() | timestamp_custom('%-I:%M') }} 

Now that i have those all working, im gonna start focusing on my korvo 1.1s. Hopefully i can get that all working.

How do you broadcast it to your sonos without the amp etc?

Are you talking about piping the TTS out to a different speaker? I do that through HA.
Voice Assistant/ESPHome > HA media_player > speaker.

1 Like

@Alextrical did you ever print a case for your korvo 1.1? If so, mind sharing updates and/or print files?

I’m still using that older model posted. Printed on a bluish PETG CF combo. Took apart an old Alexa. It just fits but no way to route the 3.5mm jack anywhere and didn’t feel like doing that. That ribbon cable freaks me out sometimes. Literally the worst place for it on the korvo-1 (I don’t :speech_balloon: there’s a huge hardware payout difference between the 2, outside the -1 has 2 micro USB ports, one for power and one for UART). That Alexa had a surprisingly well built speaker that I’ll be snagging. I should know how to cut a hole out for the 3.5mm audio jack in a slicer but haven’t yet.





I thought you had to allow the ESPHome device to make HA service calls. If you go to devices, then ESPHome where the list shows up and each device has a “configure” option, you get the below

ESPHome devices can make service calls to any Home Assistant service. This functionality is not enabled by default for newly configured device, but can be turned on the options flow on a per device basis.

Yup, has to be done but then you could use one of the 2 below services to output audio, or should be able to.

https://esphom


e.io/components/api.html#homeassistant-service-action

Yep, that will allow the communication between esphome and HA but you also need to configure the on_tts_end to pipe the wav to a media player which is what i was talking about.

Okay, that makes total sense now. I was just a bit confused. I don’t think you used to have to specifically allow a device to make HA service calls from ESPHome. I think that might have been added at some point but if/when it was I’m not sure.

Did anyone else have any issues with the latest core update? All my voice assistants, regardless of microwakeword just stopped working. This included Korvo-1, USB speakerphone hooked directly to HA server using Assist Microphone and Openeakeword, and Android phone which I have to long press the power button so no wake word involved. They all reply but nothing happens. Like turning a light on, it says “okay” but it doesn’t trigger actions.

I always take a full backup before a core upgrade so restring is the solution for now. Lastly, has any tried the below board? It seems like a Korvo V1.1 clone but doesn’t use a ribbon cable. There is a male pin connector on the bottom board and a female connector on the bottom of the mic array board. While it only has 4MB ROM, that’s still more than enough for me. It does have 8MB of external PSRAM. Seems like a bargain for the price.

ESP32-LyraTD-MSC Development Board ESP32 WiFi Bluetooth-compatible Audio Module ESP32-WROVER-E 4MB Flash 8MB P-SRAM
https://a.aliexpress.com/_mK98axG

My devices are all working currently although im just using 2 box-s3’s. I havent plugged my korvo 1.1’s in recently as i was hoping for things to harden and for someone smarter than me to figure out the secret sauce. I’ve heard a few folks talking about the lyra boards and have had some success with them. The pin seems like such a better design than the ribbon cables for sure.

Edit: ok so i tested my boxes after updating HA and EspHome and what im seeing happening is the wakeword not stopping after executing an action. Its just stuck listening until I can ask it again to cancel. Then it goes back to idle.

Hmm. Original issue was some old deprecated stuff for the recorder in my configuration.yaml and then everything started working although some things have been flaky. Might roll back. Just an FYI, the I2/tts doesn’t work with esp_adf which to my knowledge is required for microwakeword. When trying to do the below (per the docs) and from the HA forums to actually send audio to an external speaker. Once I got the config correct it instantly went red. While hovering over it it said “i2s_audio doesn’t work with esp_adf” or something similar and per the docs you have to install the i2s_audio component now to play on an external speaker for audio.

Today’s release of Music Assistant 2.0 has me even more stoked for this project! It would be awesome if the device ends up being supported as a player by Music Assistant.

3 Likes

Yep you’re right. I meant openWakeWord vs microWakeWord. My bad.

Hi, I’ve recently completed this:

It needs some minor tweaks but will be being uploaded here: GitHub - joey-90/ESP32-S3-Korvo-1---Voice-Assistant: Yaml configuration for ESP Home using an Espressif ESP32-S3-KORVO-1 board. shortly.

6 Likes

Looks very nice, looking forward to seeing construction details.

1 Like

where did you get that case? did you design it?