Voice Assistant esphome with Esp32-s3 - Max98357 - Inmp441

v2 is a standard normal version that works without problems (at least until a couple of weeks ago but I think it still works).

v3 is a similar version (based on the code of mchk user but with a led light) that allows you to turn down the music if you give the wakeword (if you also use it as a music player). Also every time it finishes a response it will always remain listening for a few seconds always (even if the response does not imply another command) which may or may not be convenient depends on the use.

they both work well It depends on your usage

Thanks for your feedback, I will try v3 here

In version V3 - I want to have led light effect when the speaker is “responding/replying”, how do I edit it? Thanks
image

V3 is more complicated for me because there are scripts related to the LED actions and for me, besides, I’m not at home it would require hours of testing… V2 is definitely easier to manage.
I think the V2 has nicer effects though.

During adaptation the author missed the launch of the - script.execute: control_leds action. add it to the on_tts_end trigger.

I decide to start over again from the latest a fresh respeaker lite code

esp32-s3-devkitc-1-esp-idf-v4.yaml

Great, I’ll try it out right away - is there still a chat continuation feature in this version?

after some problems due perhaps to the network… or to the wifi signal… after restarting HA, the satellite and the router, it seems to work.
I’m evaluating if more or less than v3.
For now I have two devices with the two versions installed.

Thanks for sharing

I am testing it. and it’s not.
I mean…
the continuous conversation works if you ask a question that requires a further question from the assistant BUT it does not always listen like the V3.

Personally I think it can be convenient in both cases.
With the V3 that remains always active sometimes it could be a problem if there are other people talking and sometimes VERY convenient for making other requests and if you want to keep a longer and more open conversation (which sometimes is not a bad thing).

I don’t know which one could be more convenient for me… it’s a question of habits and needs.

For now I’m doing a test with the two devices and we’ll see…

I decide to remove it, in my testings with 2 voice assistant in the room they start talking in a loop forever, also if I was watching the TV/YouTube or other peoples in the room talking never stop listening and responding, so I think is better to only listening when is necessary

also, if you are using an AI (I’m using Genesis and ChatGPT) you can ask to do multiple tings at the same time

like Turn on light bedroom and turn off light bathroom and give me the temperature of kitchen

it will do all the thing, so the listening after a request is not necessary in my case

I don’t have reboots, what pins are you using? And do you have another sensors?

In one of mine I add a ldr and a dth11, I need to do some code to avoid reboot because of the update intervals of the sensors

No. I was just thinking about adding a LD2410c but I wanted to find the right configuration first… to make sure the problems couldn’t be due to multiple factors.

initially yes, as soon as I turned it on I had problems but I think it was due to the wifi connection. So far no problems for a few hours.

Which pins?

  i2s_lrclk: GPIO06
  i2s_bclk: GPIO07
  i2s_ws: GPIO03
  i2s_sck: GPIO02
  mic: GPIO04
  spk: GPIO08
  ledpin: GPIO09
  vol_up: GPIO21
  vol_down: GPIO47

I also have the Respeaker and honestly I find it at the same level…but it definitely costs much less

uart:
  id: ld2410_uart
  tx_pin: GPIO17 # Cambia esto a un pin disponible en tu ESP32-S3
  rx_pin: GPIO16 # Cambia esto a un pin disponible en tu ESP32-S3
  baud_rate: 256000 # Velocidad de baudios predeterminada del LD2410c

sensor:
  - platform: ld2410
    uart_id: ld2410_uart
    has_target:
      name: "Radar Presencia"
    has_moving_target:
      name: "Radar Movimiento"
    has_still_target:
      name: "Radar Persona Quieta"

I personally like more the LDR2450 you can make zones and that’s something that’s really useful

external_components:
  - source: github://TillFleisch/ESPHome-HLK-LD2450@main # O la rama/tag más reciente
    components: [ ld2450 ]

uart:
  id: ld2450_uart
  tx_pin: GPIO17
  rx_pin: GPIO18
  baud_rate: 256000

sensor:
  - platform: ld2450
    uart_id: ld2450_uart
    # Estas son las entidades básicas de conteo
    target_count:
      name: "Personas Detectadas"
    still_target_count:
      name: "Personas Quietas"
    moving_target_count:
      name: "Personas en Movimiento"

    # Puedes obtener sensores individuales para cada objetivo
    target_1:
      x_coordinate:
        name: "Target 1 X"
      y_coordinate:
        name: "Target 1 Y"
      speed:
        name: "Target 1 Velocidad"
      presence:
        name: "Target 1 Presente"
    target_2:
      x_coordinate:
        name: "Target 2 X"
      y_coordinate:
        name: "Target 2 Y"
      speed:
        name: "Target 2 Velocidad"
      presence:
        name: "Target 2 Presente"
    target_3:
      x_coordinate:
        name: "Target 3 X"
      y_coordinate:
        name: "Target 3 Y"
      speed:
        name: "Target 3 Velocidad"
      presence:
        name: "Target 3 Presente"

    # Opcional: Configurar zonas de detección para mayor precisión
    # zones:
    #   - name: "Zona Cama"
    #     polygon: # Define un polígono de la zona
    #       - x: 1.0 # Coordenadas en metros
    #         y: 0.5
    #       - x: 2.0
    #         y: 0.5
    #       - x: 2.0
    #         y: 1.5
    #       - x: 1.0
    #         y: 1.5
    #     presence:
    #       name: "Presencia en Cama"
    # Puedes añadir múltiples zonas, incluyendo zonas de exclusión.

OT: are u using also an Atom Echo?
it seems like after the last update I’m having problems… I use this yaml code:

I saw a version somewhere that controls the volume based on the noise in the room. But I can’t find it anymore, does anyone have a link?

you should look in the blueprints maybe…

1 Like

i use this code for my m5stak no problem in esphome 2025.5.2 and HAS 2025.6

substitutions:
  name: m5stack-atom-echo
  friendly_name: M5Stack Atom Echo

esphome:
  name: ${name}
  friendly_name: ${friendly_name}
  min_version: 2025.5.0

esp32:
  board: m5stack-atom
  cpu_frequency: 240MHz
  framework:
    type: esp-idf

logger:
api:
  encryption:
    key: N91its1/e6ELWr3rENYf9lCNlJvfQyuD65hQR+Fj4b0=

ota:
  - platform: esphome
    id: ota_esphome

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password  
  ap:

captive_portal:

button:
  - platform: factory_reset
    id: factory_reset_btn
    name: Factory reset

i2s_audio:
  - id: i2s_audio_bus
    i2s_lrclk_pin: GPIO33
    i2s_bclk_pin: GPIO19

microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO23
    adc_type: external
    pdm: true
    sample_rate: 16000
    correct_dc_offset: true

speaker:
  - platform: i2s_audio
    id: echo_speaker
    i2s_dout_pin: GPIO22
    dac_type: external
    bits_per_sample: 16bit
    sample_rate: 16000
    channel: stereo  # The Echo has poor playback audio quality when using mon audio
    buffer_duration: 60ms

media_player:
  - platform: speaker
    name: None
    id: echo_media_player
    announcement_pipeline:
      speaker: echo_speaker
      format: WAV
    codec_support_enabled: false
    buffer_size: 6000
    volume_min: 0.4
    files:
      - id: timer_finished_wave_file
        file: https://github.com/esphome/wake-word-voice-assistants/raw/main/sounds/timer_finished.wav
    on_announcement:
      - if:
          condition:
            - microphone.is_capturing:
          then:
            - script.execute: stop_wake_word
      - light.turn_on:
          id: led
          blue: 100%
          red: 0%
          green: 0%
          brightness: 100%
          effect: none
    on_idle:
      - script.execute: start_wake_word
      - script.execute: reset_led

voice_assistant:
  id: va
  micro_wake_word:
  microphone:
    microphone: echo_microphone
    channels: 0
    gain_factor: 4
  media_player: echo_media_player
  noise_suppression_level: 2
  auto_gain: 31dBFS
  on_listening:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Slow Pulse"
  on_stt_vad_end:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Fast Pulse"
  on_tts_start:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: none
  on_end:
    # Handle the "nevermind" case where there is no announcement
    - wait_until:
        condition:
          - media_player.is_announcing:
        timeout: 0.5s
    # Restart only mWW if enabled; streaming wake words automatically restart
    - if:
        condition:
          - lambda: return id(wake_word_engine_location).state == "On device";
        then:
          - wait_until:
              - and:
                  - not:
                      voice_assistant.is_running:
                  - not:
                      speaker.is_playing:
          - lambda: id(va).set_use_wake_word(false);
          - micro_wake_word.start:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led
        red: 100%
        green: 0%
        blue: 0%
        brightness: 100%
        effect: none
    - delay: 2s
    - script.execute: reset_led
  on_client_connected:
    - delay: 2s  # Give the api server time to settle
    - script.execute: start_wake_word
  on_client_disconnected:
    - script.execute: stop_wake_word
  on_timer_finished:
    - script.execute: stop_wake_word
    - wait_until:
        not:
          microphone.is_capturing:
    - switch.turn_on: timer_ringing
    - light.turn_on:
        id: led
        red: 0%
        green: 100%
        blue: 0%
        brightness: 100%
        effect: "Fast Pulse"
    - wait_until:
        - switch.is_off: timer_ringing
    - light.turn_off: led
    - switch.turn_off: timer_ringing

binary_sensor:
  # button does the following:
  # short click - stop a timer
  # if no timer then restart either microwakeword or voice assistant continuous
  - platform: gpio
    pin:
      number: GPIO39
      inverted: true
    name: Button
    disabled_by_default: true
    entity_category: diagnostic
    id: echo_button
    on_multi_click:
      - timing:
          - ON for at least 50ms
          - OFF for at least 50ms
        then:
          - if:
              condition:
                switch.is_on: timer_ringing
              then:
                - switch.turn_off: timer_ringing
              else:
                - script.execute: start_wake_word
      - timing:
          - ON for at least 10s
        then:
          - button.press: factory_reset_btn

light:
  - platform: esp32_rmt_led_strip
    id: led
    name: None
    disabled_by_default: true
    entity_category: config
    pin: GPIO27
    default_transition_length: 0s
    chipset: SK6812
    num_leds: 1
    rgb_order: grb
    effects:
      - pulse:
          name: "Slow Pulse"
          transition_length: 250ms
          update_interval: 250ms
          min_brightness: 50%
          max_brightness: 100%
      - pulse:
          name: "Fast Pulse"
          transition_length: 100ms
          update_interval: 100ms
          min_brightness: 50%
          max_brightness: 100%

script:
  - id: reset_led
    then:
      - if:
          condition:
            - lambda: return id(wake_word_engine_location).state == "On device";
            - switch.is_on: use_listen_light
          then:
            - light.turn_on:
                id: led
                red: 100%
                green: 89%
                blue: 71%
                brightness: 60%
                effect: none
          else:
            - if:
                condition:
                  - lambda: return id(wake_word_engine_location).state != "On device";
                  - switch.is_on: use_listen_light
                then:
                  - light.turn_on:
                      id: led
                      red: 0%
                      green: 100%
                      blue: 100%
                      brightness: 60%
                      effect: none
                else:
                  - light.turn_off: led
  - id: start_wake_word
    then:
      - if:
          condition:
            and:
              - not:
                  - voice_assistant.is_running:
              - lambda: return id(wake_word_engine_location).state == "On device";
          then:
            - lambda: id(va).set_use_wake_word(false);
            - micro_wake_word.start:
      - if:
          condition:
            and:
              - not:
                  - voice_assistant.is_running:
              - lambda: return id(wake_word_engine_location).state == "In Home Assistant";
          then:
            - lambda: id(va).set_use_wake_word(true);
            - voice_assistant.start_continuous:
  - id: stop_wake_word
    then:
      - if:
          condition:
            lambda: return id(wake_word_engine_location).state == "In Home Assistant";
          then:
            - lambda: id(va).set_use_wake_word(false);
            - voice_assistant.stop:
      - if:
          condition:
            lambda: return id(wake_word_engine_location).state == "On device";
          then:
            - micro_wake_word.stop:

switch:
  - platform: template
    name: Use listen light
    id: use_listen_light
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - script.execute: reset_led
    on_turn_off:
      - script.execute: reset_led
  - platform: template
    id: timer_ringing
    optimistic: true
    restore_mode: ALWAYS_OFF
    on_turn_off:
      # Turn off the repeat mode and disable the pause between playlist items
      - lambda: |-
              id(echo_media_player)
                ->make_call()
                .set_command(media_player::MediaPlayerCommand::MEDIA_PLAYER_COMMAND_REPEAT_OFF)
                .set_announcement(true)
                .perform();
              id(echo_media_player)->set_playlist_delay_ms(speaker::AudioPipelineType::ANNOUNCEMENT, 0);
      # Stop playing the alarm
      - media_player.stop:
          announcement: true
    on_turn_on:
      # Turn on the repeat mode and pause for 1000 ms between playlist items/repeats
      - lambda: |-
            id(echo_media_player)
              ->make_call()
              .set_command(media_player::MediaPlayerCommand::MEDIA_PLAYER_COMMAND_REPEAT_ONE)
              .set_announcement(true)
              .perform();
            id(echo_media_player)->set_playlist_delay_ms(speaker::AudioPipelineType::ANNOUNCEMENT, 1000);
      - media_player.speaker.play_on_device_media_file:
          media_file: timer_finished_wave_file
          announcement: true
      - delay: 15min
      - switch.turn_off: timer_ringing

select:
  - platform: template
    entity_category: config
    name: Wake word engine location
    id: wake_word_engine_location
    optimistic: true
    restore_value: true
    options:
      - In Home Assistant
      - On device
    initial_option: On device
    on_value:
      - if:
          condition:
            lambda: return x == "In Home Assistant";
          then:
            - micro_wake_word.stop:
            - delay: 500ms
            - lambda: id(va).set_use_wake_word(true);
            - voice_assistant.start_continuous:
      - if:
          condition:
            lambda: return x == "On device";
          then:
            - lambda: id(va).set_use_wake_word(false);
            - voice_assistant.stop:
            - delay: 500ms
            - micro_wake_word.start:

micro_wake_word:
  on_wake_word_detected:
    - voice_assistant.start:
        wake_word: !lambda return wake_word;
  vad:
  models:
    - model: okay_nabu
    - model: hey_mycroft
    - model: hey_jarvis

yes it’s seems the same that i posted… I don’t know it seems like I’m having some problems with this setup

what error do you have?

I have tested many times, there is one problem here is the output sound suddenly stops while announcing - text to speech. It seems to be the problem of the latest esphome. Do you encounter such an error ?

  • checked in debug - text content is complete but sound stops suddenly,…

Are you using additional sensors?
what device you are using ?
what code do you hav?