LEGO R2D2 Voice Assistant

unkyfox · April 23, 2025, 9:53pm

Hello, again!!!
I tried to install the code sent by colleague Mikecoscia, but I had the following error when installing by ESPHOME from HA, could anyone help me?

INFO ESPHome 2025.4.0
INFO Reading configuration /config/esphome/m5stack-atom-r2d2.yaml…
INFO Updating https://github.com/esphome/esphome.git@pull/5230/head
Failed config

light.esp32_rmt_led_strip: [source /config/esphome/m5stack-atom-echo-r2d2.yaml:207]
platform: esp32_rmt_led_strip
id: led
name: None
disabled_by_default: True
entity_category: config
pin: GPIO27
default_transition_length: 0s
chipset: SK6812
num_leds: 1
rgb_order: grb

This feature is not available for the IDF framework version 5.
rmt_channel: 0
effects:
- pulse:

markyskus · May 3, 2025, 12:38pm

Hi, could you please share your whole config again? After the 2025.3 or 2025.4 update it’s unustable. I think the problem with the In Home Assistant wake word

BramNH · May 3, 2025, 1:36pm

I also ran into some issues months back… dont have it up and running currently. I shared my config in an earlier comment, haven’t changed it.

miguelpineiroo · May 4, 2025, 6:36am

Hi @BramNH !
Amazing work. The R2D2 is lovely. I am trying to replicate the setup with the same micro servo and I am not having good luck so far. From the pictures, I understand that you have connected the servo head to a lego piece hole directly, and then you have screwed the servo to the same piece (white) with the help of two lego flat pieces (red + brown), or have you used anything else?
Thanks for your help!
Miguel

markyskus · May 4, 2025, 6:51am

thanks, it works hectic now and I underqualified for solve the problem

BramNH · May 4, 2025, 8:32am

I have only used one flat white piece, the servo head can fit in one of the holes + a small screw to secure. And one red flat piece to attached to the servo with a screw which is used to connect to the body of R2

miguelpineiroo · May 4, 2025, 4:20pm

Hi @BramNH !
Thanks for the clarifications. Not sure of fully understanding it. Just to confirm: so, you used 2 screws:
-Screw 1: red flat piece and servo hole 1 (not attached to flat white piece)
-Screw 2: servo hole 2 and connected to what?

-Servo head attached to one hole of white piece
-Flat red piece not connected with flat white piece thorough its holes
Thanks again !
Miguel

BramNH · May 5, 2025, 8:24am

Lego pieces are not connected. Red piece is used to attach to body. White piece to head. I wouldn’t follow my design that strictly though, its not perfect.

For the white piece, use a very small screw, the servo should come with one for it.

miguelpineiroo · May 5, 2025, 1:55pm

Hi @BramNH ,
Your design is awesome, I have seen the video and it seems to work very well. From your answer it is now clear to me that you use 2 screws and their use:

Screw 1: small screw attached to the flat white piece, I have understood that you use it to join the white piece and the servo head. Do you insert it perpendicularly to the servo head, don’t you?
Screw 2: to join the red piece and the servo body. The other screw hole in the servo body is empty. For this to work, you require to attach the red piece holes in the R2D2 body (black piece).

Do you think I’ve got it?

Thanks again!
Miguel

BramNH · May 5, 2025, 4:13pm

Yes, that’s how I have it, for me it works okay. As I said not perfect. There is a little too much space between the head and body, so not a perfect fit. I also had to remove some Lego pieces from the body that I can remember.

Just try to make a solution that works best for you. This isn’t meant as a tutorial on how to make this.

miguelpineiroo · May 5, 2025, 5:43pm

Hi @BramNH,
Thanks for your quick answer. I will try to be inspired by your setup. I know it is not a perfect fit, but it is a very good starting point!
Thanks again
Miguel!

miguelpineiroo · May 9, 2025, 10:46pm

miguelpineiroo · May 9, 2025, 10:51pm

Just my picture contribution. I finally have it working ! : White piece attached to the body and the servo with screw. Green piece short screw to servo head and attached to head (with a red piece)

BramNH · May 10, 2025, 7:11am

Awesome! Do you also have the software working?

miguelpineiroo · May 12, 2025, 9:08pm

Yes, it’s working perfectly!! There is a bigger Lego R2D2. It may be interesting to explore how to make it work

BramNH · May 13, 2025, 6:07am

Is it the same as my software? Or were there updates required? Could you share if any additional steps had to be taken for the software, so this page is up to date again

miguelpineiroo · May 13, 2025, 7:20am

It is based in your code, but I remember I got some errors, so it needed minor updates, probably due to changes in ESP version. I share my working code, with ESPHome 2025.4.2, using ATOM Echo and the same servo model as you, feeding the servo from the ATOM Echo

miguelpineiroo · May 13, 2025, 7:25am

substitutions:
  name: m5stack-atom-echo-r2d2
  friendly_name: M5Stack Atom Echo r2d2
  micro_wake_word_model: okay_nabu  # alexa, hey_jarvis, hey_mycroft are also supported
 
esphome:
  name: ${name}
  name_add_mac_suffix: true
  friendly_name: ${friendly_name}
  min_version: 2025.2.0
 
external_components:
 - source: github://esphome/[email protected]
   components: [psram]
   
esp32:
  board: m5stack-atom
  framework:
    type: esp-idf
    version: 4.4.8
    platform_version: 5.4.0
 
# Enable logging
logger:
 
api:
  encryption:
    key: xxxxxxxxxxxxx
 
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip: 
    static_ip: xxxxxxxxxxxxx
    gateway: xxxxxxxxxxxxx
    subnet: xxxxxxxxxxxxx
    dns1: xxxxxxxxxxxxx
    dns2: xxxxxxxxxxxxx  ap:
    ssid: "M5Stack-Atom-Echo"
    password: " xxxxxxxxxxxxx "
 
captive_portal:
 
button:
  - platform: factory_reset
    id: factory_reset_btn
    name: Factory reset
 
i2s_audio:
  - id: i2s_audio_bus
    i2s_lrclk_pin: GPIO33
    i2s_bclk_pin: GPIO19
 
microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO23
    adc_type: external
    pdm: true
 
speaker:
  - platform: i2s_audio
    id: echo_speaker
    i2s_dout_pin: GPIO22
    dac_type: external
    bits_per_sample: 32bit
    channel: right
    buffer_duration: 60ms
 
media_player:
  - platform: speaker
    name: None
    id: echo_media_player
    announcement_pipeline:
      speaker: echo_speaker
      format: WAV
    codec_support_enabled: false
    buffer_size: 6000
    volume_min: 0.4
    files:
      - id: timer_finished_wave_file
        file: https://github.com/esphome/wake-word-voice-assistants/raw/main/sounds/timer_finished.wav
    on_announcement:
      - if:
          condition:
            - microphone.is_capturing:
          then:
            - if:
                condition:
                  lambda: return id(wake_word_engine_location).state == "On device";
                then:
                  - micro_wake_word.stop:
                else:
                  - voice_assistant.stop:
            - script.execute: reset_led
      - light.turn_on:
          id: led
          blue: 100%
          red: 0%
          green: 0%
          brightness: 100%
          effect: none
    on_idle:
      - script.execute: start_wake_word
 
voice_assistant:
  id: va
  microphone: echo_microphone
  media_player: echo_media_player
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  on_listening:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Slow Pulse"
  on_stt_vad_end:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Fast Pulse"
  on_tts_start:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: none
    - output.set_level:
        id: salida_servo
        level: 0%
    - delay: 500ms
    - output.set_level:
        id: salida_servo
        level: 100%
 
  on_end:
    - delay: 100ms
    - script.execute: start_wake_word
  on_error:
    - light.turn_on:
        id: led
        red: 100%
        green: 0%
        blue: 0%
        brightness: 100%
        effect: none
    - delay: 2s
    - script.execute: reset_led
  on_client_connected:
    - delay: 2s  # Give the api server time to settle
    - script.execute: start_wake_word
  on_client_disconnected:
    - voice_assistant.stop:
    - micro_wake_word.stop:
  on_timer_finished:
    - voice_assistant.stop:
    - micro_wake_word.stop:
    - wait_until:
        not:
          microphone.is_capturing:
    - switch.turn_on: timer_ringing
    - light.turn_on:
        id: led
        red: 0%
        green: 100%
        blue: 0%
        brightness: 100%
        effect: "Fast Pulse"
    - wait_until:
        - switch.is_off: timer_ringing
    - light.turn_off: led
    - switch.turn_off: timer_ringing
 
binary_sensor:
  # button does the following:
  # short click - stop a timer
  # if no timer then restart either microwakeword or voice assistant continuous
  - platform: gpio
    pin:
      number: GPIO39
      inverted: true
    name: Button
    disabled_by_default: true
    entity_category: diagnostic
    id: echo_button
    on_multi_click:
      - timing:
          - ON for at least 50ms
          - OFF for at least 50ms
        then:
          - if:
              condition:
                switch.is_on: timer_ringing
              then:
                - switch.turn_off: timer_ringing
              else:
                - script.execute: start_wake_word
      - timing:
          - ON for at least 10s
        then:
          - button.press: factory_reset_btn
 
light:
  - platform: esp32_rmt_led_strip
    id: led
    name: None
    disabled_by_default: true
    entity_category: config
    pin: GPIO27
    default_transition_length: 0s
    chipset: SK6812
    num_leds: 1
    rgb_order: grb
    rmt_channel: 0
    effects:
      - pulse:
          name: "Slow Pulse"
          transition_length: 250ms
          update_interval: 250ms
          min_brightness: 50%
          max_brightness: 100%
      - pulse:
          name: "Fast Pulse"
          transition_length: 100ms
          update_interval: 100ms
          min_brightness: 50%
          max_brightness: 100%
 
script:
  - id: reset_led
    then:
      - if:
          condition:
            - lambda: return id(wake_word_engine_location).state == "On device";
            - switch.is_on: use_listen_light
          then:
            - light.turn_on:
                id: led
                red: 100%
                green: 89%
                blue: 71%
                brightness: 60%
                effect: none
          else:
            - if:
                condition:
                  - lambda: return id(wake_word_engine_location).state != "On device";
                  - switch.is_on: use_listen_light
                then:
                  - light.turn_on:
                      id: led
                      red: 0%
                      green: 100%
                      blue: 100%
                      brightness: 60%
                      effect: none
                else:
                  - light.turn_off: led
  - id: start_wake_word
    then:
      - wait_until:
          and:
            - media_player.is_idle:
            - speaker.is_stopped:
      - if:
          condition:
            lambda: return id(wake_word_engine_location).state == "On device";
          then:
            - voice_assistant.stop
            - micro_wake_word.stop:
            - delay: 1s
            - script.execute: reset_led
            - script.wait: reset_led
            - micro_wake_word.start:
          else:
            - if:
                condition: voice_assistant.is_running
                then:
                  - voice_assistant.stop:
                  - script.execute: reset_led
            - voice_assistant.start_continuous:
 
switch:
  - platform: template
    name: Use listen light
    id: use_listen_light
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - script.execute: reset_led
    on_turn_off:
      - script.execute: reset_led
  - platform: template
    id: timer_ringing
    optimistic: true
    restore_mode: ALWAYS_OFF
    on_turn_off:
      # Turn off the repeat mode and disable the pause between playlist items
      - lambda: |-
              id(echo_media_player)
                ->make_call()
                .set_command(media_player::MediaPlayerCommand::MEDIA_PLAYER_COMMAND_REPEAT_OFF)
                .set_announcement(true)
                .perform();
              id(echo_media_player)->set_playlist_delay_ms(speaker::AudioPipelineType::ANNOUNCEMENT, 0);
      # Stop playing the alarm
      - media_player.stop:
          announcement: true
    on_turn_on:
      # Turn on the repeat mode and pause for 1000 ms between playlist items/repeats
      - lambda: |-
            id(echo_media_player)
              ->make_call()
              .set_command(media_player::MediaPlayerCommand::MEDIA_PLAYER_COMMAND_REPEAT_ONE)
              .set_announcement(true)
              .perform();
            id(echo_media_player)->set_playlist_delay_ms(speaker::AudioPipelineType::ANNOUNCEMENT, 1000);
      - media_player.speaker.play_on_device_media_file:
          media_file: timer_finished_wave_file
          announcement: true
      - delay: 15min
      - switch.turn_off: timer_ringing
 
select:
  - platform: template
    entity_category: config
    name: Wake word engine location
    id: wake_word_engine_location
    optimistic: true
    restore_value: true
    options:
      - In Home Assistant
      - On device
    initial_option: On device
    on_value:
      - if:
          condition:
            lambda: return x == "In Home Assistant";
          then:
            - micro_wake_word.stop
            - delay: 500ms
            - lambda: id(va).set_use_wake_word(true);
            - voice_assistant.start_continuous:
      - if:
          condition:
            lambda: return x == "On device";
          then:
            - lambda: id(va).set_use_wake_word(false);
            - voice_assistant.stop
            - delay: 500ms
            - micro_wake_word.start
 
micro_wake_word:
  on_wake_word_detected:
    - output.set_level:
        id: salida_servo
        level: 50%
    - voice_assistant.start:
        wake_word: !lambda return wake_word;
  vad:
  models:
    - model: ${micro_wake_word_model}
 
output:
  - platform: ledc
    pin: 32
    id: salida_servo
    frequency: 50 Hz
    min_power: 5.0%
    max_power: 10.0%
 
servo:
  - id: r2d2servo
    output: salida_servo
 
number:
  - platform: template
    name: Control Servo R2D2
    max_value: 100
    initial_value: 0
    min_value: -100
    step: 1
    optimistic: True
    set_action: 
      then:
        - servo.write:
            id: r2d2servo
            level: !lambda 'return x / 100.0;'
   
 
type or paste code here