R2D2 Satellite

Hello all!

I am working on my ‘smart’ R2D2 for some time and i just finished making it a Home Assistant Satellite. The project has two main goals - first, making the robot drive independently, patrol my house and follow me, recharge when battery is low, detect obstacles, map the rooms - much like a vacuum robot. The second goal is to make it smart and integrate with HA.

This goal is ready now, so i wanna share the current setup here:

The 40cm remote controlled R2D2 toy was gutted and now has a Pi Motor HAT (…had an Arduino with MotorShield) and a Raspberry Pi 4(no, 3…) inside, with a Respeaker 4-mic array in the top dome. I tried other ways to turn it into an assistant, but recently things got a lot easier thanks to HA Focus on Assistants.
I used:

and parts of

So, what does it do? I listens locally to the wakeword ‘R2D2’, then acts as a remote satellite in Home Assistant and reacts to the wakeword with a nice R2-Sound, then records the voice command and HA tries to understand and execute the command. It does not play responses with TTS, it just plays another sound after finishing it’s task.

Heres a Dropbox-Link to a short video demo:

R2D2-Video

Many thanks for making this so easy - now i can continue working on the robot part - currently i can stear it with a PS4 Controller, but sadly all my tinkering has shifted the center of gravity and it falls over when braking :smiley:

Keep on tinkering,

Abrakhim

Wowowowowowowwwwwww TAKE MY VOTE THIS IS SO COOL !
I just love the fact the lights react to the sound even before the wake word. It makes it so snappy !!!

1 Like

Yep… very cool!
You have my vote too…

Are you aware of the Astromech TTS integration that lets you text to R2D2 speak?

Might be the perfect addition for your project if you get around to implementing TTS… :slight_smile:

5 Likes

Dude, that’s awesome. I wasn’t aware of it and will integrate it RIGHT NOW :slight_smile:

1 Like

… made my day :smiley:

After ObiDenKenobi remark about TTAstromech i went straight to work and now R2 answers in proper Astromech-Talk. I can even already understand his answer for ‘Sorry, I couldn’t understand that’ and ‘Turned on the lights’ hrhrhr

…but, hear for yourself …

Turned on the lights

Sorry, I couldn’t understand that

btw… this is using TTAstromech Voice-Option SHORT… the other one made me wanna kick him instantly

3 Likes

Very cool. Great work.

Glad I could help… ! :slight_smile:
Yeah, the short voice option is much preferable… unless you’re into sm-ing R2.

Love the new vid’s. Shame I can’t vote twice…

1 Like

Really cool to see these kind of projects starting to take shape! :slight_smile: You have my vote!
I’m struggling a bit to get this wake word detected sound to work.

Hey there!

Anyone got a tip how to improve STT? I think i may have a lot of noise.
Although i repeat “R2D2, Turn of the lights” like a calm teacher, this is what whisper gets :smiley: :

INFO:wyoming_faster_whisper.handler: Turn off the lights.
INFO:wyoming_faster_whisper.handler: Oh shit.  What I know, it's like-
INFO:wyoming_faster_whisper.handler: I'm trying to talk to you inhabit this video!
INFO:wyoming_faster_whisper.handler: Oh shit.  One of the lights.
INFO:wyoming_faster_whisper.handler: Oh shit.  Turn off the lights.
INFO:wyoming_faster_whisper.handler: Ocean...  Turn off
INFO:wyoming_faster_whisper.handler: who actually is like this?

The wake-word works very nice though…

As a first countermeasure i will try replacing the 4mic-Respeaker 3.5 audio jack and the little 2.5 W amp with a pimoroni Audio Amp SHIM - but on the software side - like wyoming-satellite - any ideas?

BTW: this is the PI ( 3, not 4 as i remembered it…) and the Pi Motor HAT i use.

I also added a little LED Flashlight, replaced the foil speaker with a decent one, added a PIR Sensor, connected the Blue and Red LEDs the Bot came with and the ‘Leia-Projector’ to the GPIOs, the two propulsion motors and the two motors that change it’s stance. I will upload a video, once this beast is finished… like in some years hrhrhr.

And this is a look into it’s guts:

Really interested to see if you have done any other modifications \ additions… ?

By the way, did you end up making the robot move and follow you as you were saying in your original post :smiling_face:?

Thx for your entry, it’s amazing!

The results of the contest are out!
They may be of interest to you :wink:
Have a look!

This is very cool, you have my vote too. I trained my R2D2 wake word myself but this seems to be much better. But one question where di you get the cool R2D2 from?

Kind reagrds
Elmar

Yes I would also like to know where to get the R2D2 from and do the same, so cool.