I am working on my ‘smart’ R2D2 for some time and i just finished making it a Home Assistant Satellite. The project has two main goals - first, making the robot drive independently, patrol my house and follow me, recharge when battery is low, detect obstacles, map the rooms - much like a vacuum robot. The second goal is to make it smart and integrate with HA.
This goal is ready now, so i wanna share the current setup here:
The 40cm remote controlled R2D2 toy was gutted and now has a Pi Motor HAT (…had an Arduino with MotorShield) and a Raspberry Pi 4(no, 3…) inside, with a Respeaker 4-mic array in the top dome. I tried other ways to turn it into an assistant, but recently things got a lot easier thanks to HA Focus on Assistants.
I used:
So, what does it do? I listens locally to the wakeword ‘R2D2’, then acts as a remote satellite in Home Assistant and reacts to the wakeword with a nice R2-Sound, then records the voice command and HA tries to understand and execute the command. It does not play responses with TTS, it just plays another sound after finishing it’s task.
Many thanks for making this so easy - now i can continue working on the robot part - currently i can stear it with a PS4 Controller, but sadly all my tinkering has shifted the center of gravity and it falls over when braking
Wowowowowowowwwwwww TAKE MY VOTE THIS IS SO COOL !
I just love the fact the lights react to the sound even before the wake word. It makes it so snappy !!!
After ObiDenKenobi remark about TTAstromech i went straight to work and now R2 answers in proper Astromech-Talk. I can even already understand his answer for ‘Sorry, I couldn’t understand that’ and ‘Turned on the lights’ hrhrhr
Anyone got a tip how to improve STT? I think i may have a lot of noise.
Although i repeat “R2D2, Turn of the lights” like a calm teacher, this is what whisper gets :
INFO:wyoming_faster_whisper.handler: Turn off the lights.
INFO:wyoming_faster_whisper.handler: Oh shit. What I know, it's like-
INFO:wyoming_faster_whisper.handler: I'm trying to talk to you inhabit this video!
INFO:wyoming_faster_whisper.handler: Oh shit. One of the lights.
INFO:wyoming_faster_whisper.handler: Oh shit. Turn off the lights.
INFO:wyoming_faster_whisper.handler: Ocean... Turn off
INFO:wyoming_faster_whisper.handler: who actually is like this?
The wake-word works very nice though…
As a first countermeasure i will try replacing the 4mic-Respeaker 3.5 audio jack and the little 2.5 W amp with a pimoroni Audio Amp SHIM - but on the software side - like wyoming-satellite - any ideas?
BTW: this is the PI ( 3, not 4 as i remembered it…) and the Pi Motor HAT i use.
I also added a little LED Flashlight, replaced the foil speaker with a decent one, added a PIR Sensor, connected the Blue and Red LEDs the Bot came with and the ‘Leia-Projector’ to the GPIOs, the two propulsion motors and the two motors that change it’s stance. I will upload a video, once this beast is finished… like in some years hrhrhr.
?