ESP32 S3 Box3

Yes, I have configured my own wake word ā€œLocal TTS STTā€ that works flawless with the M5Stack Atom Echo devices. I see that youā€™re also using the Home Assistant Cloud for STT and TTS.

Will get myself the 1 month free Home Assistant Cloud and test my setup with that and set it up as ā€œPreferredā€ VA. Thanks for now yā€™all.

Could you possibly give some more details on what you did? Steps would be super helpful and Iā€™d be really appreciative if you did that. Did you get it working with the ready-made projects then add it in esphome then update it in esphome with the provided yaml or did you just drop the ready-made projects thing all together and just set it all up in esp home?? Just a bit confused about the process.

The configuration via esphome dashboard (copy pasting yaml from github) actualy bricked my box - see log above. And the steps I vaguely described were about unbricking esp box. Because the device was in reboot loop it was hard to just flash it. To unbrick the box I used Espressif Launchpad - after a few tries the stock fw (demo file) brought box back to life. After that I used ESPHome Ready made projects to flash it with fw from webserial. After a coulpe of tries (mainly difficulties with improv serial or improv bluetooth) I managed to connect it with HA and ESPHome dashboard. I had to try a few times with adopting the device in esphome dashboard and then after it was adpoted I could update it because the update notfication (that blue line under device box) appeard. After an update (latest version) the box started working but the response sound is still hard to hear but better that the first versions which broke the voice.

I hope this could be for any help.

1 Like

Yes, that is very helpful! I appreciate you spelling it out for me. So when I get mine, I should just use the ready made projects one right away I guess?

Tried using the box-3 via Home Assistant Cloud, yet still no action nor any reaction returned.

What tiny little tweak - code check - did I miss at each sequence flashing it and to get the box-3 up-and-running for VA. What should the last couple of code-lines be, when pulling a log from the box-3 after a HA restart while box-3 is turned off?

[20:55:24][C][api:142]:   Using noise encryption: YES
[20:55:24][C][improv_serial:032]: Improv Serial:
[20:56:27][I][ota:117]: Boot seems successful, resetting boot loop counter.
[20:56:27][D][esp32.preferences:114]: Saving 1 preferences to flash...
[20:56:27][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed

Try using the ESPHome Interface directly :slight_smile:
In theory, the ESPHome should identify your device when you delete your current configurationā€¦

ESPHome auto-discovers the Box-3
grafik

You can now Adopt the device:
grafik

After adopting it, a Dialog with the Encryption Key should appear which allows you to install the device:
grafik

Installationā€¦ this will take some timeā€¦

Personally, I had to install the device several times in the past, because I had a ā€˜customā€™ Firmware running - and also copied the SourceCode from Git directly.
This caused issues with the DNS resolution in my network, and so onā€¦

Also, try to delete already existing build files - that might help:

The configuration in your device yaml should be pretty straight forward:

substitutions:
  name: esp32-s3-box-3-va-5acd3c
  friendly_name: ESP32 S3 Box 3 Voice Assist
packages:
  esphome.voice-assistant: github://esphome/firmware/voice-assistant/esp32-s3-box-3.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
api:
  encryption:
    key: ******


wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
2 Likes

Install sequence startedā€¦ brb

Edit 12/04/2023 - 00:33

This is amazing. Thank you so very much for reaching out to me, Iā€™m so glad itā€™s working now. Your workflow was flawless and was easy to follow, it will help other users as well.

Kudos for that! :sunglasses:

note-to-self: oh my gosh, was I on the wrong trackā€¦

1 Like

soā€¦ letā€™s hope, the sensors from the dock will be utilized also soonā€¦ :slight_smile:
I can imagine, that the focus is on the VA itself right nowā€¦ but I would really love to have the presence sensor, temp-sensor and IR being utilized as wellā€¦ I think, it would make sense to use the presence sensor in combination with VoiceAssist (getting activated, when presence is detected, else, sleep - or similar scenarios)

5 Likes

For sure I hope too, that we can utilize more features of the box-3. For now itā€™s a nice additional VA.

I think, it would make sense to use the presence sensor in combination with VoiceAssist (getting activated, when presence is detected, else, sleep - or similar scenarios)

Great idea! :smiley:

1 Like

Can anyone offer up an option on the performance as a VA input/output device? I used Rhasspy for a while with a ReSpeaker Mic Array v2.0. It worked ok, but not great. It was fine in a quiet room, but if music or the TV were on performance would drop off significantly.

@bkprath I donā€™t have any experience with Rhasspy but here are my 2 cents: It might be an unfair comparison but compared to Amazon Echo for example VA is way behind, no matter on which device you install it. BUT VA has made a big step forward since the release of Year-of-the-Voice Chapter 4. If it develops further at that speed it might eventually get there.

The biggest issue for me right now with Box-3 is that the speaker volume is very low. You hardly can hear a response even in a quiet room. I tried to increase the volume by increasing the ā€œvolume_multiplierā€ setting but that didnā€™t change anything. So no matter how it performs otherwise, it is not very useful right now.

I also tested a homeassistant-satellite https://github.com/synesthesiam/homeassistant-satellite on raspberry 3b using a USB conference system. As a VA this is much better because it has better mics and a much loader speaker. On the other hand this is more expensive and the form factor is not as nice as the Box-3.

My approach right now is to get experience with VA on various devices hoping that in the meantime VA will develop to become a great solution.

1 Like

What microphone did you utilize with your homassistant-satellite? I tried two different mic arrays from the same company and wasnā€™t satisfied with either when i used rhasspy. I didnā€™t have any auto output problems, it was mic pickup and detection in the presents of auto output that gave me trouble.

I used this [Anker PowerConf S330 USBā€¦ Amazon.de](Anker PowerConf S330 USBā€¦ Amazon.de)

After testing the BOX3 VA performance I agree itā€™s still much work ahead.
For me the most important is to allow independent speaker and microphone devices usage in the pipeline.
Have several HA integrated speakers of much better quality than the small speaker in Echo/Box3 and waiting until I can use these for outputā€¦
Also have some separate microphones without speaker ā€œReSpeaker USB mic Arrayā€, ā€œPS3 Eye USBā€ or M5Stock AtomU which is partly working in ESPhome but needs support for sending audio to some HA speakerā€¦

Hi all!

On @m4v3r1ck post above, I see the ā€œTop Left Buttonā€ reachable by Home Assistant.

While mine, shows like this ā€œThis Entity is unavailableā€:

Iā€™ve tried to edit the yaml on espHome, changing this to ā€œfalseā€:

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO0
      mode: INPUT_PULLUP
      inverted: true
    name: Top Left Button
    disabled_by_default: true

But I canā€™t save after editing this (probably a different issue, but thought to mention this try)

Any ideas?

PS: I was about to post as an issue on ā€œIssues Ā· esphome/firmware Ā· GitHubā€, but as I notice Maverick post, likely is something with my install, not with whole ESP thingā€¦

If your VA is based on a device that runs some kind of Linux, e.g. Raspberry Pi with Raspberry Pi OS, then you can install MPD (https://github.com/MusicPlayerDaemon/MPD in addition to your VA software. That gives you a media player in HA that you can use to output vioce. I have homeassistant-satellite and MPD running in parallel just fine.

Have HassOS installation on OdroidC4 and already use 2 network devices running MPD daemon for TTS and audio automations/notifications.
But I still havenā€™t figured out how to configure the VA pipeline to use them together with independent microphone deviceā€¦
Seems to me VA at this moment only supports combined(microphone+speaker) devices.

Agree, I think so too.

Did this and it worked like a charm. I can talk with gpt4, but I canā€™t control any of my devices or read sensors or anything. It could definitely be my home assistant setup though bc I really donā€™t know home assistant very well.

Gpt cannot control your HA entities because it does not know them. That is a limitation right now and it may not change in the future either but Iā€™m not sure.