ESP32 S3 Box3

ESPHome
275

Hi Jason,

Radar and battery sensors are available. Add this to your device yaml:

packages:
  ...
  esphome.voice-assistant-sensor: github://jaymunro/esphome_firmware/esp32-box-docks/box3-sensor-for-voice-assistant.yaml@sensor_dock

And if you are interested in exciting upcoming features I have been working on (continued conversation and display text):
Change this

packages:
  esphome.voice-assistant: github://esphome/firmware/wake-word-voice-assistant/esp32-s3-box-3.yaml
  ...

to this:

packages:
  # esphome.voice-assistant: github://esphome/firmware/wake-word-voice-assistant/esp32-s3-box-3.yaml
  esphome.voice-assistant: github://jaymunro/esphome_firmware/wake-word-voice-assistant/esp32-s3-box-3.yaml@continued-conversation
  ...

I have a PR in draft that I am looking for testers on so it can be released on the esphome main repository. I’ll add a link to the pr with more description and video once I’ve got the video done.

1 Like
276

Thanks! I’ll test these out.

277

Ok, I tried these and could not get them to work. Here is the config file along with the logs from failed installs. The names of the logs tell you which of the packages were uncommented. Hopefully you can tell me what I did wrong.
https://drive.google.com/drive/folders/1xRoJ3QdyhAM6F1htR1cNLpeQ2c7Kfxjp

278

I had a look at your logs. The errors/warnings are typical of a need to do a clean build.

Not sure which you did first, but you need to do a “Clean build files” when installing new components (which will be many the first time round). Did you try that? It’s just a few items below “Install”.

This is common with the espidf framework.

1 Like
279

Ah, thanks! I was hitting the install button in the editor. Had to back out of the editor to find the “clean build files” button. It’s working now. I really like having the sensors and the last command/response saved.

280

Here is a demo video of this beta of Voice assistant with Continued conversation and text.

IMG_1020
Demo of Continued conversation BETA

I’m looking for testers who can give feedback on issues, further suggestions for improvement, etc. I’ve tested it with the HA streaming wake word but it works much better with the on-device wake word. Once it is verified, I have a PR ready to go in ESPHome repository. Please leave issues and suggestions in the related issue tracker (not here because they will get lost) Feature: Continued conversation with text on screen · Issue #174 · esphome/firmware · GitHub

To install, in your device’s yaml change this:

packages:
  esphome.voice-assistant: github://esphome/firmware/wake-word-voice-assistant/esp32-s3-box-3.yaml
  ...

to this:

packages:
  # esphome.voice-assistant: github://esphome/firmware/wake-word-voice-assistant/esp32-s3-box-3.yaml
  esphome.voice-assistant: github://jaymunro/esphome_firmware/wake-word-voice-assistant/esp32-s3-box-3.yaml@continued-conversation
  ...

If you change the wake word model, add another substitution for the wake word text as follows:

substitutions:
  ...
  micro_wake_word_model: hey_jarvis
  wake_word_text: "Hey Jarvis"

Note this is a beta and the repo above is temporary until after the PR is accepted.

Before doing an Install, be sure to do a Clean build files from the 3-dot hamburger menu in ESPHome.

2 Likes
281

Great @HJM!!! Just installed your code! Thanks for your work and effort! The best is continued conversation!!

The output sound is from time to time laggy, but ondevice wake word is much faster than on HA :slight_smile:
The date and time - how to configure the time and date? Mine shows Mon, 5 Jan, 1970 - 8:41 PM (both are wrong :slight_smile:

The display text is working pretty good - except localization (it’s in EN) - is there any chance to get it working in system default language (mine is Slovenian for example)?

But I’m not sure about the fonts - it doesn’t show out characters (I think it’s ISO/IEC 8859-2).

Wake word: from time to time the I get in the log:
09:31:48][D][voice_assistant:349]: Speaker buffer full, trying again next loop
and after few minutes it’s ok and the box starts to listen to wake word.

I’m really looking forward on this (yours) code, I hope the PR is getting approval soon :slight_smile: Thanks again for your work!!!

282

I finally succeed to install it, If you don’t have more than 4GB it will not compile, I update to have 6GB of ram and It worked, (didn’t use more than 4,5GB).
Right now, everythings seems to work, I gonna continue testing

1 Like
283

a few things I realized :

  • after flashing it never trigger again… even after a reset but a boot cycle make it work again.
  • the text is only in english would be nicer to be in substitions to be able to change it
  • the text doesn’t support some char, I’m french and turn off is “Éteins” It show a black rectangle instead of the É
  • would it be possible to have change the format of the date ? and time ?

I really like the time display in the center.
and the text also it’s nice too
maybe it could be nice to hide the say “okay nabu” which couldl be wrong since it can’t follow the actual wake word name but keep the rest of the text ?

284

As in my post above, if you change the wake word model, add another substitution for the wake word text as follows:

substitutions:
  ...
  micro_wake_word_model: hey_jarvis
  wake_word_text: "Hey Jarvis"

Good suggestion about using substitutions to change the text for other languages. I’ll work on that.

For characters in other languages, I need to expand the list of glyphs. Can you list all the characters needed (beyond the standard English A-Z, a-z) so I can add them?

For changing the date or time formats, I’ll need to think about the best method. Maybe substitutions, but this requires the user knows strftime formats that work in ESPHome. It is a smaller subset of the full strftime. Can you give some examples of date and time that you’d like?

285

I’m pasting glyphs from one of my yaml file in ESPHome:

glyphs:
      ['&', '@', '!', '?', ',', '.', '"', '%', '(', ')', '+', '-', '_', ':', '°', '0',
       '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E',
       'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S',
       'T', 'U', 'V', 'W', 'X', 'Y', 'Z', ' ', 'a', 'b', 'c', 'č', 'd', 'e', 'f',
       'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 'š', 't',
       'u', 'v', 'w', 'x', 'y', 'z', 'ž', '/', '€', '’' ]

but in fact I need just: ščž ŠČŽ

As for date and time: "%d.%m.%y"

Thanks!!!

1 Like
286

Great job. Really impressed.
I would vote for a 24-hour clock (no AM/PM)
And I would like the Danish letters æ ø å Æ Ø Å to be included. They are the last three letters in the Danish alphabet
Yours
Lars /Denmark

1 Like
287

you can dynamically change the pipeline and so the wake word when not using the in device, that’s why the text can’t be good, or we could have a text_sensor to change it (I would love that in fact)

288

latin like french and spanish would need :
é É ç Ç è È ê Ê à À ô
there is more but not sure if we need them maybe we should find a way to have all of them, don’t know about the correct way to do that

289

@Bluhme @bobzer @andrej33
If you could add your comments to the issue tracker on GitHub, this would help get the attention of the people who can authorise the addition of these features.

290

GitHub feature #174 above now has way more languages added to the glyphs, fixes to a stuck condition at boot, and methods for selecting date and time formats. Please check it out and give feedback.

291

How do we test this ?
Do we just have to copy paste the glyphs line ?

292

Just reinstall using the same package as before

packages:
  esphome.voice-assistant: github://jaymunro/esphome_firmware/wake-word-voice-assistant/esp32-s3-box-3.yaml@continued-conversation
1 Like
293

I don’t suppose you have a version for the original ESP32-S3-Box? (non box3)

295

Put an issue request in on esphome/firmware GitHub and I can look into it if there’s enough interest and someone to help test it. I don’t have the hardware which makes testing impossible for me.