ESP32 S3 Box3

[stu247 (Jim)]

The file esp32-s3-box-3.yaml at https://github.com/esphome/firmware/blob/d584c44151100d5ef36d2994347807b76027eef0/voice-assistant/esp32-s3-box-3.yaml is getting updated on a regular basis. To avoid editing each time with my wifi credentials, I created a second file with my specifics and it does an include of esp32-s3-box-3.yaml. Here is an example file called red-box-3.yaml:

---
packages:
  base: !include esp32-s3-box-3.yaml

esphome:
  name: red-box-3
  friendly_name: Red ESP32 S3 Box 3
  name_add_mac_suffix: false

ota:
  password: !secret api_password
  
wifi:
  ssid: !secret ssid
  password: !secret w_password

voice_assistant:
  volume_multiplier: 3.0

Just run red-box-3.yaml instead of esp32-s3-box-3.yaml. See Configuration Types — ESPHome for more details.

[CChris (Christoph)]

hm… to be honest, I don’t really understand the “auto discovery” mechanism…

When I delete my configuration, ESPHome will auto-detect the device - but the installation usually fails, because it’s waiting for a password - which is not given.

If I cancel the auto-detect, I do have only the following config:

substitutions:
  name: esp32-s3-box-3-va-5acd3c
  friendly_name: ESP32 S3 Box 3 Voice Assist
packages:
  esphome.voice-assistant: github://esphome/firmware/voice-assistant/esp32-s3-box.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
api:
  encryption:
    key: {{key}}


wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

When I install this directly, then I will get the controlls, but can’t controll the display, nor do I see any of the pictures.

Only, when I copy the whole firmware.yaml from git, it is working as expected - but I thought, this should happen with the auto-discovery?

[stu247 (Jim)]

There may be an issue with the yaml file. The github URL has “box.yaml”, but should it be “box-3.yaml”?

[CChris (Christoph)]

thanks …

wow… seems that the auto-detection is pulling the wrong firmware then… o.O

[andrej33 (Andrej)]

Is it just me or has anyone else also problem with broken installation from latest config file from official repo?

Older version installed worked somehow, now I can’t get to setup wifi connection.

[starob]

I just installed the latest version of https://github.com/esphome/firmware/blob/d584c44151100d5ef36d2994347807b76027eef0/voice-assistant/esp32-s3-box-3.yaml by copying the source code into my esphome dashboard config. I added my WiFi settings and removed “name_add_mac_suffix: true” because I didn’t want to use this setting.

Now the reponse stream is much better in terms of cut off. I believe this is due to changes in ESPHome which is now version 2023.11.6. Only sometimes there is a little bit missing at the beginning. Unfortunately the volume is still very low and increasing the “volume_multiplier” did not change anything for me. Everything else works fine for me.

[stu247 (Jim)]

UPDATE: Starting with HA 2024.5 and ESPHome 2024.4.0, UDP is no longer used for the sending and receiving audio. The ESPHome-to-HA native API, using TCP, is used instead. This means special UDP VLAN firewall rules are no longer needed for voice assistant.

This is a post for people who want to run their Box 3 on a VLAN. I originally tried to run the Box 3 on my IoT VLAN without success. When I moved the Box 3 to my regular VLAN where home assistant runs, it worked fine. By using Wireshark, I determined that the Box 3 sent the audio streams to HA on randomly assigned UDP ports. Each audio stream was sent on a different UDP port. By adding firewall rules to allow the audio streams, I was able to get the Box 3 to work on the IoT VLAN. The firewall rule was to allow any UDP traffic from any Box 3 to the HA server.

[m4v3r1ck]

Just installed the code on my BOX-3 using your recommendations - I added my WiFi settings and removed “name_add_mac_suffix: true” because I didn’t want to use this setting - but the display shows a sad smiley.

So, almost there… is there a way to troubleshoot this? What do I have to look for in the logfiles?

[17:47:05][W][api.connection:107]: : Reading failed: BAD_INDICATOR errno=11
[17:47:06][W][component:214]: Component api took a long time for an operation (0.17 s).
[17:47:06][W][component:215]: Components should block for at most 20-30ms.

Edit: using regular VLAN where Home Assistant also resides.

Thanks.

[m4v3r1ck]

Great post, going to try the same. Will report back asap.

[starob]

I don’t use VLAN but see @stu247 comment above.

[m4v3r1ck]

This post got me on the right track.

1. Added the code below into my my_secrets.yaml

# Your `my_device` envryption key
esphome_encryption_key: "my_key"

2. Added the esphome_encryption_key to the my_device.yaml

# Enable Home Assistant API
api:
  encryption:
    key: !secret esphome_encryption_key
  on_client_connected:
    - script.execute: reset_display
  on_client_disconnected:
    - script.execute: reset_display

3. Added my WiFi credentials to my_device.yaml from within my_sectrets.yaml

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
# Static IP
  manual_ip:
    # Set this to the IP of the ESP
    static_ip: my_ip
    # Set this to the IP address of the router. Often ends with .1
    gateway: my_gw
    # The subnet of the network. 255.255.255.0 works for most home networks.
    subnet: 255.255.255.0
    dns1: my_gw
    dns2: 0.0.0.0
    
# wifi:
#  ap:
#  on_connect:
#    - delay: 5s # Gives time for improv results to be transmitted 
#    - ble.disable:
#    - script.execute: reset_display        
#  on_disconnect:
#    - ble.enable:
#    - script.execute: reset_display

ESPHome (beta) 2023.11.6

Can’t seem to get the BOX-3 working using my wake-word. Anyone the same issue? Almost there…

[starob]

Do you have additional voice satellites? If yes, turn them off.
In any case turn the box off. Restart HA and then turn the box back on.
This did the trick for me when my wakeword was not recognized.

And verify you have a “default” voice assistant defined in HA.

[CChris (Christoph)]

@starob

And verify you have a “default” voice assistant defined in HA.

you can also use the box with other assistants

but yes, restarting HA is required…

[m4v3r1ck]

Thanks for your swift reply, appreciated.

1. I have 2 additional M5Stack Atom Echo devices, turned them off.
2. Turned the box off, restarted HA and turned the box.
3. Rechecked my settings:

4. Using wake word “ok nabu” I get no response from the box.

[starob]

What do the ESPHome logs say when you start the Box-3? Especially anything about connecting to HA.

[m4v3r1ck]

Settings > System >> Logs:

Logger: aioesphomeapi.connection
Source: runner.py:188
First occurred: 10:07:30 (6 occurrences)
Last logged: 12:53:02
esp32-s3-box-3 @ IP-address: Connection error occurred: [Errno 104] Connection reset by peer

Settings > System >> Logs > ESPHome:

2023-12-02 12:49:41,059 INFO 304 GET /devices (0.0.0.0) 1.03ms
2023-12-02 12:49:45,318 INFO 200 GET /static/js/esphome/c.L2HzI1eb.js (0.0.0.0) 7.29ms
2023-12-02 12:49:45,321 INFO 200 GET /edit?configuration=esp32-s3-box-3.yaml (0.0.0.0) 1.19ms
2023-12-02 12:49:45,403 INFO 200 GET /static/js/esphome/monaco-editor/esm/vs/editor/editor.worker.js (0.0.0.0) 3.64ms
2023-12-02 12:49:45,438 INFO 101 GET /ace (0.0.0.0) 0.78ms
2023-12-02 12:49:45,452 INFO Running command 'esphome --dashboard -q vscode --ace /config/esphome'
2023-12-02 12:49:46,093 INFO 200 GET /static/fonts/codicon.ttf (0.0.0.0) 7.39ms
2023-12-02 12:49:49,363 INFO 200 GET /devices (0.0.0.0) 0.98ms
2023-12-02 12:50:38,782 INFO 304 GET /devices (0.0.0.0) 1.10ms
2023-12-02 12:54:49,363 INFO 200 GET /devices (0.0.0.0) 1.34ms
2023-12-02 12:59:49,363 INFO 200 GET /devices (0.0.0.0) 1.12ms
2023-12-02 13:04:49,364 INFO 200 GET /devices (0.0.0.0) 1.19ms

[starob]

I meant the ESPHome log from the Box-3 device. Go into your ESPHome Dashboard and click on LOGS for Box-3.

[m4v3r1ck]

Seems I have a wireless network issue atm. The log hangs after 255.255.255.0 and box-3 sad face reappears. Will report back asap. Thanks for the help so far.

[CChris (Christoph)]

the ‘sad’ face will be shown in these cases:

• general error
• no API connection
• no Wifi connection
• no Pipeline selected

      - id: error_page
        lambda: |-
          it.fill(Color::BLACK);
          it.image((it.get_width() / 2) - 100, (it.get_height() / 2) - 100, id(casita_error));
      - id: no_api_page
        lambda: |-
          it.fill(Color::BLACK);
          it.image((it.get_width() / 2) - 100, (it.get_height() / 2) - 100, id(casita_error));
      - id: no_wifi_page
        lambda: |-
          it.fill(Color::BLACK);
          it.image((it.get_width() / 2) - 100, (it.get_height() / 2) - 100, id(casita_error));
      - id: no_pipeline_page
        lambda: |-
          it.fill(Color::BLACK);
          it.image((it.get_width() / 2) - 100, (it.get_height() / 2) - 100, id(casita_error));

the error in your logs:

esp32-s3-box-3 @ IP-address: Connection error occurred: [Errno 104] Connection reset by peer

really indicates an issue with your network connection.
Not sure, why the connection was reset, but it seems, that your box was the device which reset the connection.

You could probably do a test and use your ‘default’ Wifi rather than configure a manual IP to see, if this would work.

I haven’t used Vlan and manual IP configuration on these devices yet - so I can’t say for sure, if these are working as expected atm.

[m4v3r1ck]

Thank you both for helping me out. The log from the box-3 starts with a warning:

INFO ESPHome 2023.11.6
INFO Reading configuration /config/esphome/esp32-s3-box-3.yaml...
INFO Updating https://github.com/esphome/esphome.git@pull/5230/head
WARNING GPIO0 is a strapping PIN and should only be used for I/O with care.
Attaching external pullup/down resistors to strapping pins can cause unexpected failures.
See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins

Have read the linked page, but don’t know the meaning of it yet, I just don’t comprehend the correlation for now.

Box-3 now makes a connection - restored fixed IP - with Home Assistant. I disconnected the Atom Echo’s, but still no interaction from within the box-3.

[17:26:25][D][api:102]: Accepted IP-address_HA
[17:26:25][D][api.connection:1089]: Home Assistant 2023.11.3 (IP-address_HA): Connected successfully
[17:26:25][W][component:214]: Component api took a long time for an operation (0.18 s).
[17:26:25][W][component:215]: Components should block for at most 20-30ms.