ESP32 S3 Box3

starob · December 1, 2023, 12:24pm

I just installed the latest version of https://github.com/esphome/firmware/blob/d584c44151100d5ef36d2994347807b76027eef0/voice-assistant/esp32-s3-box-3.yaml by copying the source code into my esphome dashboard config. I added my WiFi settings and removed “name_add_mac_suffix: true” because I didn’t want to use this setting.

Now the reponse stream is much better in terms of cut off. I believe this is due to changes in ESPHome which is now version 2023.11.6. Only sometimes there is a little bit missing at the beginning. Unfortunately the volume is still very low and increasing the “volume_multiplier” did not change anything for me. Everything else works fine for me.

stu247 · December 1, 2023, 4:44pm

UPDATE: Starting with HA 2024.5 and ESPHome 2024.4.0, UDP is no longer used for the sending and receiving audio. The ESPHome-to-HA native API, using TCP, is used instead. This means special UDP VLAN firewall rules are no longer needed for voice assistant.

This is a post for people who want to run their Box 3 on a VLAN. I originally tried to run the Box 3 on my IoT VLAN without success. When I moved the Box 3 to my regular VLAN where home assistant runs, it worked fine. By using Wireshark, I determined that the Box 3 sent the audio streams to HA on randomly assigned UDP ports. Each audio stream was sent on a different UDP port. By adding firewall rules to allow the audio streams, I was able to get the Box 3 to work on the IoT VLAN. The firewall rule was to allow any UDP traffic from any Box 3 to the HA server.

m4v3r1ck · December 1, 2023, 4:49pm

Just installed the code on my BOX-3 using your recommendations - I added my WiFi settings and removed “name_add_mac_suffix: true” because I didn’t want to use this setting - but the display shows a sad smiley.

So, almost there… is there a way to troubleshoot this? What do I have to look for in the logfiles?

[17:47:05][W][api.connection:107]: : Reading failed: BAD_INDICATOR errno=11
[17:47:06][W][component:214]: Component api took a long time for an operation (0.17 s).
[17:47:06][W][component:215]: Components should block for at most 20-30ms.

Edit: using regular VLAN where Home Assistant also resides.

Thanks.

m4v3r1ck · December 1, 2023, 4:53pm

Great post, going to try the same. Will report back asap.

starob · December 1, 2023, 5:15pm

I don’t use VLAN but see @stu247 comment above.

m4v3r1ck · December 1, 2023, 8:53pm

This post got me on the right track.

Added the code below into my my_secrets.yaml

# Your `my_device` envryption key
esphome_encryption_key: "my_key"

Added the esphome_encryption_key to the my_device.yaml

# Enable Home Assistant API
api:
  encryption:
    key: !secret esphome_encryption_key
  on_client_connected:
    - script.execute: reset_display
  on_client_disconnected:
    - script.execute: reset_display

Added my WiFi credentials to my_device.yaml from within my_sectrets.yaml

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
# Static IP
  manual_ip:
    # Set this to the IP of the ESP
    static_ip: my_ip
    # Set this to the IP address of the router. Often ends with .1
    gateway: my_gw
    # The subnet of the network. 255.255.255.0 works for most home networks.
    subnet: 255.255.255.0
    dns1: my_gw
    dns2: 0.0.0.0
    
# wifi:
#  ap:
#  on_connect:
#    - delay: 5s # Gives time for improv results to be transmitted 
#    - ble.disable:
#    - script.execute: reset_display        
#  on_disconnect:
#    - ble.enable:
#    - script.execute: reset_display

ESPHome (beta) 2023.11.6

Can’t seem to get the BOX-3 working using my wake-word. Anyone the same issue? Almost there…

starob · December 2, 2023, 8:45am

Do you have additional voice satellites? If yes, turn them off.
In any case turn the box off. Restart HA and then turn the box back on.
This did the trick for me when my wakeword was not recognized.

And verify you have a “default” voice assistant defined in HA.

CChris · December 2, 2023, 9:32am

you can also use the box with other assistants

grafik

but yes, restarting HA is required…

m4v3r1ck · December 2, 2023, 9:37am

Thanks for your swift reply, appreciated.

I have 2 additional M5Stack Atom Echo devices, turned them off.
Turned the box off, restarted HA and turned the box.
Rechecked my settings:

Using wake word “ok nabu” I get no response from the box.

starob · December 2, 2023, 10:41am

What do the ESPHome logs say when you start the Box-3? Especially anything about connecting to HA.

m4v3r1ck · December 2, 2023, 12:03pm

Settings > System >> Logs:

Logger: aioesphomeapi.connection
Source: runner.py:188
First occurred: 10:07:30 (6 occurrences)
Last logged: 12:53:02
esp32-s3-box-3 @ IP-address: Connection error occurred: [Errno 104] Connection reset by peer

Settings > System >> Logs > ESPHome:

2023-12-02 12:49:41,059 INFO 304 GET /devices (0.0.0.0) 1.03ms
2023-12-02 12:49:45,318 INFO 200 GET /static/js/esphome/c.L2HzI1eb.js (0.0.0.0) 7.29ms
2023-12-02 12:49:45,321 INFO 200 GET /edit?configuration=esp32-s3-box-3.yaml (0.0.0.0) 1.19ms
2023-12-02 12:49:45,403 INFO 200 GET /static/js/esphome/monaco-editor/esm/vs/editor/editor.worker.js (0.0.0.0) 3.64ms
2023-12-02 12:49:45,438 INFO 101 GET /ace (0.0.0.0) 0.78ms
2023-12-02 12:49:45,452 INFO Running command 'esphome --dashboard -q vscode --ace /config/esphome'
2023-12-02 12:49:46,093 INFO 200 GET /static/fonts/codicon.ttf (0.0.0.0) 7.39ms
2023-12-02 12:49:49,363 INFO 200 GET /devices (0.0.0.0) 0.98ms
2023-12-02 12:50:38,782 INFO 304 GET /devices (0.0.0.0) 1.10ms
2023-12-02 12:54:49,363 INFO 200 GET /devices (0.0.0.0) 1.34ms
2023-12-02 12:59:49,363 INFO 200 GET /devices (0.0.0.0) 1.12ms
2023-12-02 13:04:49,364 INFO 200 GET /devices (0.0.0.0) 1.19ms

starob · December 2, 2023, 12:19pm

I meant the ESPHome log from the Box-3 device. Go into your ESPHome Dashboard and click on LOGS for Box-3.

m4v3r1ck · December 2, 2023, 12:43pm

Seems I have a wireless network issue atm. The log hangs after 255.255.255.0 and box-3 sad face reappears. Will report back asap. Thanks for the help so far.

CChris · December 2, 2023, 1:07pm

the ‘sad’ face will be shown in these cases:

general error
no API connection
no Wifi connection
no Pipeline selected

      - id: error_page
        lambda: |-
          it.fill(Color::BLACK);
          it.image((it.get_width() / 2) - 100, (it.get_height() / 2) - 100, id(casita_error));
      - id: no_api_page
        lambda: |-
          it.fill(Color::BLACK);
          it.image((it.get_width() / 2) - 100, (it.get_height() / 2) - 100, id(casita_error));
      - id: no_wifi_page
        lambda: |-
          it.fill(Color::BLACK);
          it.image((it.get_width() / 2) - 100, (it.get_height() / 2) - 100, id(casita_error));
      - id: no_pipeline_page
        lambda: |-
          it.fill(Color::BLACK);
          it.image((it.get_width() / 2) - 100, (it.get_height() / 2) - 100, id(casita_error));

the error in your logs:

esp32-s3-box-3 @ IP-address: Connection error occurred: [Errno 104] Connection reset by peer

really indicates an issue with your network connection.
Not sure, why the connection was reset, but it seems, that your box was the device which reset the connection.

You could probably do a test and use your ‘default’ Wifi rather than configure a manual IP to see, if this would work.

I haven’t used Vlan and manual IP configuration on these devices yet - so I can’t say for sure, if these are working as expected atm.

m4v3r1ck · December 2, 2023, 4:42pm

Thank you both for helping me out. The log from the box-3 starts with a warning:

INFO ESPHome 2023.11.6
INFO Reading configuration /config/esphome/esp32-s3-box-3.yaml...
INFO Updating https://github.com/esphome/esphome.git@pull/5230/head
WARNING GPIO0 is a strapping PIN and should only be used for I/O with care.
Attaching external pullup/down resistors to strapping pins can cause unexpected failures.
See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins

Have read the linked page, but don’t know the meaning of it yet, I just don’t comprehend the correlation for now.

Box-3 now makes a connection - restored fixed IP - with Home Assistant. I disconnected the Atom Echo’s, but still no interaction from within the box-3.

[17:26:25][D][api:102]: Accepted IP-address_HA
[17:26:25][D][api.connection:1089]: Home Assistant 2023.11.3 (IP-address_HA): Connected successfully
[17:26:25][W][component:214]: Component api took a long time for an operation (0.18 s).
[17:26:25][W][component:215]: Components should block for at most 20-30ms.

CChris · December 2, 2023, 4:45pm

ok, you can ignore the warning about “GPIO is a strapping PIN” right now, that’s not related to your issues.

The Other logs seem also to be OK.
Now, You probably need to restart your HomeAssistant… at least, that was required for me after first uploading the Firmware

starob · December 2, 2023, 4:55pm

Yes, I needed to do that too.

DieterKoblenz · December 2, 2023, 6:49pm

Wow, this works great. Initially I bought this esp to experiment with Willow, a project designed to use voice to control things (i.e. HA). However I already use esphome, so to integrate this was no problem at all.

starob · December 2, 2023, 6:56pm

@DieterKoblenz Another confirmation that this works. Great! Do you also experience responses starting a little late sometimes?

blackie333 · December 2, 2023, 7:47pm

For me the updated yaml is finally working.
Now I see the reason why it was not working for so long before.
I had “Google TTS” set in the VA settings and expected it will work. NO!
The pipeline (WakeWord + Whisper) started to work just AFTER I desperately added Piper to Wyoming. Without Piper nothing worked, not even WakeWord.
It’s hard to identify the problem in the VA pipeline.
I lost much time trying to get microphone in M5Stack AtomU working with the same VA setup…