ESP32 S3 Box3

HJM · January 5, 2024, 11:20am

Where can I see this @RT1080?

CChris · January 5, 2024, 11:23am

I will try it as soon as I am at home

RT1080 · January 5, 2024, 11:42am

Esphome shared their yaml on GitHub. Curious to know if you used theirs or your own config specifically in relation to audio (seeing you dont experience volume issues)

Can you share your yaml?

CChris · January 5, 2024, 2:55pm

Hm… the first time the configuration has not worked as intended, but I re-uploaded it and now, everything seems to work as expected.
Great work & thanks

Now, only the i2c issue remains - and I hope, that the changes, someone provided on the github issue will work - then, all components are working for the Box3

[Update]
I really need to get used to the fact, that my Box3 Display is now turning off - when I am in front of the computer but does not move enough for the radar to recognize me
Bit strange, after weeks continuesly running display - and I always think, the device is crashing and rebooting lol

HJM · January 5, 2024, 9:59pm

Hi Ronald. I am using the standard method that uses the esphome repo as a package.

Can you share an objective measurement of the sound dB at 50cm like I did where I got 71dB? It may be just that we have different expectations.

Anyhow, my yaml is:

substitutions:
  name: esp32-s3-box-3-5ad0fc
  friendly_name: Jarvis

packages:
  esphome.voice-assistant: github://esphome/firmware/voice-assistant/esp32-s3-box-3.yaml@main

esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}

api:
  encryption:
    key: <redacted>=


wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password


# i2c:
#   - id: bus_a
#     sda: GPIO08
#     scl: GPIO18
#     scan: false

#   - sda: GPIO41
#     scl: GPIO40
#     id: bus_b

sensor:
  - platform: adc
    pin: GPIO10
    name: "Battery voltage"
    id: battery_voltage
    unit_of_measurement: "V"
    accuracy_decimals: 3
    device_class: "voltage"
    entity_category: "diagnostic"
    disabled_by_default: true
    update_interval: 30s
    attenuation: auto
    filters:
        - multiply: 4.01

  - platform: copy
    source_id: battery_voltage
    name: "Battery level"
    unit_of_measurement: "%"
    accuracy_decimals: 1
    device_class: "battery"
    entity_category: "diagnostic"
    filters:
      - lambda: return (x - 2.5) / (4.2 - 2.5) * 100;
      - clamp:
          min_value: 0
          max_value: 100
          ignore_out_of_range: true

  # - platform: aht10
  #   i2c_id: bus_b
  #   variant: AHT20
  #   temperature:
  #     name: "Temperature"
  #     accuracy_decimals: 2
  #   humidity:
  #     name: "Humidity"
  #     accuracy_decimals: 1
  #   update_interval: 60s


switch:
  - platform: template
    name: "Mute when absent"
    id: mute_when_absent
    icon: mdi:account-right-arrow
    optimistic: true
    entity_category: config
    restore_mode: RESTORE_DEFAULT_OFF

number:
  - platform: template
    name: "Presence duration"
    id: radar_delayed_off
    icon: mdi:account-clock
    optimistic: true
    restore_value: true
    initial_value: 60
    min_value: 0
    step: 5
    max_value: 300
    unit_of_measurement: s
    entity_category: config
    mode: box

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO21
    name: "Presence detect"
    disabled_by_default: false
    device_class: "occupancy"
    filters:
      - delayed_off: !lambda return id(radar_delayed_off).state * 1000;
    on_release:
      then:
        - if:
            condition: 
              switch.is_on: mute_when_absent
            then:
              - switch.turn_on: mute
              - light.turn_off: led
    on_press:
      then:
        - if:
            condition: 
              switch.is_on: mute_when_absent
            then:
              - switch.turn_off: mute
              - light.turn_on: led

HJM · January 5, 2024, 10:03pm

You could always increase the Presence duration to a longer time from the default of 60 secs. I’ve allowed a max of 5 mins. It’d be hard to stay still for 5 mins.

In the longer term when we get the I2C bus working nicely with the VA side we may be able to tweak the radar sensitivity.

CChris · January 6, 2024, 10:04am

Hi, yes I am aware of these

Right now, I am not sure if it is related to the integration of the Presence detector, but it seems, that my wakeword-detection failed… it is somehow the same behave as I observed it when the i2c was configured…

Even after uploading the configuration the device does not seem to detect the wake-word.
I would like to monitor, if this happens also for someone else, using the radar sensor.

I saw an error that might be related to it, but unfortunately, I had to reboot my system due to other stuff (installed updates and a new integration) before I made a copy of the message…

So I will monitor this further and update again.

m4v3r1ck · January 6, 2024, 10:23am

My BOX-3 is not processing its wake word anymore. My two atom echo’s now also fail to process the wake words. Something is off. Have to start troubleshooting…

CChris · January 6, 2024, 10:25am

did you add the presence sensor, too?

m4v3r1ck · January 6, 2024, 11:14am

No, changed nothing for the BOX-3, nor the atom’s

The wake word toggle for BOX-3 is unavailable:

Have now updated my Home Assistant to:

Home Assistant

Core 2024.1.1
Supervisor 2023.12.0
Operating System 11.3
Frontend 20240104.0

Studio Code Server (5.14.2), Home Assistant Google Drive Backup (0.112.1), InfluxDB (4.8.0), Grafana (9.1.1), Terminal & SSH (9.8.1), File editor (5.7.0), Samba share (12.2.0), Whisper (1.0.0), Piper (1.4.0), openWakeWord (1.8.2), ESPHome (beta) (2023.12.5)

After the updates I rebooted my Home Assistant VM instance on my NAS. Still no luck and the BOX-3 wake word remains unavailable, atom’s don’t react as well.

CChris · January 6, 2024, 11:28am

ah!
The “use-wakeword” switch was replaced and the “mute” switch took over the same function.

m4v3r1ck · January 6, 2024, 12:01pm

When I “mute” the display goes black. So I don’t understand this ‘change’ in functions for the both toggles. Very confusing.

edit 13:12 All is working again, after I gave the command “Update All” in the ESPHOME (beta) tab:

2024-01-06 13:05:11,069 INFO 304 GET /devices (0.0.0.0) 1.88ms
2024-01-06 13:05:15,525 INFO 200 GET /edit?configuration=esp32-s3-box-3.yaml (0.0.0.0) 2.33ms
2024-01-06 13:05:15,602 INFO 101 GET /ace (0.0.0.0) 3.03ms
2024-01-06 13:05:15,604 INFO Running command 'esphome --dashboard -q vscode --ace /config/esphome'
2024-01-06 13:05:18,994 INFO 304 GET /devices (0.0.0.0) 1.98ms
2024-01-06 13:05:20,430 INFO 200 GET /edit?configuration=everything-presence-lite-301838.yaml (0.0.0.0) 1.59ms
2024-01-06 13:05:20,449 INFO 101 GET /ace (0.0.0.0) 0.53ms
2024-01-06 13:05:20,462 INFO Running command 'esphome --dashboard -q vscode --ace /config/esphome'
2024-01-06 13:05:22,009 INFO 200 GET /static/schema/substitutions.json (0.0.0.0) 2.67ms
2024-01-06 13:05:22,807 INFO 304 GET /devices (0.0.0.0) 1.76ms
2024-01-06 13:05:23,877 INFO 304 GET /edit?configuration=m5stack-atom-echo-803494.yaml (0.0.0.0) 1.22ms
2024-01-06 13:05:23,893 INFO 101 GET /ace (0.0.0.0) 0.77ms
2024-01-06 13:05:23,895 INFO Running command 'esphome --dashboard -q vscode --ace /config/esphome'
2024-01-06 13:05:26,661 INFO 304 GET /devices (0.0.0.0) 1.92ms
2024-01-06 13:05:29,215 INFO 304 GET /edit?configuration=m5stack-atom-echo-80b520.yaml (0.0.0.0) 1.29ms
2024-01-06 13:05:29,247 INFO 101 GET /ace (0.0.0.0) 0.66ms
2024-01-06 13:05:29,250 INFO Running command 'esphome --dashboard -q vscode --ace /config/esphome'
2024-01-06 13:05:31,273 INFO 304 GET /devices (0.0.0.0) 5.61ms
2024-01-06 13:05:36,267 INFO 304 GET /devices (0.0.0.0) 1.41ms
2024-01-06 13:05:41,273 INFO 304 GET /devices (0.0.0.0) 2.10ms
2024-01-06 13:05:46,277 INFO 304 GET /devices (0.0.0.0) 3.33ms
2024-01-06 13:05:51,284 INFO 304 GET /devices (0.0.0.0) 2.65ms
2024-01-06 13:05:56,282 INFO 304 GET /devices (0.0.0.0) 1.30ms
2024-01-06 13:06:01,283 INFO 304 GET /devices (0.0.0.0) 1.31ms
2024-01-06 13:06:06,283 INFO 304 GET /devices (0.0.0.0) 1.34ms
2024-01-06 13:07:26,607 INFO 200 GET /devices (0.0.0.0) 1.41ms
2024-01-06 13:09:44,238 INFO 304 GET / (0.0.0.0) 1.88ms
2024-01-06 13:09:44,286 INFO 304 GET /devices (0.0.0.0) 3.15ms
2024-01-06 13:09:49,284 INFO 304 GET /devices (0.0.0.0) 1.75ms

Settings BOX-3:

CChris · January 6, 2024, 12:14pm

that’s correct.
When the device is “muted” - the display goes off and voice assistant would not use the wakeword detection.

The same rule was applied to the “use wakeword” switch before - and this swich has been removed and the muted switch took over the same function.

Just remove the “use wakeword” switch from your dashboard - everything should be as intended then

here’s the commit in the firmware repository that describe the change
grafik

m4v3r1ck · January 6, 2024, 12:17pm

Thanks for your clarification, appreciated.

Screenshot 2024-01-06 at 13.38.18

m4v3r1ck · January 6, 2024, 3:09pm

Nice additional use of the BOX-3

Screenshot 2024-01-06 at 16.08.50

CChris · January 6, 2024, 3:43pm

be prepared:

github.com/esphome/issues

ESP32-S3-BOX-3 Voice assistant stability issue when i2c is activated

opened 11:19AM - 26 Dec 23 UTC

moonilightshaker

### The problem Hello, Voice Assistant is installed on the ESP32-S3-BOX-3. Af…ter activating i2c. The Voice Assistant pipeline no longer activates after a few hours. A factory reset does not help. Only deactivating the i2c components and reflashing helps. The Red Button under the Display does not work either. Best regards, M. ### Which version of ESPHome has the issue? 2023.12.3 ### What type of installation are you using? Home Assistant Add-on ### Which version of Home Assistant has the issue? 2023.12.3 ### What platform are you using? ESP32-IDF ### Board ESP32-S3-BOX-3 ### Component causing the issue Voice assistant ### Example YAML snippet ```yaml substitutions: name: esp32-s3-box-3-5ace4c friendly_name: ESP32 S3 Box 3 loading_illustration_file: https://github.com/esphome/firmware/raw/main/voice-assistant/casita/loading_320_240.png idle_illustration_file: https://github.com/esphome/firmware/raw/main/voice-assistant/casita/idle_320_240.png listening_illustration_file: https://github.com/esphome/firmware/raw/main/voice-assistant/casita/listening_320_240.png thinking_illustration_file: https://github.com/esphome/firmware/raw/main/voice-assistant/casita/thinking_320_240.png replying_illustration_file: https://github.com/esphome/firmware/raw/main/voice-assistant/casita/replying_320_240.png error_illustration_file: https://github.com/esphome/firmware/raw/main/voice-assistant/casita/error_320_240.png loading_illustration_background_color: '000000' idle_illustration_background_color: '000000' listening_illustration_background_color: 'FFFFFF' thinking_illustration_background_color: 'FFFFFF' replying_illustration_background_color: 'FFFFFF' error_illustration_background_color: '000000' voice_assist_idle_phase_id: '1' voice_assist_listening_phase_id: '2' voice_assist_thinking_phase_id: '3' voice_assist_replying_phase_id: '4' voice_assist_not_ready_phase_id: '10' voice_assist_error_phase_id: '11' voice_assist_muted_phase_id: '12' #wifi_update_interval: 3s #packages: # esphome.voice-assistant: github://esphome/firmware/voice-assistant/esp32-s3-box-3.yaml@main esphome: name: ${name} name_add_mac_suffix: false friendly_name: ${friendly_name} platformio_options: board_build.flash_mode: dio project: name: esphome.voice-assistant version: "1.0" min_version: 2023.11.5 on_boot: priority: 600 then: - script.execute: draw_display - delay: 30s - if: condition: lambda: return id(init_in_progress); then: - lambda: id(init_in_progress) = false; - script.execute: draw_display api: encryption: key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx on_client_connected: - script.execute: draw_display on_client_disconnected: - script.execute: draw_display wifi: ssid: !secret wifi_ssid password: !secret wifi_password ap: on_connect: - script.execute: draw_display - delay: 5s # Gives time for improv results to be transmitted - ble.disable: on_disconnect: - script.execute: draw_display - ble.enable: esp32: board: esp32s3box flash_size: 16MB framework: type: esp-idf sdkconfig_options: CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y" CONFIG_ESP32S3_DATA_CACHE_64KB: "y" CONFIG_ESP32S3_DATA_CACHE_LINE_64B: "y" CONFIG_AUDIO_BOARD_CUSTOM: "y" CONFIG_ESP32_S3_BOX_3_BOARD: "y" components: - name: esp32_s3_box_3_board source: github://jesserockz/esp32-s3-box-3-board@main refresh: 0s psram: mode: octal speed: 80MHz external_components: - source: github://pr#5230 components: esp_adf refresh: 0s ota: logger: hardware_uart: USB_SERIAL_JTAG # level: VERBOSE dashboard_import: package_import_url: github://esphome/firmware/voice-assistant/esp32-s3-box-3.yaml@main improv_serial: esp32_improv: authorizer: none button: - platform: factory_reset id: factory_reset_btn name: Factory reset binary_sensor: - platform: gt911 name: "Home Button" index: 1 gt911_id: gt911_touchscreen on_press: then: - light.toggle: led - platform: gpio pin: number: GPIO1 inverted: true name: "Mute" disabled_by_default: true entity_category: diagnostic - platform: gpio pin: number: GPIO0 mode: INPUT_PULLUP inverted: true name: Top Left Button disabled_by_default: true entity_category: diagnostic on_multi_click: - timing: - ON for at least 10s then: - button.press: factory_reset_btn output: - platform: ledc pin: GPIO47 id: backlight_output light: - platform: monochromatic id: led name: LCD Backlight entity_category: config output: backlight_output restore_mode: RESTORE_DEFAULT_ON default_transition_length: 250ms esp_adf: board: esp32s3box3 microphone: - platform: esp_adf id: box_mic speaker: - platform: esp_adf id: box_speaker voice_assistant: id: va microphone: box_mic speaker: box_speaker use_wake_word: true noise_suppression_level: 2 auto_gain: 31dBFS volume_multiplier: 2.0 #2.0 vad_threshold: 3 on_listening: - lambda: id(voice_assistant_phase) = ${voice_assist_listening_phase_id}; - script.execute: draw_display - light.turn_on: led on_stt_vad_end: - lambda: id(voice_assistant_phase) = ${voice_assist_thinking_phase_id}; - script.execute: draw_display on_tts_stream_start: - lambda: id(voice_assistant_phase) = ${voice_assist_replying_phase_id}; - script.execute: draw_display on_tts_stream_end: - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id}; - script.execute: draw_display - light.turn_off: led on_error: - if: condition: lambda: return !id(init_in_progress); then: - lambda: id(voice_assistant_phase) = ${voice_assist_error_phase_id}; - script.execute: draw_display - delay: 1s - if: condition: switch.is_off: mute then: - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id}; - light.turn_off: led else: - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id}; - light.turn_on: led - script.execute: draw_display on_client_connected: - if: condition: switch.is_off: mute then: - wait_until: not: ble.enabled - voice_assistant.start_continuous: - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id}; - light.turn_off: led else: - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id}; - light.turn_on: led - lambda: id(init_in_progress) = false; - script.execute: draw_display on_client_disconnected: - lambda: id(voice_assistant_phase) = ${voice_assist_not_ready_phase_id}; - script.execute: draw_display - light.turn_on: led script: - id: draw_display then: - if: condition: lambda: return !id(init_in_progress); then: - if: condition: wifi.connected: then: - if: condition: api.connected: then: - lambda: | switch(id(voice_assistant_phase)) { case ${voice_assist_listening_phase_id}: id(s3_box_lcd).show_page(listening_page); id(s3_box_lcd).update(); break; case ${voice_assist_thinking_phase_id}: id(s3_box_lcd).show_page(thinking_page); id(s3_box_lcd).update(); break; case ${voice_assist_replying_phase_id}: id(s3_box_lcd).show_page(replying_page); id(s3_box_lcd).update(); break; case ${voice_assist_error_phase_id}: id(s3_box_lcd).show_page(error_page); id(s3_box_lcd).update(); break; case ${voice_assist_muted_phase_id}: id(s3_box_lcd).show_page(muted_page); id(s3_box_lcd).update(); break; case ${voice_assist_not_ready_phase_id}: id(s3_box_lcd).show_page(no_ha_page); id(s3_box_lcd).update(); break; default: id(s3_box_lcd).show_page(idle_page); id(s3_box_lcd).update(); } else: - display.page.show: no_ha_page - component.update: s3_box_lcd else: - display.page.show: no_wifi_page - component.update: s3_box_lcd else: - display.page.show: initializing_page - component.update: s3_box_lcd switch: - platform: template name: Mute id: mute optimistic: true restore_mode: RESTORE_DEFAULT_OFF entity_category: config on_turn_off: - if: condition: lambda: return !id(init_in_progress); then: - lambda: id(va).set_use_wake_word(true); - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id}; - if: condition: not: - voice_assistant.is_running then: - voice_assistant.start_continuous - script.execute: draw_display on_turn_on: - if: condition: lambda: return !id(init_in_progress); then: - voice_assistant.stop - lambda: id(va).set_use_wake_word(false); - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id}; - script.execute: draw_display globals: - id: init_in_progress type: bool restore_value: no initial_value: 'true' - id: voice_assistant_phase type: int restore_value: no initial_value: ${voice_assist_not_ready_phase_id} image: - file: ${error_illustration_file} id: casita_error resize: 320x240 type: RGB24 use_transparency: true - file: ${idle_illustration_file} id: casita_idle resize: 320x240 type: RGB24 use_transparency: true - file: ${listening_illustration_file} id: casita_listening resize: 320x240 type: RGB24 use_transparency: true - file: ${thinking_illustration_file} id: casita_thinking resize: 320x240 type: RGB24 use_transparency: true - file: ${replying_illustration_file} id: casita_replying resize: 320x240 type: RGB24 use_transparency: true - file: ${loading_illustration_file} id: casita_initializing resize: 320x240 type: RGB24 use_transparency: true - file: https://github.com/esphome/firmware/raw/main/voice-assistant/error_box_illustrations/error-no-wifi.png id: error_no_wifi resize: 320x240 type: RGB24 use_transparency: true - file: https://github.com/esphome/firmware/raw/main/voice-assistant/error_box_illustrations/error-no-ha.png id: error_no_ha resize: 320x240 type: RGB24 use_transparency: true color: - id: idle_color hex: ${idle_illustration_background_color} - id: listening_color hex: ${listening_illustration_background_color} - id: thinking_color hex: ${thinking_illustration_background_color} - id: replying_color hex: ${replying_illustration_background_color} - id: loading_color hex: ${loading_illustration_background_color} - id: error_color hex: ${error_illustration_background_color} spi: clk_pin: 7 mosi_pin: 6 display: - platform: ili9xxx id: s3_box_lcd model: S3BOX data_rate: 40MHz cs_pin: 5 dc_pin: 4 reset_pin: number: 48 inverted: true update_interval: never pages: - id: idle_page lambda: |- it.fill(id(idle_color)); it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_idle), ImageAlign::CENTER); - id: listening_page lambda: |- it.fill(id(listening_color)); it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_listening), ImageAlign::CENTER); - id: thinking_page lambda: |- it.fill(id(thinking_color)); it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_thinking), ImageAlign::CENTER); - id: replying_page lambda: |- it.fill(id(replying_color)); it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_replying), ImageAlign::CENTER); - id: error_page lambda: |- it.fill(id(error_color)); it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_error), ImageAlign::CENTER); - id: no_ha_page lambda: |- it.image((it.get_width() / 2), (it.get_height() / 2), id(error_no_ha), ImageAlign::CENTER); - id: no_wifi_page lambda: |- it.image((it.get_width() / 2), (it.get_height() / 2), id(error_no_wifi), ImageAlign::CENTER); - id: initializing_page lambda: |- it.fill(id(loading_color)); it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_initializing), ImageAlign::CENTER); - id: muted_page lambda: |- it.fill(Color::BLACK); i2c: - id: bus_a sda: GPIO08 scl: GPIO18 - id: bus_b sda: GPIO41 scl: GPIO40 sensor: - platform: aht10 i2c_id: bus_b variant: AHT20 temperature: name: "Temperature" humidity: name: "Humidity" update_interval: 60s - platform: adc pin: GPIO10 name: "Battery voltage" id: battery_voltage unit_of_measurement: "V" accuracy_decimals: 3 device_class: "voltage" entity_category: "diagnostic" update_interval: 30s attenuation: auto filters: - multiply: 4.01 - platform: copy source_id: battery_voltage name: "Battery level" unit_of_measurement: "%" accuracy_decimals: 1 device_class: "battery" entity_category: "diagnostic" filters: - lambda: return (x - 2.5) / (4.2 - 2.5) * 100; - clamp: min_value: 0 max_value: 100 ignore_out_of_range: true # - platform: uptime # name: Uptime # id: uptime_sensor # entity_category: "diagnostic" # update_interval: $wifi_update_interval # on_raw_value: # then: # - text_sensor.template.publish: # id: uptime_human # state: !lambda |- # int seconds = round(id(uptime_sensor).raw_state); # int days = seconds / (24 * 3600); # seconds = seconds % (24 * 3600); # int hours = seconds / 3600; # seconds = seconds % 3600; # int minutes = seconds / 60; # seconds = seconds % 60; # return ( # (days ? to_string(days) + "d " : "") + # (hours ? to_string(hours) + "h " : "") + # (minutes ? to_string(minutes) + "m " : "") + # (to_string(seconds) + "s") # ).c_str(); # - platform: wifi_signal # name: "WiFi Signal dB" # id: wifi_signal_db # update_interval: 60s # entity_category: "diagnostic" - platform: internal_temperature name: "Internal Temperature" entity_category: "diagnostic" touchscreen: platform: gt911 i2c_id: bus_a id: gt911_touchscreen interrupt_pin: GPIO3 on_touch: # - logger.log: # format: Touch at (%d, %d) # args: [touch.x, touch.y] then: - light.toggle: led #text_sensor: # - platform: wifi_info # ip_address: # update_interval: 60s # name: IP Address # entity_category: "diagnostic" # ssid: # update_interval: 60s # name: Connected SSID # entity_category: "diagnostic" # bssid: # update_interval: 60s # name: Connected BSSID # entity_category: "diagnostic" # mac_address: # name: Mac Wifi Address # entity_category: "diagnostic" # scan_results: # update_interval: 60s # name: Latest Scan Results # entity_category: "diagnostic" # Device Uptime Information # - platform: template # name: Uptime Human Readable # id: uptime_human # icon: mdi:clock-start ``` ### Anything in the logs that might be useful for us? ```txt Log after a factory reset: [12:14:40][I][app:102]: ESPHome version 2023.12.3 compiled on Dec 26 2023, 11:38:44 [12:14:40][I][app:104]: Project esphome.voice-assistant version 1.0 [12:14:40][C][wifi:573]: WiFi: [12:14:40][C][wifi:405]: Local MAC: xxxxxxxxxxxxxxx [12:14:40][C][wifi:410]: SSID: [redacted] [12:14:40][C][wifi:411]: IP Address: xxxxxxxxxxxxxxx [12:14:40][C][wifi:413]: BSSID: [redacted] [12:14:40][C][wifi:414]: Hostname: 'esp32-s3-box-3-5ace4c' [12:14:40][C][wifi:416]: Signal strength: -47 dB ▂▄▆█ [12:14:40][C][wifi:420]: Channel: 1 [12:14:40][C][wifi:421]: Subnet: 255.255.255.0 [12:14:40][C][wifi:422]: Gateway: xxxxxxxxxx [12:14:40][C][wifi:423]: DNS1: xxxxxxxxxx [12:14:40][C][wifi:424]: DNS2: xxxxxxxxxxx [12:14:40][C][logger:443]: Logger: [12:14:40][C][logger:444]: Level: DEBUG [12:14:40][C][logger:445]: Log Baud Rate: 115200 [12:14:40][C][logger:447]: Hardware UART: USB_SERIAL_JTAG [12:14:40][C][spi:067]: SPI bus: [12:14:40][C][spi:068]: CLK Pin: GPIO7 [12:14:40][C][spi:069]: SDI Pin: [12:14:40][C][spi:070]: SDO Pin: GPIO6 [12:14:40][C][spi:072]: Using HW SPI: SPI2_HOST [12:14:40][C][i2c.idf:061]: I2C Bus: [12:14:40][C][i2c.idf:062]: SDA Pin: GPIO8 [12:14:40][C][i2c.idf:063]: SCL Pin: GPIO18 [12:14:40][C][i2c.idf:064]: Frequency: 50000 Hz [12:14:40][C][i2c.idf:067]: Recovery: bus successfully recovered [12:14:40][I][i2c.idf:077]: Results from i2c bus scan: [12:14:40][I][i2c.idf:083]: Found i2c device at address 0x18 [12:14:40][I][i2c.idf:083]: Found i2c device at address 0x40 [12:14:40][I][i2c.idf:083]: Found i2c device at address 0x5D [12:14:40][I][i2c.idf:083]: Found i2c device at address 0x68 [12:14:40][C][i2c.idf:061]: I2C Bus: [12:14:40][C][i2c.idf:062]: SDA Pin: GPIO41 [12:14:40][C][i2c.idf:063]: SCL Pin: GPIO40 [12:14:40][C][i2c.idf:064]: Frequency: 50000 Hz [12:14:40][C][i2c.idf:067]: Recovery: bus successfully recovered [12:14:40][I][i2c.idf:077]: Results from i2c bus scan: [12:14:40][I][i2c.idf:083]: Found i2c device at address 0x28 [12:14:40][I][i2c.idf:083]: Found i2c device at address 0x38 [12:14:40][C][gpio.binary_sensor:015]: GPIO Binary Sensor 'Mute' [12:14:40][C][gpio.binary_sensor:016]: Pin: GPIO1 [12:14:40][C][gpio.binary_sensor:015]: GPIO Binary Sensor 'Top Left Button' [12:14:40][C][gpio.binary_sensor:016]: Pin: GPIO0 [12:14:40][C][ledc.output:164]: LEDC Output: [12:14:40][C][ledc.output:165]: Pin GPIO47 [12:14:40][C][ledc.output:166]: LEDC Channel: 0 [12:14:40][C][ledc.output:167]: PWM Frequency: 1000.0 Hz [12:14:40][C][ledc.output:168]: Bit depth: 14 [12:14:40][C][ili9xxx:069]: ili9xxx [12:14:40][C][ili9xxx:069]: Rotations: 0 ° [12:14:40][C][ili9xxx:069]: Dimensions: 320px x 240px [12:14:40][C][ili9xxx:070]: Width Offset: 0 [12:14:40][C][ili9xxx:071]: Height Offset: 0 [12:14:40][C][ili9xxx:077]: Color mode: 16bit [12:14:40][C][ili9xxx:086]: Data rate: 40MHz [12:14:40][C][ili9xxx:088]: Reset Pin: GPIO48 [12:14:40][C][ili9xxx:089]: CS Pin: GPIO5 [12:14:40][C][ili9xxx:090]: DC Pin: GPIO4 [12:14:40][C][ili9xxx:092]: Swap_xy: NO [12:14:40][C][ili9xxx:093]: Mirror_x: YES [12:14:40][C][ili9xxx:094]: Mirror_y: YES [12:14:40][C][ili9xxx:099]: Update Interval: never [12:14:40][C][light:103]: Light 'LCD Backlight' [12:14:40][C][light:105]: Default Transition Length: 0.2s [12:14:40][C][light:106]: Gamma Correct: 2.80 [12:14:40][C][template.switch:068]: Template Switch 'Mute' [12:14:40][C][template.switch:091]: Restore Mode: restore defaults to OFF [12:14:40][C][template.switch:057]: Optimistic: YES [12:14:40][C][psram:020]: PSRAM: [12:14:40][C][psram:021]: Available: YES [12:14:40][C][psram:024]: Size: 16383 KB [12:14:40][C][factory_reset.button:011]: Factory Reset Button 'Factory reset' [12:14:40][C][factory_reset.button:011]: Icon: 'mdi:restart-alert' [12:14:40][C][aht10:150]: AHT10: [12:14:40][C][aht10:151]: Address: 0x38 [12:14:40][C][aht10:155]: Temperature 'Temperature' [12:14:40][C][aht10:155]: Device Class: 'temperature' [12:14:40][C][aht10:155]: State Class: 'measurement' [12:14:40][C][aht10:155]: Unit of Measurement: '°C' [12:14:40][C][aht10:155]: Accuracy Decimals: 2 [12:14:40][C][aht10:156]: Humidity 'Humidity' [12:14:40][C][aht10:156]: Device Class: 'humidity' [12:14:40][C][aht10:156]: State Class: 'measurement' [12:14:40][C][aht10:156]: Unit of Measurement: '%' [12:14:40][C][aht10:156]: Accuracy Decimals: 2 [12:14:40][C][adc:097]: ADC Sensor 'Battery voltage' [12:14:40][C][adc:097]: Device Class: 'voltage' [12:14:40][C][adc:097]: State Class: 'measurement' [12:14:40][C][adc:097]: Unit of Measurement: 'V' [12:14:40][C][adc:097]: Accuracy Decimals: 3 [12:14:40][C][adc:107]: Pin: GPIO10 [12:14:40][C][adc:109]: Attenuation: auto [12:14:40][C][adc:142]: Update Interval: 30.0s [12:14:40][C][copy.sensor:015]: Copy Sensor 'Battery level' [12:14:40][C][copy.sensor:015]: Device Class: 'battery' [12:14:40][C][copy.sensor:015]: State Class: 'measurement' [12:14:40][C][copy.sensor:015]: Unit of Measurement: '%' [12:14:40][C][copy.sensor:015]: Accuracy Decimals: 1 [12:14:40][C][internal_temperature:059]: Internal Temperature Sensor 'Internal Temperature' [12:14:40][C][internal_temperature:059]: Device Class: 'temperature' [12:14:40][C][internal_temperature:059]: State Class: 'measurement' [12:14:40][C][internal_temperature:059]: Unit of Measurement: '°C' [12:14:40][C][internal_temperature:059]: Accuracy Decimals: 1 [12:14:41][C][gt911.touchscreen:105]: GT911 Touchscreen: [12:14:41][C][gt911.touchscreen:106]: Address: 0x5D [12:14:41][C][gt911.touchscreen:107]: Interrupt Pin: GPIO3 [12:14:41][C][GT911.binary_sensor:015]: GT911 Button 'Home Button' [12:14:41][C][GT911.binary_sensor:016]: Index: 1 [12:14:41][C][esp32_ble:379]: ESP32 BLE: bluetooth stack is not enabled [12:14:41][C][esp32_ble_server:200]: ESP32 BLE Server: [12:14:41][C][esp32_improv.component:257]: ESP32 Improv: [12:14:41][C][esp32_improv.component:262]: Status Indicator: 'NO' [12:14:41][C][mdns:115]: mDNS: [12:14:41][C][mdns:116]: Hostname: esp32-s3-box-3-5ace4c [12:14:41][C][ota:097]: Over-The-Air Updates: [12:14:41][C][ota:098]: Address: esp32-s3-box-3-5ace4c.local:3232 [12:14:41][C][api:139]: API Server: [12:14:41][C][api:140]: Address: esp32-s3-box-3-5ace4c.local:6053 [12:14:41][C][api:142]: Using noise encryption: YES [12:14:41][C][improv_serial:032]: Improv Serial: [12:14:42][D][aht10:097]: AHT10 is busy, waiting... [12:14:42][D][sensor:094]: 'Temperature': Sending state 38.71040 °C with 2 decimals of accuracy [12:14:42][D][sensor:094]: 'Humidity': Sending state 22.04561 % with 2 decimals of accuracy [12:14:42][W][component:214]: Component aht10.sensor took a long time for an operation (0.08 s). [12:14:42][W][component:215]: Components should block for at most 20-30ms. [12:14:43][D][button:010]: 'Factory reset' Pressed. [12:14:43][I][factory_reset.button:013]: Resetting to factory defaults... [12:14:43][D][esp32.preferences:176]: Cleaning up preferences in flash... [12:14:43][I][app:134]: Rebooting safely... INFO Processing expected disconnect from ESPHome API for esp32-s3-box-3-5ace4c @ 192.168.0.167 WARNING Disconnected from API INFO Successfully connected to esp32-s3-box-3-5ace4c @ 192.168.0.167 in 7.141s INFO Successful handshake with esp32-s3-box-3-5ace4c @ 192.168.0.167 in 0.320s [12:14:57][D][voice_assistant:422]: State changed from IDLE to START_MICROPHONE [12:14:57][D][voice_assistant:428]: Desired state set to WAIT_FOR_VAD [12:14:57][D][light:036]: 'LCD Backlight' Setting: [12:14:57][D][light:047]: State: OFF [12:14:57][D][light:085]: Transition length: 0.2s [12:14:57][E][spi-esp-idf:081]: Transmit failed - err 101 [12:14:57][W][component:214]: Component voice_assistant took a long time for an operation (0.19 s). [12:14:57][W][component:215]: Components should block for at most 20-30ms. [12:14:57][D][esp32_ble:255]: Disabling BLE... [12:14:57][D][esp-idf:000]: W (14577) BT_APPL: bta_dm_disable BTA_DISABLE_DELAY set to 200 ms [12:14:58][W][component:214]: Component esp32_ble took a long time for an operation (0.21 s). [12:14:58][W][component:215]: Components should block for at most 20-30ms. [12:14:58][D][voice_assistant:159]: Starting Microphone [12:14:58][D][voice_assistant:422]: State changed from START_MICROPHONE to STARTING_MICROPHONE [12:14:58][D][esp-idf:000]: I (14818) I2S: DMA Malloc info, datalen=blocksize=512, dma_buf_count=8 [12:14:58][D][esp-idf:000]: I (14825) I2S: I2S0, MCLK output by GPIO2 [12:14:58][D][esp-idf:000]: I (14831) AUDIO_PIPELINE: link el->rb, el:0x3d036c2c, tag:i2s, rb:0x3d037040 [12:14:58][D][esp-idf:000]: I (14834) AUDIO_PIPELINE: link el->rb, el:0x3d036da0, tag:filter, rb:0x3d039080 [12:14:58][D][esp-idf:000]: I (14841) AUDIO_ELEMENT: [i2s-0x3d036c2c] Element task created [12:14:58][D][esp-idf:000]: I (14848) AUDIO_THREAD: The filter task allocate stack on external memory [12:14:58][D][esp-idf:000]: I (14851) AUDIO_ELEMENT: [filter-0x3d036da0] Element task created [12:14:58][D][esp-idf:000]: I (14856) AUDIO_ELEMENT: [raw-0x3d036ed0] Element task created [12:14:58][D][esp-idf:000]: I (14861) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:16583563 Bytes, Inter:48188 Bytes, Dram:48188 Bytes [12:14:58][D][esp-idf:000]: I (14868) AUDIO_ELEMENT: [i2s] AEL_MSG_CMD_RESUME,state:1 [12:14:58][D][esp-idf:000]: I (14872) AUDIO_ELEMENT: [filter] AEL_MSG_CMD_RESUME,state:1 [12:14:58][D][esp-idf:000]: I (14876) RSP_FILTER: sample rate of source data : 16000, channel of source data : 2, sample rate of destination data : 16000, channel of destination data : 1 [12:14:58][D][esp-idf:000]: I (14879) AUDIO_PIPELINE: Pipeline started [12:14:58][D][esp_adf.microphone:294]: Microphone started [12:14:58][D][voice_assistant:422]: State changed from STARTING_MICROPHONE to WAIT_FOR_VAD [12:14:58][D][voice_assistant:176]: Waiting for speech... [12:14:58][D][voice_assistant:422]: State changed from WAIT_FOR_VAD to WAITING_FOR_VAD [12:15:05][D][sensor:094]: 'Battery voltage': Sending state 3.56853 V with 3 decimals of accuracy [12:15:05][D][sensor:094]: 'Battery level': Sending state 62.85499 % with 1 decimals of accuracy [12:15:27][D][aht10:097]: AHT10 is busy, waiting... [12:15:27][D][sensor:094]: 'Temperature': Sending state 38.69686 °C with 2 decimals of accuracy [12:15:27][D][sensor:094]: 'Humidity': Sending state 22.14909 % with 2 decimals of accuracy [12:15:27][W][component:214]: Component aht10.sensor took a long time for an operation (0.07 s). [12:15:27][W][component:215]: Components should block for at most 20-30ms. [12:15:31][D][esp32.preferences:114]: Saving 1 preferences to flash... [12:15:31][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed [12:15:35][D][sensor:094]: 'Battery voltage': Sending state 3.57047 V with 3 decimals of accuracy [12:15:35][D][sensor:094]: 'Battery level': Sending state 62.96910 % with 1 decimals of accuracy [12:15:37][D][esp-idf:000]: I (54079) tsens: Config temperature range [-10°C ~ 80°C], error < 1°C [12:15:37][D][sensor:094]: 'Internal Temperature': Sending state 40.05940 °C with 1 decimals of accuracy ``` ### Additional information _No response_

m4v3r1ck · January 6, 2024, 5:54pm

Thanks for the heads up.

2024-01-06 18:45:22.304 WARNING (MainThread) [aioesphomeapi.connection] esp32-s3-box-3 @ IP-adres: Connection error occurred: [Errno 104] Connection reset by peer

Will recode it without the i2c. Great new updates for de BOX-3 so far.

Thanks to all participating.

samperk1 · January 6, 2024, 7:57pm

Ollama Conversation

The Ollama integration adds a conversation agent powered by Ollama in Home Assistant.

This conversation agent is unable to control your house. The Ollama conversation agent can be used in automations, but not as a sentence trigger. It can only query information that has been provided by Home Assistant. To be able to answer questions about your house, Home Assistant will need to provide Ollama with the details of your house, which include areas, devices and their states.
the is a custom hacs for home assistant

smarthoo · January 7, 2024, 5:27am

thanks a lot, first some info for forum readers. I am an AI guy and often I see on youtube or read on forums that the local models are considered slow or not as good. The thing is it is up to your budget entirely. Smart home users often spend 1k USD or more on home theater systems and if a 100% private AI assistant is important for you then with about the same spending on hardware you can have such.
You need a computer that runs 24/7/365 with a good AI capable NVIDIA GPU 4090, A6000 or older Tesla T4 and then you can run opensource, 7B or 13B sized, fine tuned and even UNCENSORED and almost weekly updated AI models which are already at ChatGPT3.5 level and some are at ChatGPT4 level. Side note, this hardware can be installed as proxmox hypervisor so you can use the same GPU as virtual GPU for other AI tasks too for example computer vision AI with Frigate home security.

Users often back off due to the software complexity. Yes, you need linux and AI experience and ability to troubleshoot on AI setup however here comes the good news. ollama.ai is offloading you from all the pain of software setup and troubleshooting. So, this is the reason I highly recommend ollama AI over any other local AI solutions (eg localai.io) for non AI experienced people like average home assistant users. So, hopefully people stop promoting local AI as “slow” because this is simply not true unless you don’t have the hardware. Back to the topic, I just really like that ollama has been already discovered by Home Assistant community.

ClaireRovic · January 7, 2024, 9:17am

Thank you for everyone’s help here, my presence sensor is working after selecting ‘Clean Build’ from the drop down and updating using a downloaded file & USB Cable.

For those who want a bit more, I stumbled on the video and added extended AI into HAC’s. So I get a valid answer back when I ask who is ‘Bruce Wayne’.

My project next week is to added a old NVidia 1070 into my Proxmox box, my Bitcoin days are over anyway. @smarthoo will that be a suitable card.