ESP32 a1s Audio kit for voice assistant

ESPHome
23

I put the @Night.Dav3 patches from the PR3552 discussion in a repository if anyone wants to test them out. Change the external_components section from above yaml as shown below.

external_components:
  - source: github://mulcmu/esphome-es8388-a1s@nightdav-patch
    components: [es8388]
    refresh: 0s

  #### Not needed if on 2023.5.0 or later
  # - source: github://pr#4775
  #   components: [adc, i2s_audio, microphone]
  #   refresh: 0s
1 Like
24

Moving the caps looks easy enough do you need to also update the register too or is that in the new patch ?

25

The external component I posted above now has the setup changes recommended by @Night.Dav3 up through this post. So the register changes to get mics working and now enable automatic gain correction for improved voice detection.

It compiles fine for me but I haven’t tested as I messed up while moving one of the capacitors.

26

You have to modify the caps and use the new code to have it working.

@mulcmu : voice assistant uses the right mic in mono mode, so if you do the mod on the right mic only it might work…

1 Like
27

I can’t find anywhere what the size of the JST connectors are for connecting the speakers.
Can anyone provide information?
Thank you

28

Jst-xh 2.54mm

1 Like
29

Thanks for your hard work and assi6 have it working now after moving the caps and using the new code

30

I have the same v2.2 A247 board.
I will move the capacitors to the correct positions.
I would just like to know what is the distance from the microphone so that the assist recognizes the voice command correctly?

31

I had the beard at arms length on the desk and it could still function

32

if got everything working but i wanted to connect a screen to the i2c buss what io pins are these where on the board has to be scl sda ?

33

Nevermind if got it working but the voice recognition or mic is not really god. Is there a possibility to add a external i2s mic ?

34

The esp32 v 2.2 A149 board also had misplaced capacitors. After the fix, voice control works.
I must be doing something wrong. Pressing KEY6 and saying the command to turn the light on and off works, but no voice from the player.
Player esp32-A1s tested via NabuCasa (play voice) and it works.
What else do I need to set in HA for the esp32-A1s player to work (voice response).

1 Like
35

I would like to use an external Mic with mine as well. Did you ever get this working?

36

Has anyone managed to get the wake word assist to work on this board?

37

In my house she appreciates the unfinished projects. They keep me busy.

1 Like
38

A new temporal PR is opened to @mulcmu 's repo. It gives support for the headphone output as well.

39

Fortunatelly, the tts response will be supported in the upcoming release(s), since now only mp3 tts (behind http) is the only supported now. Piper returns wav/raw by default:

Unfortunatelly, https streams (such as tts) are still unsupported:

40

Hello , I got the board to work (STT and TTS) with Home-Assistant but like @pepe59 asked , does anybody knows how to make OpenWakeWord work for this board?

41

Hi, first you have to know that there are at least 7 versions of the A1S Audio Kit. The major difference is a version with AC101 codec and another one with ES8388 but as we can read in this topic there’s also some other differences (like the caps on the wrong places sometimes).

About these versions :

On my side, I have an old A1S board with AC101 codec. Fortunately I’ve found this issue & this repo. It works well (Thanks Sehraf !), I took inspiration from Muse Luxe configuration to create my own ESP configuration forht ethe A1S with AC101 codec that I share with you :

substitutions:
  friendly_name: ESP32 AudioKit - voice assistant

  # Audiokit Buttons
  KEY1: "36"
  KEY2: "13" # may be in use for other purposes, see onboard config switch
  KEY3: "19" # also activates LED D4 if pressed
  KEY4: "23" # do not use on A1S V2.3 with initial pinout -> I2C ES8388
  KEY5: "18" # do not use on A1S V2.3 with initial pinout -> I2C ES8388
  KEY6: "5"  # do not use on A1S V2.3 with initial pinout -> I2S

  # Audiokit LEDs
  LED_D4: "22"
  LED_D5: ${KEY3}
  
  
esphome:
  name: esp32-audio-kit

esp32:
  board: esp-wrover-kit
  framework:
    type: arduino
  # amplifier is not required for headphones
  # on_boot:
  #   then:
  #     - output.turn_on: gpio_amp
external_components:
  source:
    type: git
    url: https://github.com/sehraf/esphome-components
    ref: main
  components: [ac101]
  refresh: 1s
logger:

api:
  encryption:
    key: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  services:
    - service: start_va
      then:
        - voice_assistant.start
    - service: stop_va
      then:
        - voice_assistant.stop



ota:
  password: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

wifi:
  ssid: "MySSID"
  password: "MyWifiPassword"

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "esp32-audio-kit Fallback Hotspot"
    password: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

captive_portal:


#status_led:
#  pin:
#    number: ${LED_D5}
#    inverted: true

web_server:
  port: 80

# AC101 (I2C)
i2c:
  sda: 33
  scl: 32
  scan: true

ac101:
  # address: 0x1a
 

 
media_player:
  - platform: i2s_audio
    id: AudioKit
    name: Media Player
    dac_type: external
    i2s_dout_pin: 25
    mode: stereo
    # on_play:
    #   output.turn_on: gpio_amp
    # on_idle:
    #   output.turn_off: gpio_amp

# I2S Audio
i2s_audio:
  i2s_lrclk_pin: 26
  i2s_bclk_pin: 27

microphone:
  - platform: i2s_audio
    adc_type: external
    i2s_din_pin: 35
    pdm: false
    id: mic_id

switch:
  - platform: gpio
    pin: GPIO21
    name: "AMP Switch"
    restore_mode: ALWAYS_ON
  - platform: template
    name: Use Wake Word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    on_turn_on:
      - lambda: id(va).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);
      - script.execute: reset_led




voice_assistant:
  id: va
  microphone: mic_id
  media_player: AudioKit
  use_wake_word: true
  on_listening:
    - light.turn_on:
        id: top_led
        brightness: 100%
        effect: pulse
  on_tts_start:
    - light.turn_on:
        id: top_led
        effect: none
  on_tts_end:
    - media_player.play_media: !lambda return x;
    - light.turn_on:
        id: top_led
        effect: pulse
  on_client_connected:
    - if:
        condition:
          - switch.is_on: use_wake_word
        then:
          - voice_assistant.start_continuous:
  on_client_disconnected:
    - if:
        condition:
          - switch.is_on: use_wake_word
        then:
          - voice_assistant.stop:
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          media_player.is_playing: AudioKit
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: top_led
        effect: none
    - delay: 1s
    - script.execute: reset_led
    - script.wait: reset_led
    - lambda: |-
        if (code == "wake-provider-missing" || code == "wake-engine-missing") {
          id(use_wake_word).turn_off();
        }
        
script:
  - id: reset_led
    then:
       - light.turn_off: top_led
    
binary_sensor:
  - platform: gpio
    pin:
      number: GPIO39
      inverted: true
      mode:
        input: true
    name: "Jack Status"

  - platform: gpio
    pin:
      number: GPIO036
      inverted: true
    name: "Key 1"
    filters:
      - delayed_off: 10ms

  - platform: gpio
    pin:
      number: GPIO013
      inverted: true
    name: "Key 2"
    filters:
      - delayed_off: 10ms

#  - platform: gpio
#    pin:
#      number: GPIO019
#      inverted: true
#    name: "Key 3"
#    filters:
#      - delayed_off: 10ms
      
  - platform: gpio
    pin:
      number: GPIO023
      inverted: true
      mode:
        input: true
        pullup: true
    name: "Key 4"
    filters:
      - delayed_off: 10ms
      
  - platform: gpio
    pin:
      number: GPIO018
      inverted: true
      mode:
        input: true
        pullup: true
    name: "Key 5"
    filters:
      - delayed_off: 10ms
      
  - platform: gpio
    pin: 
      number: GPIO005
      inverted: true
      mode:
        input: true
        pullup: true
    name: "Key 6"
    filters:
     - delayed_off: 10ms

    on_press:
      - voice_assistant.start:
    on_release:
      - voice_assistant.stop:
    on_click:
      - media_player.toggle: AudioKit  

light:
  - platform: binary
    name: "LED D4"
    id: top_led
    output: light_output_1
  - platform: binary
    name: "LED D5"
    output: light_output_2
 
output:
  - id: light_output_1
    platform: gpio
    pin: GPIO22
    inverted: true
  - id: light_output_2
    platform: gpio
    pin: GPIO19
    inverted: true

It’s not perfect but it allows to start to play with Voice Assistant with this great board.
Some issues :

  • At boot it often enter in a crazy loop on wake word initialization (like this on this topic or this one). Just disable “Use Wake Word” toggle and enable it again to stop this loop. It seems to be related to the “voice_assistant” automations configured. A workaround could be to start Voice Assistant only after few seconds (as it seems related to the I2S microphone not initialized or something like that).
  • The jack is working however I didn’t found how to detect the jack to disable the amp if necessary (pin 39 doesn’t seem to work).
  • GPIO19 is muxed so I didn’t found how to use the key3 and the LED D5 simultaneously (ESPHOME refuse to use the same GPIO two times)
  • The media player is working but require to disable “Use Wake Word” first. May be we could find a trick in HA to disable it automatically when something is played.
  • Volume is reset to 100% after each restart.

So it can be improved a lot, for example the keys could be mapped for Media Player. I’m sure that the issues could be fixed by someone with a great debugging level on ESPHome

I hope that it will help some of you, once configured the result is not bad at all, even if only one microphone is used, the wake word detection is not bad at all even at several meters or by whispering. The media player is working fine, the speakers and the audio jack output too. Even if Voice Assist is far from perfect, at least my board has a use now :wink:

5 Likes
42

Thanks for sharing!