Esphome media player glitching

So I have been building some devices based on esp32 (wroom devkits). Had them for a while but always had an intermittent issue with the media_player.

I’m using a MAX98357a board into a 3w speaker and using TTS (Piper) to output notifications. It works often, but also often it gets weird and crackly, and sometimes doesn’t even bother to play audio at all.

Below is a video which includes some of the glitching when playing an mp3 (excuse the choice it’s just the one where I got a glitch whilst recording :slight_smile:)

My connections are soldered so it’s not a loose jumper issue. I’ve also tried a small resistor on the data and clock lines as suggested elsewhere. I get the symptoms on both devices I’ve built so far, so it’s not specific to a given build.

It feels like the media player gets “stuck” and so sometimes it simply doesn’t output audio even though the media player is “playing”.

Anyone had similar and got any ideas how to root cause the issue?

Thanks

I have a few media players setup, they use S3 boards though so have a bit more memory. They are also used for Voice assistant so a slightly different setup.

Anyhow the only way I can get them to successfully play everything that is thrown at them is to do it via music assistant, and have the speakers setup as loosy in the audio settings.

Hmm OK that doesn’t sound promising. Mine are meant to be assistants too - I did get wakeword working, but but sound is even worse when it’s listening so I’ve removed that and working from the ground up.

I do have some WROVER items (which are separately deployed for my LMS players) but it sounds like if you’re having issues with S3 boards then they won’t help - I would guess the media_player component doesn’t use/benefit from SRAM.

On the music assistant side - I use LMS instead of music assistant, but are you saying that music assistant can both stream text-to-speed to your ESP media_players as well as music - and also be used to respond to voice assists?

It also sounds like, if you get better sound via music assistant than the native media player, that it may be more a bitrate issue of some kind?

Thanks

On my S3 boards the voice assistant with wake word works perfectly, as does playing 75% of the media I ask it to play. Some online radio stations will only play successfully via music assistant. The TTS in voice assistant with wake word is fine, but it only really works well with the S3 n16r8 boards. The S3 boards handle TTS fine.

Use the code from Bigbobbas. This has been working perfectly for a few weeks now. Except the few radio stations I mentioned.

1 Like

Hi

It looks quite similar at issues I have running Voice Assistant on Respeaker Lite that use also I2S. I suspect the I2S implementation is not yet fully working as it should (I have some cracks sometimes before or at end of playing some audios and each time I reflash the ESP !

Vincèn

Ah OK might be worth getting hold of a couple of S3 boards to play.

Having said that, I notice in that yaml it’s using esp-idf framework and media_player. When I tried that previously it said media_player wasn’t supported so I have to use arduino framework.

I notice it’s using a custom audio component so I might give that a try with esp-idf framework see if that makes a difference.

Those examples from gnumpi don’t work anymore for me, the wake word failed to restart so Bigbobas changed the code slightly to the one I linked.

Yes the media player did not work in in earlier versions esp-idf, but it has worked for some time now. I use the beta version of Esphome if makes a difference.

This is my code including the wiring details, and wire colours used.

#  MIC

#  LRC   GPIO5   WS    wt
#  BLC   GPIO6   SCK   gn
#  Din   GPIO4   SD    tq
#  L/R   GND  

#  Speaker

#  LRC   GPIO11   pk
#  BLC   GPIO9    ye
#  Dout  GPIO7    vi

# LED
  
# GPIO14   bl

#  5V

#  Speaker
#  led

#  3V

#  Mic

#  GND

#  Mic
#  Speaker

substitutions:
  node_name: office-assist

ota:
  - platform: esphome
    password: "6add55dxxxxxxxxxxxxxxxxxxx6f5990b0213e902a"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

api:
  encryption:
    key: "b3TMjvVDmS3qitgCw/NjYvoSHjhMdGYfHauRIqRfnBc="

# Copy below text

  on_client_connected:
        then:
          - delay: 50ms
          - light.turn_on:
              id: led_ww
              red: 0%
              green: 0%
              blue: 100%
              brightness: 60%
              effect: none
          - micro_wake_word.start:
  on_client_disconnected:
        then:
          - voice_assistant.stop:


esphome:
  name: ${node_name}
  friendly_name: ${node_name}
  platformio_options:
    board_build.flash_mode: dio
  on_boot:
    - light.turn_on:
        id: led_ww
        blue: 100%
        brightness: 60%
        effect: fast pulse
esp32:
  board: esp32-s3-devkitc-1
  variant: esp32s3
  framework:
    type: esp-idf
    version: recommended

    sdkconfig_options:
        CONFIG_ESP32_S3_BOX_BOARD: "y"
        CONFIG_ESP32_WIFI_STATIC_RX_BUFFER_NUM: "16"
        CONFIG_ESP32_WIFI_DYNAMIC_RX_BUFFER_NUM: "512"
        CONFIG_TCPIP_RECVMBOX_SIZE: "512"
        CONFIG_TCP_SND_BUF_DEFAULT: "65535"
        CONFIG_TCP_WND_DEFAULT: "512000"
        CONFIG_TCP_RECVMBOX_SIZE: "512"

psram:
  mode: octal
  speed: 80MHz

logger:
 # hardware_uart: UART0

button:
  - platform: restart
    name: ${node_name} Restart
    id: but_rest

switch:
  - platform: template
    id: mute
    name: ${node_name} mute
    optimistic: true
    on_turn_on:
      - micro_wake_word.stop:
      - voice_assistant.stop:
      - light.turn_on:
          id: led_ww
          red: 100%
          green: 0%
          blue: 0%
          brightness: 60%
          effect: fast pulse
      - delay: 2s
      - light.turn_off:
          id: led_ww
      - light.turn_on:
          id: led_ww
          red: 100%
          green: 0%
          blue: 0%
          brightness: 30%
    on_turn_off:
      - micro_wake_word.start:
      - light.turn_on:
          id: led_ww
          red: 0%
          green: 100%
          blue: 0%
          brightness: 60%
          effect: fast pulse
      - delay: 2s
      - light.turn_on:
          id: led_ww
          red: 0%
          green: 0%
          blue: 100%
          brightness: 60%
          effect: none
light:
  - platform: esp32_rmt_led_strip
    id: led_ww
    name: ${node_name} LED
    rgb_order: RGB
    pin: GPIO14
    num_leds: 1
    rmt_channel: 0
    chipset: ws2812
    effects:
      - pulse:
      - pulse:
          name: "Fast Pulse"
          transition_length: 0.5s
          update_interval: 0.5s
          min_brightness: 0%
          max_brightness: 100%

external_components:
  - source:
      type: git
      url: https://github.com/gnumpi/esphome_audio
      ref: dev-next
    components: [ adf_pipeline, i2s_audio ]
   # refresh: 0s

adf_pipeline:
  - platform: i2s_audio
    type: audio_out
    id: adf_i2s_out
    i2s_audio_id: i2s_out
    i2s_dout_pin: GPIO7

  - platform: i2s_audio
    type: audio_in
    id: adf_mic
    i2s_audio_id: i2s_in
    i2s_din_pin: GPIO4
    channel: left
    sample_rate: 16000
    bits_per_sample: 32bit
    pdm: false

microphone:
  - platform: adf_pipeline
    id: adf_microphone
    gain_log2: 3
    keep_pipeline_alive: false
    pipeline:
      - adf_mic
      - self

media_player:
  - platform: adf_pipeline
    id: adf_media_player
    name: Media player
    keep_pipeline_alive: false
    internal: false
    pipeline:
      - self
    #  - adf_spk
      - adf_i2s_out

i2s_audio:
  - id: i2s_in
    i2s_lrclk_pin: GPIO5
    i2s_bclk_pin: GPIO6
  - id: i2s_out
    i2s_lrclk_pin: GPIO11
    i2s_bclk_pin: GPIO9

micro_wake_word:
  on_wake_word_detected:
    # then:
    - voice_assistant.start:
        wake_word: !lambda return wake_word;
    - light.turn_on:
        id: led_ww
        red: 0%
        green: 0%
        blue: 100%
        brightness: 60%
        effect: fast pulse
  models:
    - model: hey_mycroft

voice_assistant:
  id: va
  microphone: adf_microphone
  media_player: adf_media_player

  noise_suppression_level: 1
  auto_gain: 31dBFS
  volume_multiplier: 4

  on_stt_end:
    then:
    - light.turn_on:
        id: led_ww
        red: 0%
        green: 100%
        blue: 0%
        brightness: 60%
        effect: fast pulse

  on_error:
    - micro_wake_word.start:
    - light.turn_on:
        id: led_ww
        red: 100%
        green: 0%
        blue: 0%
        brightness: 60%
        effect: fast pulse

 # on_end:
 #   then:
  #  - delay: 2s
 #   - light.turn_on:
 #       id: led_ww
   #     red: 0%
  #      green: 0%
  #      blue: 100%
  #      brightness: 60%
  #      effect: none
   # - wait_until:
     #   not:
     #     voice_assistant.is_running:
 #   - micro_wake_word.start:

  on_end:
    then:
    - light.turn_on:
        id: led_ww
        red: 0%
        green: 0%
        blue: 100%
        brightness: 60%
        effect: none
    - voice_assistant.stop
    - delay: 100ms
    - wait_until:
        not:
          voice_assistant.is_running
    - delay: 100ms
    - micro_wake_word.start