"ReSpeaker Lite" - new Seeed Studio Voice Assistant Development Kit hardware combine ESP32 with XMOS XU316 DSP chip for advanced audio processing as a ESPHome-based Home Assistant Assist Satellite voice devkit

Can anyone explain why this happens now if I try to validate or update via ESPHome dashboard?

INFO ESPHome 2024.10.0
INFO Reading configuration /config/esphome-web-7e1a70.yaml...
INFO Updating https://github.com/esphome/voice-kit@dev
INFO Unable to import component audio: No module named 'esphome.components.audio'
Failed config

audio: None
  {}
Component not found: audio

I’m guessing that something changed @ https://github.com/esphome/voice-kit@dev that has broken things.

1 Like

I’m getting the same error now that I have updated ESPHome to 2024.10.0!

Is it something I need to load locally or is it a bug that needs sorting by the experts at source.?

Same here
Works fine before, and now i get this validation error.
Not sure if its from https://github.com/esphome/voice-kit@dev or from the ESPHome Update

There was a breaking change in the upstream repo. It relies on a PR that is only merged into the ESPHome dev branch. As an alternative, you can use the code in the PR as an external component:

external_components:
  - source: github://pr#7605
    components: [ audio, i2s_audio, speaker]

The Nabu media player now outputs to a speaker component, not to the I2S bus directly. So try something like this (borrowed from formatBCE on Discord):

speaker:
  - platform: i2s_audio
    sample_rate: 16000
    i2s_mode: secondary
    i2s_dout_pin: GPIO43
    bits_per_sample: 32bit
    i2s_audio_id: i2s_output
    dac_type: external
    channel: left

media_player:
  - platform: nabu
    id: nabu_media_player
    name: Media Player
    internal: false
    speaker:
    sample_rate: 16000
    volume_increment: 0.05
    volume_min: 0.4
    volume_max: 0.85
    on_announcement:
      - nabu.set_ducking:
          decibel_reduction: 20
          duration: 0.0s
    on_state:
      if:
        condition:
          and:
            - switch.is_off: timer_ringing
            - not:
                voice_assistant.is_running:
            - not:
                lambda: return id(nabu_media_player)->state == media_player::MediaPlayerState::MEDIA_PLAYER_STATE_ANNOUNCING;
        then:
          - nabu.set_ducking:
              decibel_reduction: 0
              duration: 1.0s

I don’t have a Respeaker device to test this on directly, so you may need to make some adjustments here or there depending on your current yaml config.

1 Like

I’ve updated my YAML in repo with these changes. Everything works great. Also I’ve added newest XMOS firmware by Seeed there.

P.S. Added README to the repo. PRs are welcome. :slight_smile:

3 Likes

Created enclosure for it, now i can place it in my living room.
P.S. Yes i know it’s “heavily inspired” by Echo Pop - i just liked that design. :slight_smile:

6 Likes

Are you noticing errors with audio out? Like the assistant tts speech output? I keep running into the issue where I only hear the last half a second of the reply (usually last few milliseconds). Which is annoying if there was an error or it didn’t understand me because I need to go to the Hass assistant config and go through the debug modals.

Same thing with audio files, I have a doorbell.mp3 and on the first play, I get the last millisecond of it, but on the 2nd+ play it works fine.

Makes me think that the output isn’t waiting for the speaker to be initialized or something, just blindly fires the audio data out.

With newest YAML it shouldn’t be the case. Before i had troubles with TTS/audio cutting off. But after yesterday’s breaking changes it all gone.

2 Likes

Yay, going to give the new stuff a try. Thanks for your work!

1 Like

Do you know what the change log notes are for the XMOS 1.0.9 firmware? on the seeed wiki it reports 1.0.8 as the current release, just curious.

I have tried the newer 1.0.9 firmware, and the latest code from formatBCE and I still have the same issue. It works flawlessly except for this reboot whenever it tries to play audio issue. And it is frustrating because it plays audio just fine using the seeed studio config yaml, so my device is capable of playing audio.

Is anyone else having/had this issue? To clarify, I don’t get any part of the return audio. It hard reboots immediately before playing anything.

Are you using the Kit (or another speaker) via JST connector or 3.5mm for audio output? Does it hard reboot Everytime you do a command or is it sporadic? If you can reproduce then open ESPHome and go to logs for the respeaker and post it here. Logs should clearly show it lose connection at a minimum.

It’s also never a bad step to delete the device from integration, clean build files on respeaker in ESPHome, then do new install, reboot HA and add it back. Reboot may not be needed but at that point might as well. That or clean rebuild, delete integration and reboot.

EDIT: just wanted to add that I have not had this issue and now I just jinxed myself

1.0.9 has changed to expose mute state through i2c. I want to make custom component to expose mute switch programmatically.

1 Like

Strange, I have several devices on 1.0.9, no issues playing media… How do you play it? Via HA media picker? There’s trouble playing some media, it’s ongoing issue with PE software. Try playing via Music Assistant maybe?

What do the esphome logs say before, during and after the reboot?

I am still using 1.0.7 xmos fw.

@formatBCE
Thanks for sharing!
First one works like a charm.
Couple of hours later there is again an error.

src/esphome/components/voice_assistant/voice_assistant.cpp:28:32: error: definition of implicitly-declared 'esphome::voice_assistant::VoiceAssistant::VoiceAssistant()'
 VoiceAssistant::VoiceAssistant() {
                                ^

Is this the same on your side?

1 Like

I’ve replace my yaml with your new version and only edited it to add my ssid, password and the bit for giving a fixed IP address. Also commented out the ota: section as I was getting an error about no password.

Now when I validate it I get the following.

INFO ESPHome 2024.10.0
INFO Reading configuration /config/esphome/respeaker.yaml...
INFO Updating https://github.com/esphome/voice-kit@dev
INFO Updating https://github.com/esphome/esphome.git@pull/7605/head
Failed config

button.safe_mode: [source /config/esphome/respeaker.yaml:168]
  
  Component button.safe_mode requires component safe_mode.
  platform: safe_mode
  id: button_safe_mode
  name: Safe Mode Boot

I know nothing about ESPHome - just a blind copier of code.
Any help would be appreciated. Thanks.

I actually have no idea, never seen this… :frowning:

Yes, there’s change in HA PE software that broke things. They promised to fix it tomorrow, but I also forked their repo and rolled back breaking change. But I didn’t post new YAML, pointing to my fork, sorry. Will do it in couple hours.

1 Like

Great to hear.
No hurry.
I will try bymyself. I was just curious if i breakup something.