"ReSpeaker Lite" - new Seeed Studio Voice Assistant Development Kit hardware combine ESP32 with XMOS XU316 DSP chip for advanced audio processing as a ESPHome-based Home Assistant Assist Satellite voice devkit

@formatBCE
Thanks for sharing!
First one works like a charm.
Couple of hours later there is again an error.

src/esphome/components/voice_assistant/voice_assistant.cpp:28:32: error: definition of implicitly-declared 'esphome::voice_assistant::VoiceAssistant::VoiceAssistant()'
 VoiceAssistant::VoiceAssistant() {
                                ^

Is this the same on your side?

1 Like

I’ve replace my yaml with your new version and only edited it to add my ssid, password and the bit for giving a fixed IP address. Also commented out the ota: section as I was getting an error about no password.

Now when I validate it I get the following.

INFO ESPHome 2024.10.0
INFO Reading configuration /config/esphome/respeaker.yaml...
INFO Updating https://github.com/esphome/voice-kit@dev
INFO Updating https://github.com/esphome/esphome.git@pull/7605/head
Failed config

button.safe_mode: [source /config/esphome/respeaker.yaml:168]
  
  Component button.safe_mode requires component safe_mode.
  platform: safe_mode
  id: button_safe_mode
  name: Safe Mode Boot

I know nothing about ESPHome - just a blind copier of code.
Any help would be appreciated. Thanks.

I actually have no idea, never seen this… :frowning:

Yes, there’s change in HA PE software that broke things. They promised to fix it tomorrow, but I also forked their repo and rolled back breaking change. But I didn’t post new YAML, pointing to my fork, sorry. Will do it in couple hours.

1 Like

Great to hear.
No hurry.
I will try bymyself. I was just curious if i breakup something.

You can use my fork.
url: https://github.com/formatBCE/home-assistant-voice-pe
ref: format_bce

2 Likes

I’ve updated the YAML in the repo.

2 Likes

Thanks for your efforts on this.

I inferred that the updated yaml would get me over the error; but it doesn’t; I get the same error. Do I need to wait till the HA PE software gets fixed tomorrow?

If you replaced external component with my repo, it should be working. Probably something is with your YAML… This shouldn’t happen.
Try cleaning build files, restarting ESPHome.

Make sure you’re building on good machine - it requires a lot of CPU power.

I’ve re-started ESPHome and cleaned the build files and I still get the error

INFO ESPHome 2024.10.0
INFO Reading configuration /config/esphome/respeaker.yaml...
INFO Updating https://github.com/formatBCE/home-assistant-voice-pe@format_bce
INFO Updating https://github.com/esphome/esphome.git@pull/7605/head
Failed config

button.safe_mode: [source /config/esphome/respeaker.yaml:170]
  
  Component button.safe_mode requires component safe_mode.
  platform: safe_mode
  id: button_safe_mode
  name: Safe Mode Boot

The external components bit looks like this

external_components:
  - source:
      type: git
      #url: https://github.com/esphome/voice-kit
      #ref: dev
      url: https://github.com/formatBCE/home-assistant-voice-pe
      ref: format_bce
    components:
      - aic3204
      - audio_dac
      - media_player
      - micro_wake_word
      - microphone
      - nabu
      - nabu_microphone
      - voice_assistant
      - voice_kit
    refresh: 0s
  - source: github://pr#7605
    components: [ audio, i2s_audio, speaker]
    refresh: 0s

I tried commenting out the 3 lines relating to the safe_mode button and that validates OK. I will try and install it but I don’t know what the button is about!

My host for HA is an HP T630 thin client running generic x86-64.

Well, you can delete that button. IDK why it’s giving you hard times tho. :slight_smile:

My guess is you are running into a stack overflow error due to the https connection. Getting serial logs would confirm this. I merged a change today into the main repo that increases the task stack size to hopefully avoid this. If you would try it out, I would appreciate it!

1 Like

PE repo is fixed, I synced latest changes to my fork, just use ref: dev. Or use direct link to voice-kit repo, as it was before.

1 Like

I also made some additions to the kit functionality.
It has now software mute switch, bound to hardware one, and sensor with current firmware version.

Will update repo tomorrow.

Also, I want to investigate possibility to use voice-kit component (maybe, extend it) with Respeaker DFU. They should be very similar, looks like.

2 Likes

I updated to latest esphome and respeaker wont update. Just FYI.

INFO ESPHome 2024.10.1
INFO Reading configuration /config/esphome/respeaker.yaml…
INFO Updating https://github.com/esphome/voice-kit@dev
INFO Unable to import component audio: No module named ‘esphome.components.audio’
Failed config

audio: None
{}
Component not found: audio

Did you update your external components like below? There was a PR for that a while ago so you have to use the below. I just changed some things and it ran with zero issues.

external_components:
  - source:
      type: git
      url: https://github.com/esphome/voice-kit
      ref: dev
    components:
      - aic3204
      - audio_dac
      - media_player
      - micro_wake_word
      - microphone
      - nabu
      - nabu_microphone
      - voice_assistant
      - voice_kit
    refresh: 0s
  - source: github://pr#7605
    components: [ audio, i2s_audio, speaker]
    refresh: 0s

Also, looking at the PE repo, I got stop to work which is nice! Now I can tell it to shut up.

I got it installed and working without the safe.mode button (I still don’t understand what it is I’m missing).

But now I get no voice response to anything, whereas I did with the Seeed code. Looking at the logs I see a section like this each time

20:50:47][D][voice_assistant:697]: Response: "Turned on the light"
[20:50:47][D][light:036]: 'respeaker-satellite' Setting:
[20:50:47][D][light:051]:   Brightness: 60%
[20:50:47][D][light:059]:   Red: 20%, Green: 100%, Blue: 100%
[20:50:47][D][light:109]:   Effect: 'Slow Pulse'
[20:50:47][D][voice_assistant:641]: Event Type: 8
[20:50:47][D][voice_assistant:719]: Response URL: "https://[MyURL]:8123/api/tts_proxy/104c89b5f9053e4751d03002aab527c96124bd77_en-gb_183ef190f5_tts.piper.mp3"
[20:50:47][D][voice_assistant:518]: State changed from AWAITING_RESPONSE to STREAMING_RESPONSE
[20:50:47][D][voice_assistant:524]: Desired state set to STREAMING_RESPONSE
[20:50:47][D][media_player:080]: 'Media Player' - Setting
[20:50:47][D][media_player:087]:   Media URL: https://[MyURL]:8123/api/tts_proxy/104c89b5f9053e4751d03002aab527c96124bd77_en-gb_183ef190f5_tts.piper.mp3
[20:50:47][D][media_player:093]:  Announcement: yes
[20:50:47][D][voice_assistant:641]: Event Type: 2
[20:50:47][D][voice_assistant:733]: Assist Pipeline ended
[20:50:47][D][esp-idf:000][ann_read]: E (102090910) esp-tls: couldn't get hostname for :[MyURL]: getaddrinfo() returns 202, addrinfo=0x0

[20:50:47][D][esp-idf:000][ann_read]: E (102090914) esp-tls: Failed to open new connection

[20:50:47][D][esp-idf:000][ann_read]: E (102090915) TRANSPORT_BASE: Failed to open a new connection

[20:50:47][D][esp-idf:000][ann_read]: E (102090916) HTTP_CLIENT: Connection failed, sock < 0

[20:50:47][E][nabu_media_player.pipeline:173]: Media reader encountered an error: ESP_ERR_HTTP_CONNECT
[20:50:47][E][nabu_media_player:308]: The announcement pipeline's file reader encountered an error.
[20:50:47][D][voice_assistant:518]: State changed from STREAMING_RESPONSE to IDLE
[20:50:47][D][voice_assistant:524]: Desired state set to IDLE
[20:50:47][D][light:036]: 'respeaker-satellite' Setting:
[20:50:47][D][light:047]:   State: OFF
[20:50:47][D][light:109]:   Effect: 'None'

Any ideas?

Did you strip out the IP and put in [URL] or is that straight from the logs? If it’s straight from the logs there should be the IP of you HA server there. This is the logs from a voice command that runs successfully for me. Only difference I see is you are using Piper and I am using Nabu cloud but that is not the issue.

[15:09:46][D][voice_assistant:516]: State changed from STOPPING_MICROPHONE to AWAITING_RESPONSE
[15:09:46][D][voice_assistant:516]: State changed from AWAITING_RESPONSE to AWAITING_RESPONSE
[15:09:46][D][voice_assistant:639]: Event Type: 4
[15:09:46][D][voice_assistant:667]: Speech recognised as: "What's tomorrow's forecast?"
[15:09:46][D][voice_assistant:639]: Event Type: 5
[15:09:46][D][voice_assistant:672]: Intent started
[15:09:46][D][voice_assistant:639]: Event Type: 6
[15:09:46][D][voice_assistant:639]: Event Type: 7
[15:09:46][D][voice_assistant:695]: Response: "Tomorrow's forecast is Warm with the temperature approaching the record of 32 set in 1927 with plenty of sun"
[15:09:46][D][light:036]: 'respeakerv3' Setting:
[15:09:46][D][light:051]:   Brightness: 60%
[15:09:46][D][light:059]:   Red: 20%, Green: 100%, Blue: 100%
[15:09:46][D][light:109]:   Effect: 'Slow Pulse'
[15:09:46][D][voice_assistant:639]: Event Type: 8
[15:09:46][D][voice_assistant:717]: Response URL: "http://192.168.0.30:8123/api/tts_proxy/5f1067e2e0df89c9763b15a174481a9dfca55550_en-us_753ca9f990_tts.home_assistant_cloud.flac"
[15:09:46][D][voice_assistant:516]: State changed from AWAITING_RESPONSE to STREAMING_RESPONSE
[15:09:46][D][voice_assistant:522]: Desired state set to STREAMING_RESPONSE
[15:09:46][D][media_player:080]: 'Media Player' - Setting
[15:09:46][D][media_player:087]:   Media URL: http://192.168.0.30:8123/api/tts_proxy/5f1067e2e0df89c9763b15a174481a9dfca55550_en-us_753ca9f990_tts.home_assistant_cloud.flac
[15:09:46][D][media_player:093]:  Announcement: yes
[15:09:46][D][voice_assistant:639]: Event Type: 2
[15:09:46][D][voice_assistant:731]: Assist Pipeline ended
[15:09:47][D][nabu_media_player.pipeline:174]: Reading FLAC file type
[15:09:49][D][nabu_media_player.pipeline:186]: Decoded audio has 1 channels, 16000 Hz sample rate, and 16 bits per sample
[15:09:49][D][nabu_media_player.pipeline:211]: Converting mono channel audio to stereo channel audio

i’m curious why it’s https for you. This is causing a lot of troubles.

I would go ahead and post your full config taking out the API, OTA, and AP keys out. AP probably doesn’t matter much as that is just so you can connect to give it your wifi creds if it can’t connect. As @formatBCE said and I didn’t even notice, it shouldn’t be doing anything over https, it should be http…

Do you have DuckDNS setup for external access and can you get tp HA on your LAN via http://xxx.xxx.x.x:8123 using whatever the internal IP of your HA server is? Only thing I can think of that would try to force https