ReSpeaker XMOS XVF3800 ESPHome integration

Hmm, “next timer” field should update name, if it’s not - it’s bug.

On the other things: don’t confuse timers with alarm :slight_smile:

Timers are the same as Voice PE does them. They’re voice-only, and use ring to show progress.

Alarm will go off AT the given time, not IN some time like timers. I just use behaviour and sound from timers for the alarm.

I did confuse those. Thanks.

I did this:

“Set a timer for 3 minutes and call it hamburger”

a bit later

“Set a timer for 3 minutes and call it Hot Dog”

shows timer seconds and hamburger. Timer goes off on schedule,. I say ‘stop’. Next timer goes to 20s, but next timer name does not update.

Did you ever make a 3d print for this? i would love to do the same thing, but i havent used cad for 15 years or so, so i would love it if i could get the file? :slight_smile:

Been using it for almost a year, and it’s pretty good. Recently I started playing my music through it so I could take advantage of the AEC, but my soundbar auto turns off and can’t be woken from 3.5", so it’s not great for tts responses. So is it possible to plug a speaker into the speaker port on the board and have it use that for voice and the 3.5" for music?

If it would be possible, it would need changes in DFU firmware - not something I can do, it’s courtesy of Seeed.
Also it would make audio routing very hard on ESPHome side.

Okay, quick question, because I don’t have a speaker with the right connector to test right now; how does the board determine where audio goes? It it simply “something is plugged into 3.5” port means that’s the target" or does it try both? I’m wondering if I can hook a speaker into the onboard port and still leave it plugged into speakers on the 3.5" port, and have it use the 3.5" port when it’s active and the onboard speaker if it’s not. Or would I have to plug/unplug the speakers manually?

Honestly - I don’t know. It depends on DFU firmware. It’s definitely switching automatically to 3.5mm, when it’s inserted - don’t know about active/off.

I have four of these in use now, and three VPE’s, and want to share some thoughts.

First, when I started I found I could not get STT to work with channel 0 (as in the code) and switched to channel 1. I used it ever since, but did some analysis now to find out why and I am baffled. Back initially my speech was transcribed garbled with channel 0 using the default (Wyoming I think). I did not know at the time how to capture the wav files, only that 1 worked, zero did not. Well now I have a lot more data, but more baffling is that channel 0 now works fine.

Channel 1 is different from 0, notably it has MUCH lower gain (or something similar to gain) a test with a pre-recorded sentence analyzing the wav file afterwards on two separate reSpeaker devices showed a 7dbFS and 14dbFS difference with channel 0 being louder/hotter. Channel 0 also had more energy in the speech frequencies whereas channel 1 had a lot of low/high frequency (and this from identical recording played in the same room). So they are processed differently.

What I lack is any explanation for my garbled STT back on day 1, despite it being with 1.0.7 and this code. But… it’s not garbled now.

This also plays into the biggest issue I have now – I have about 2700sf and 7 satellites. This matches where I had google mini’s (and one max pair and one “home” pair). The issue is that these (even the VPE’s) are VERY sensitive at hearing a wake word, and VERY bad at picking the right satellite. The satellite chosen with duplicates is the one where the packet arrives at HA first, and all of these are wifi and there’s more variation there than the speed of sound provides distinction, despite me also having 5 different AP’s to give good coverage in the 2.4 bands. So I’ll be standing right beside a satellite, and “Hey Mycroft” and nothing appears to happen – it’s because a satellite 40’ away won and is active.

Having the wrong one answer is annoying, but what was happening (and the channel plays into this I think) is that while the wake word was heard, STT might not work so my command did not work. I THINK with channel 0 selected that will happen less. It’s still the wrong satellite of course, but less likely to get the STT wrong.

However, this brings up something I changed and would like to recommend: An LED indication if a satellite was one selected as duplicate. This code:

globals:
  - id: duplicate_wake_detected
    type: bool
    restore_value: no
    initial_value: 'false'
  
voice_assistant:
  id: va
  microphone:
    microphone: i2s_mics
    channels: -0
  on_error:
    - if:
        condition:
          lambda: |-
            return code == "duplicate_wake_up_detected";
        then:
          - lambda: |-
              id(duplicate_wake_detected) = true;
  on_end:
    - if:
        condition:
          lambda: |-
            return id(duplicate_wake_detected);
        then:
          - script.execute:
              id: led_set_effect
              effect: "solid"
              r: 255.0
              g: 180.0
              b: 0.0
              speed: 0.0
              brightness: 1.0
          - delay: 1s
          - lambda: |-
              id(duplicate_wake_detected) = false;
          - lambda: |-
              id(led_force_update) = true;
          - script.execute:
              id: control_leds

will give a yellow flash of the ring if it actived on the wake word but was rejected by HA as being too late to the party. So if you are near a satellite you at least know you were heard and ignored by this one, but more to the point that some other satellite heard you. Can’t tell which without looking around the rooms for the listening lights, but at least you don’t keep saying the wake word over and over trying to wake this one up. The code is not pretty and doesn’t have a drop down selector for color and such, but it’s pretty simple to see what it does. Note this also has the channel selection left in, but at the default 0 – you don’t need that part unless you want to play with the two channels.

But I find the yellow duplicate circle a nice feature for the duplicate wake word issue.

On a related note, I played with both the window and probability cutoffs in the wake word, as well as gain (it already is at minimum 1 on channel 1) and simply cannot get them LESS sensitive to wake word, so the closer wins by default. Almost everything I say, despite maybe 30-40’ and in some cases walls in line of site (but open doors), gets at least one duplicate. We REALLY need HA to provide a more sophisticated arbitration of the winner – gain, SNR, something that would be better closer.

Of course, this works OK if you have only 1 of these in your house. The issue becomes if your house gets noisy – with people, TV, etc. Then it pays to have one close, so you are closer to it than the source of the noise. That’s why I have quite a few, not because I need them when it’s quiet, but when it’s noisy. But when it’s quiet it’s a real pain.

Anyway, just an update. Oh… I also with my nice big speaker and 3D printed diffuser set them up in Music Assistant. While mono and a $15 speaker is involved, they sound surprisingly good - much better than a google mini (about like a google home).