There is a good app note for the esp32-p4 that has no radio so very much benefits from companion radio version esp32.
It sort of goes through the various forms in terms of function and complexity.
There is a good app note for the esp32-p4 that has no radio so very much benefits from companion radio version esp32.
It sort of goes through the various forms in terms of function and complexity.
Personally I would rather use the P4’s dedicated LAN pins and use Ethernet simply due to latency and maybe a bit more bandwidth. The ESP32-S3 has about 256kbps bandwidth. You might get a spike above 500kbps at first but it quickly drops down to 200 to 250kbps. This is what’s documented and matches the speeds I get when sending a file to my P1 via SFTP so no cloud involved. I don’t think this would help with the speech recognition part as if 16000Khz is enough then I doubt any added bandwidth would help but audio playback over JST would more then likely benefit as even Spotify is like 350 to 400kbps.
I ended up getting an Nvidia Jetson Orin NX 16GB and using the whisper/piper/OpenWakeWord (not sure if it’s needed at this point) that Nvidia and Nabu worked on together is night and day compared to my Intel NUC. I’ve been meaning to try Assist Microphone on the Jetson Along with a small LLM models like llama3.2b it works great for general questions, especially with the newish “fallback for local command” option. The problem with an LLM completely controlling Home Assistant is it requires a much larger model, at least until someone creates once specifically for HA which I have no doubt will happen but I also have no idea on the timetable. I’m not sure if the new way cheaper 8GB model, which is just an Orin NX 8GB with a new name would work nearly as good. I’m already over 8GB of RAM after one question so at that point you would be on swap and that will slow things down significantly. If I would of had a motherboard and other parts I would have just bought a GPU but once your at the point were outside maybe a case and power supply is all I have the Jetson was the cheaper way to go.
The one thing that the respeaker lite and HA PE have that I wouldn’t want to not have with an LLM is a stop word. I honestly had good luck with the Wyoming Satellite but it was a stop gap measure until Nabu got it working on ESP32. I highly doubt you will see any future improvements but I could be wrong. Also, even with noise and Echo cancelation they just don’t work well around TV’s, especially something like the news so I created 3 template sensors for the assist_satellite phases. Then created 2 automations. The first takes a scene snapshot of my soundbar and lowers it to a low enough volume where voice commands always work. The second automation restores the snapshot when it goes from listening to processing or processing to replaying as local commands tend to go so fast they don’t catch the processing phase but LLM questions do. Mute works also if you can’t control the actual sound level on the audio device. For anyone wanting to make their own JST speaker connector Minor soldiering skills required.
Completely local HA command
General question
Just in case it helps anyone
respeaker_listening_phase:
value_template: >
{% if is_state('assist_satellite.respeaker_assist_satellite', 'listening') %}
listening
{% else %}
no
{% endif %}
respeaker_thinking_phase:
value_template: >
{% if is_state('assist_satellite.respeaker_assist_satellite', 'processing') %}
processing
{% else %}
no
{% endif %}
respeaker_replying_phase:
value_template: >
{% if is_state('assist_satellite.respeaker_assist_satellite', 'responding') %}
replying
{% else %}
no
{% endif %}
I made same automation, but put it into one instead of two, and made that generic (i have 5 satellites in different areas, and each of them is lowering the volume on playing players in corresponding area (except of itself, of course, because i want to hear an answer and it ducks active audio anyways).
So here I am, back with a Respeaker Lite blue LED flashing fast
Context:
Respeaker running firmware supplied by @formatBCE
Blue LED pulsing slowly
I switched off the Respeaker and moved it from one room to an adjacemt room and plugged it in.
Respeaker blue LED flashes fast and stays that way all night
I noticed there was an update available in ESPHome Builder so I ran it OTA this morning, which seemed to work okay
Blue LED still flashing fast. Again, the device seems to be working okay, I just cannot understand why the change in behaviour of the blue LED from one room to an adjacent room. In fact it’s literally sitting on the other side of a stud partition wall, connected to the same network and same IP address.
Is this going to happen every time I move one?
This is not a moan by the way. I’m just trying to understand what is going on so I can fix it if something has gone awry and avoid it again in future
There’s no reason for it to behave like this. Could it be that USR button unsoldered? Because this is the only case I saw something like this…
Everything happens for reason, and I didn’t have such problems on any of mines…
@formatBCE
Hey, first of all thanks for your git repo. Been using stuff from there in one satellite I’m currently testing in the kitchen, and works quite good so far. Few hickups but nothing critical.
I wonder if its possible to set a fixed volume for the announcements? Because they are quite loud compared to hearing music at some volume level. So it would be great to change volume to a fixed level just for the speech and music to the “normal volume slider” value.
Hmm it should be possible in the media player module itself - but I’m not owning it, it’s NabuCasa library. So someone would have to fork it and make that available…
The USR button connection is fine. I followed the same process as the last time and reflashed both the XMOS and ESP firmware. I took the latest version of YAML from your repo.
LED is still flashing fast. Interstingly, while looking through the respeaker-satellite-base.yaml
saw this section:
# Script executed during initialization
# Fast Blue pulse if Wifi is connected, Else slow blue pulse
- id: control_leds_init_state
then:
- if:
condition:
wifi.connected:
then:
- light.turn_on:
brightness: 50%
red: 9.4%
green: 73.3%
blue: 94.9%
id: led_respeaker_onboard
effect: "Fast pulse"
else:
- light.turn_on:
brightness: 50%
red: 9.4%
green: 73.3%
blue: 94.9%
id: led_respeaker_onboard
effect: "Slow Pulse"
This suggests to me that the blue LED is exhibiting normal behaviour (albeit the new normal). When the device initialises, the LED pulses slowly for a few seconds then start flashing.
However, if this is the new normal, then that doesn’t explain why the LED on my other Respeaker Satellite (which is running the same firmware) pulses slowly.
Both are connected and working fine
This script should work on initialization step only. When init goes to false, other light effects take precedence.
This is very weird stuff.
Thanks @formatBCE and @rananna for your effort, time and sharing it to all of us
My little robo have a respeaker lite with integrated xiao esp32s3 inside and it works fantastic.
Michael
That’s not satellite, that’s state of the Art!
Did you ever finish this? Do you make the STL files available somewhere?
Hey everyone!
Soon (on next ESPHome major release) the firmware for PE will be reformatted to use newly-merged media_player instead of custom component. That’s great news, and i’ll look closely to introduce these changes to the Respeaker/Koala as soon as possible.
Also there’s some improvement coming (allegedly) soon from Seeed, that will improve noice cancellation and AEC for our device. Will update you as soon as i have it.
Meanwhile i’m seeing a lot of requests for Openwakeword usage for PE-like devices. While this is inferior to Microwakeword (which is local on-device unlike OWW), i guess i could add the possibility to use OWW instead of MWW (with drop-down, i guess). What do you think?
If you have other ideas on what is missing from current firmware, throw your thoughts.
+1 for OpenWakeWord as option as a temporary solution for now to enable easy to use custom wake word models.
As it right now is too hard to make new custom Micro Wake Word models it would if nothing else it serve as as showcase for the demand of custom wake words, which might also be an incentive for more developers step up to join the effort helping to make custom Micro Wake Word models easier make in the future too.