Device, not entity.
0 sleeps, 4 hours to go…
3 Likes
Year of the Voice Chapter 4 was very impressive. Going to need to play with my Atom M5 now and create my own wake word. 
I’m a bit confused with this, using the atom echo we need to plug it directly to our home assistant machine right?
Is there any wireless solution?
Wrong. It’s wifi. You just need to feed it 5V.
1 Like
The initial setup of the whole chain (whisper, piper, openwakeword, atom echo) worked flawless in one go for me. I did get it to work, but sometimes something in the chain stops working and from there on nothing reacts anymore until I unplug the echo and plug it back in. So far the logs are not helping…
Example scenario:
- echo plugged in, LED purple
- i use the wakeword, LED turns to blue
- i speak a command
- command gets executed and voice confirmation comes
- LED goes back to purple
- i try the wakeword again
- LED blue
- I give the command
- it either doesnt stop listening (blue led stays on for a while) or doesnt execute the command
- back to violet LED
- wake word detection doesnt work until I restart the Echo
- start from the top
Anyone with a similar experience?
some logs:
Logger: homeassistant
Source: components/esphome/voice_assistant.py:316
First occurred: 10:59:40 (3 occurrences)
Last logged: 11:47:35
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 1190, in sendto
self._sock.sendto(data, addr)
^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'sendto'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/esphome/voice_assistant.py", line 316, in _send_tts
self.transport.sendto(chunk, self.remote_addr)
File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 1200, in sendto
self._fatal_error(
File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 867, in _fatal_error
self._loop.call_exception_handler({
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
Logger: homeassistant.components.esphome.manager
Source: components/esphome/manager.py:330
Integration: ESPHome (documentation, issues)
First occurred: 10:59:40 (3 occurrences)
Last logged: 11:47:34
Voice assistant UDP server was not stopped
Logger: homeassistant.components.esphome.voice_assistant
Source: components/esphome/voice_assistant.py:164
Integration: ESPHome (documentation, issues)
First occurred: 10:59:18 (18 occurrences)
Last logged: 11:47:30
Received unknown pipeline event type: stt-vad-start
Received unknown pipeline event type: stt-vad-end
Logger: homeassistant.components.wyoming.wake_word
Source: components/wyoming/wake_word.py:123
Integration: Wyoming Protocol (documentation, issues)
First occurred: 10:59:16 (2 occurrences)
Last logged: 10:59:33
Expected wake word ok_nabu_v0.1 but got ok_nabu, skipping
I thought I would try the wake word on a ESP32 Rhasspy Satellite. See https://github.com/ProrokWielki/HomeAssistantSpeaker. I am using ESPHome 2023.10b1. I got it all working except the speaker because the speaker is using the ESP32 second I2S channel on different lrclk_pin and bclk_pin pins. I could not determine how to set another set of i2s_audio: pins. I suppose I could cut and jumper my PWB 
. I am still working on getting an effect for the WS2812 led strip that I like.
UPDATE: I figured it out. I now have a working speaker ! 
Just to be clear I am running Home Assistant Supervised.
10/17/23: The assistant was not reliable on ESPHome 2023.10b2. In fact it was very flakey. I updated to 2023.10b3 today, re-started openWakeWord and the device seems to be pretty solid. I also tried a M5Stack Atom Echo and it is very unreliable on both ESPHome b2 and b3.
The updated yaml code I am using is:
---
esphome:
name: esp32-voice-assistant
friendly_name: esp32 voice assistant
name_add_mac_suffix: true
project:
name: m5stack.atom-echo-voice-assistant
version: "1.0"
min_version: 2023.10.0b1
on_boot:
- priority: -100
then:
- wait_until: api.connected
- delay: 1s
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:
esp32:
board: esp32dev
framework:
type: esp-idf
logger:
api:
ota:
dashboard_import:
package_import_url: github://esphome/firmware/voice-assistant/m5stack-atom-echo.yaml@main
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
manual_ip:
static_ip: 192.168.1.31
gateway: 192.168.1.1
subnet: 255.255.255.0
improv_serial:
i2s_audio:
- id: mic
i2s_lrclk_pin: GPIO32
i2s_bclk_pin: GPIO25
- id: spkr
i2s_lrclk_pin: GPIO12
i2s_bclk_pin: GPIO14
microphone:
- platform: i2s_audio
i2s_audio_id: mic
id: INMP441_microphone
i2s_din_pin: GPIO33
adc_type: external
pdm: fALSE
speaker:
- platform: i2s_audio
i2s_audio_id: spkr
id: max98357_speaker
i2s_dout_pin: GPIO27
dac_type: external
mode: mono
voice_assistant:
id: va
microphone: INMP441_microphone
speaker: max98357_speaker
noise_suppression_level: 2
auto_gain: 31dBFS
volume_multiplier: 2.0
vad_threshold: 3
on_listening:
- light.turn_on:
id: led
blue: 100%
red: 0%
green: 0%
brightness: 25%
effect: scan
on_tts_start:
- light.turn_on:
id: led
blue: 0%
red: 0%
green: 100%
brightness: 25%
effect: scan
on_end:
- delay: 100ms
- wait_until:
not:
speaker.is_playing:
- script.execute: reset_led
on_error:
- light.turn_on:
id: led
blue: 0%
red: 100%
green: 0%
brightness: 100%
effect: scan
- delay: 1s
- script.execute: reset_led
- script.wait: reset_led
- lambda: |-
if (code == "wake-provider-missing" || code == "wake-engine-missing") {
id(use_wake_word).turn_off();
}
binary_sensor:
- platform: gpio
pin:
number: GPIO39
inverted: true
name: Button
disabled_by_default: true
entity_category: diagnostic
id: echo_button
on_click:
- if:
condition:
switch.is_off: use_wake_word
then:
- if:
condition: voice_assistant.is_running
then:
- voice_assistant.stop:
- script.execute: reset_led
else:
- voice_assistant.start:
else:
- voice_assistant.stop
- delay: 1s
- script.execute: reset_led
- script.wait: reset_led
- voice_assistant.start_continuous:
light:
- platform: esp32_rmt_led_strip
id: led
name: None
disabled_by_default: true
entity_category: config
pin: GPIO26
default_transition_length: 0s
chipset: WS2812
num_leds: 13
rgb_order: grb
rmt_channel: 0
effects:
- pulse:
transition_length: 250ms
update_interval: 250ms
- addressable_scan:
- addressable_scan:
name: scan
move_interval: 50ms
scan_width: 1
script:
- id: reset_led
then:
- if:
condition:
switch.is_on: use_wake_word
then:
- light.turn_on:
id: led
blue: 100%
red: 100%
green: 0%
brightness: 15%
effect: none
else:
- light.turn_off: led
switch:
- platform: template
name: Use wake word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- lambda: id(va).set_use_wake_word(true);
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
- script.execute: reset_led
on_turn_off:
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
- script.execute: reset_led
external_components:
- source: github://pr#5230
components:
- esp_adf
refresh: 0s
esp_adf:
If anyone is looking to get this working, here is what I got working with my M5 esp unit.
A few caveats, I run HA with a docker stack on an x86 based machine so all my “addons” are actually docker stacks.
Here is the docker stack I have used to get wake word working…
# 2023-10-13
version: "3"
services:
## whisper must be installed in HA (settings...integrations...whisper)
whisper: # stt
container_name: whisper
command: --model small --language en
image: rhasspy/wyoming-whisper:latest
volumes:
- /home/pcwii/docker/whisper:/data
environment:
- TZ=America/Toronto
restart: unless-stopped
ports:
- 10301:10300
## piper must be installed in HA (settings...integrations...piper)
piper: # tts
container_name: piper
image: rhasspy/wyoming-piper:latest
command: --voice en-us-libritts-high
volumes:
- /home/pcwii/docker/piper:/data
environment:
- TZ=America/Toronto
restart: unless-stopped
ports:
- 10201:10200
## openwakeword must be installed in HA (settings...integrations...openwakeword)
openwakeword:
container_name: openwakeword
image: rhasspy/wyoming-openwakeword
command:
--preload-model 'hey_kelsey' # this model will be loaded as a default
--custom-model-dir /custom # this is the directory in the container that contains the custom models
--model 'alexa'
--model 'hey_jarvis'
--model 'hey_mycroft'
--model 'alexa'
--model 'hey_kelsey'
--uri 'tcp://0.0.0.0:10400'
volumes:
- /home/pcwii/docker/openwakeword-data:/data
- /home/pcwii/docker/openwakeword-data:/custom # mapping local custom model directory to the custom model directory in the container
environment:
- TZ=America/Toronto
restart: unless-stopped
ports:
- 10400:10400
Next.
I had to update esphome to the beta channel, here is the stack for that…
# 2023-10-13
version: '3'
services:
esphome:
container_name: esphome
#image: ghcr.io/esphome/esphome
image: ghcr.io/esphome/esphome:beta
volumes:
- /home/pcwii/docker/esphome/config:/config
- /etc/localtime:/etc/localtime:ro
restart: unless-stopped
privileged: true
network_mode: host
If you use it and have success let me know!
1 Like
Trying to understand what all I need. I already had whisper and piper installed via Wyoming Protocol. Now I added the openwakeword via Wyoming Protocol as well, chose in the Assistant settings at the bottom of that panel. Do I also need this? GitHub - dscripka/openWakeWord: An open-source audio wake word (or phrase) detection framework with a focus on performance and simplicity.
I plugged in USB adapter, the Jabra Link 370 for a Jabra 710 bluetooth speakerphone. I don’t know if or where I need to choose it from the settings.
I believe that you have to have the openwakeword Addon installed.
I have the docker container version, so what equivalent docker container is that, The regular openwakeword or the Wyoming protocol one?
Sorry. I don’t know. I am running the Supervised version of Home Assistant.
I have a similar experience. Initially I was able to get it to work (although most of the time it said I couldn’t understand that) however now it no longer responds. I don’t have any errors in the logs so not sure how to debug it. When I say the wake-word of “ok Nabu” it just stays the purple color and maybe on occasion I see it turn red but that seems to be rare. Were you able to get it to work consistently?
Nope, not yet :-/
Someone in the thread in the blog section described a similar behaviour
What is your overall setup? (Assist chain, underlying hardware)
Using HA OS with the openwake word add-on. I upgraded to esphome 10.2 beta and not sure if that had any impact but when I originally installed it it was with 10.1 beta. This morning I installed the update for the openwake word addon but I had this issue prior to that. Was hoping that might resolve it. I tried restarting HA also the M5 Stack but nothing seems to work. Currently on HA 2023.10.1 and have not updated yet to the new 11.0 OS
Yes, I’ve seen exactly that behavior, except for the last message. In one case, it seemed related to it choosing an unavailable entity, but I’ve seen it with an otherwise correct command as well:
2023-10-15 20:31:19.864 INFO (MainThread) [homeassistant.helpers.intent] Triggering intent handler <OnOffIntentHandler - HassTurnOff>
2023-10-15 20:31:19.879 WARNING (MainThread) [homeassistant.helpers.service] Referenced entities light.fr_scene are missing or not currently available
2023-10-15 20:31:20.279 WARNING (MainThread) [homeassistant.components.esphome.manager] Voice assistant UDP server was not stopped
2023-10-15 20:31:20.306 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 1190, in sendto
self._sock.sendto(data, addr)
^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'sendto'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/esphome/voice_assistant.py", line 316, in _send_tts
self.transport.sendto(chunk, self.remote_addr)
File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 1200, in sendto
self._fatal_error(
File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 867, in _fatal_error
self._loop.call_exception_handler({
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
The end result is that the Atom Echo stays purple and no longer responds to any commands until I unplug it and plug it back in. I just started playing with it and will file an issue if I can figure out how to reproduce it. It usually happens on the second or third command, when the preceeding commands work fine.
I’m running wyoming-openwakeword v1.7.0 in a manual container, and there are no logs from that other than this:
INFO:root:Ready
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
1 Like
What does your network/wifi setup look like?
I stumbled over Feature request: Audio input over UDP (from PC or Raspberry Pi) · Issue #3141 · Aircoookie/WLED · GitHub
"The downside of UDP sound sync - which is based on the multicast protocol - is that it depends on the capabilities of your Wifi router. Many “consumer grade” routers cannot support multicast reliably, or will even block the protocol after a few minutes."
I’ve got the default router from my ISP, but WIFI is handled by a dedicated access point in the middle of the apartment, with a cable connection from the AP to the router. HA is running on a Raspberry which also has a cable connection to the router.
I guess I’ll dig a bit into my access point model…
edit: okay, i THINK i now know what the problem is 
One of the more advanced models of my access point has the following setting option:
Enable UDP-FLOOD Filtering – Enable or Disable the UDP-FLOOD Filtering.
UDP-FLOOD Packets Threshold (5~3600) – The default value is 500. Enter a value between 5 ~ 3600. When the current UPD-FLOOD Packets number is beyond the set value, the Router will startup the blocking function immediately.
Read more: https://manuals.plus/tp-link/tl-wr840n-300mbps-wireless-n-router-manual#ixzz8GHpylq9T
I went through my AP settings, and my model doesnt have a dedicated setting for it. So I guess some kind of default filtering kicks in. Will try to reproduce it by having the atom running and speaking into it for a bit, WITHOUT using the wake word. If my theory is right, this constant speaking will still create the UDP packages to HA, and if the AP cuts them off after a bit, even the first trying out of the wake world should not reach HA.
how are you going with the openWakeWord docker version? I’m looking at getting this up and running (my HA server is already in a docker so this is kind of my only path).
I installed the wyoming variant, still need to get a proper mic plugged in via USB to see if it works. I did get it connected to HA though.
ok nice! why did you chose that varient of the wakeword (just out of interest?). Let me know how you go with it! Keen to hear the results