Hi there,
just to encourage people to use snips with hassio: it’s working like a charme for me! As I am from germany I did my own installer and can switch lights in the house (I#m just a beginner and gradually including things in hassio and snips). With a little help from tschmidty I’m even able to call different services with regard to a deviceLocation slot.
Snips and hass are great!
I have managed to get snips with home assistant up and running, but unfortunately so far my experience hasn’t been as good, and so e.g. a physical light switch is still much more reliable and pleasant to use. Although I am still excited to see it improve and hope to see if I can contribute further training examples.
Most of the struggles at the moment seem to be with the ASR (automatic speech recognition) in that snips doesn’t transcribe the spoken language well into text for me. I initially used the “custom language” model optimized for the training examples of the intents of the assistant. But that seems to have had way to many “unknown word” in the transcribed text. I am now using the “generic english” language model. It seems to perform a lot better than the custom one but still makes a fairly large number of errors, which then makes it impossible to extract the intent. Furthermore, the the generic ASR uses a lot more memory and takes up about 60% of the available 1GB of memory on the Raspberry Pi, which can cause issues if a lot of other things are running on the same Raspberry Pi.
I haven’t actually listened to the raw audio of what the microphone picks up to see if the text is clearly understandable to a human, so it is possible that the “fault” lies with the microphone (Sony Eye PS3)
I do not want to go without physical switches, I want both 
ASR is doing well for me, I do not use the generic because it is way to big.
Here a video of the Matrix Voice, Snips & Home Assistant working together.
Install was a clean Rasbian Stretch, the Kernel Modules from Matrix Voice and Snips.
The pi is brigded to my NAS, where HA is running.
I created a service myself which listens to the hotword toggle topic to control the leds.
So I jumped in and installed the Snips Add-On in HASS.IO. It’s all running on the same RPi3 with a USB Mic and audio out on the 3.5mm.
I’m not sure what I’m doing wrong, but I can’t seem to get Snips to recognize the wake word. I haven’t trained anything, so I’m just using the default assistant.
Part of my confusion is the Hass.io Snips.ai add-on UI is different than what is mentioned in the documentation.
There is an “Audio” section that seems to auto detect the devices.
There is also a “Network” section, but I’m not sure what that is for.
I have Mosquitto MQTT add-on setup, and configured as the broker in the config yaml.
The Snips logs look ok, best I can tell. But I never see the wake word triggered.
Any suggestions?
@tschmidty How to install your custom version?
I tried with adding https://github.com/tschmidty69/hassio-snips to the custom repo section in hassio but this don’t seem to work?
That should work and show a snips repo at the bottom of the list. But at this point mine is synched with the official addon. If you need help with it try the Snips discord, there is a home assistant channel there I try to help on.
Does this mean the the offical Snips Add-On in Hassio (v. 3.0) works with newest snips version? I got it working with a second raspberry with snips installed but not with “all-inclusive” (Snips + Hassio on same raspberry).
Yeah, it should work fine. Did you install the mosquitto add on also?
Yes. With this config:
Mosquitto Add-On:
{
“plain”: true,
“ssl”: false,
“anonymous”: false,
“logins”: [
{
“username”: “xxxxxxx”,
“password”: “xxxxxxx”
}
],
“customize”: {
“active”: false,
“folder”: “mosquitto”
},
“certfile”: “fullchain.pem”,
“keyfile”: “privkey.pem”
}
Snips Add-On:
{
“mqtt_bridge”: {
“active”: true,
“host”: “core-mosquitto”,
“port”: 1883,
“user”: “xxxxxx”,
“password”: “xxxxxx”
},
“assistant”: “assistant.zip”,
“language”: “en”,
“custom_tts”: {
“active”: false,
“platform”: “amazon_polly”
}
}
Hassio Config / Configuration.yaml:
snips:
mqtt:
broker: core-mosquitto
username: !secret mosq_user
password: !secret mosq_password
I do. However, only the audio server is running on the pi. PS3 Eye as mic. A beefy server does the ASR. Hass doesn’t parse the intents, I do that via custom code in node by listening to the hotword and intent parsed messages on mqtt. Switches/Lights/Action, etc are triggered on hass via the websocket.
I don’t use the available skills, I rolled my own assistant.
Works well enough, not alexa-good but good enough, definitely adds value.
I got everything working, intents detected and fired, but i’m wondering if it possible to parse the text and call the intent by telegram?
@radinsky As its working for you - can you take a look at my config 2 posts above and tell me what you’ve different?
My config looks the same. I suggest you posting your logs to see what is going wrong…
This is my log. My question: Is the line “model loaded in xxx ms” the last line or should there be some more? (Like “Hotword detected”)
INFO:snips_hotword_lib::features: Audio thread for default started
1531427565: New connection from 127.0.0.1 on port 1883.
1531427565: New client connected from 127.0.0.1 as 84093b2172d946b490fab81549f1e9d4 (c0, k10).
INFO:snips_asr: Connecting using MQTT site-id default
INFO:snips_asr_hermes: Using model from "/usr/share/snips/assistant/custom_asr"
INFO:snips_hotword_lib : Detector for default on default thread started
1531427565: New connection from 127.0.0.1 on port 1883.
1531427565: New client connected from 127.0.0.1 as 9e708d906eb849efb6347d751d21b30b (c0, k10).
[22:32:45] Watching on localhost:1883 (MQTT)
DEBUG:snips_common_cli::cli: conf lookup: snips-asr-injection.model -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-common.model -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-asr-injection.assistant -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-common.assistant -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-asr-injection.user_dir -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-common.user_dir -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-asr-injection.bus -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-common.bus -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-asr-injection.mqtt -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-asr-injection.mqtt_username -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt_username -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-asr-injection.mqtt_password -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt_password -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-asr-injection.mqtt_tls_hostname -> Ok(None)
DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt_tls_hostname -> Ok(None)
1531427565: New connection from 127.0.0.1 on port 1883.
1531427565: New client connected from 127.0.0.1 as 7619ae3f343c4ff6a9d7878bb2b2e676 (c0, k10).
INFO:snips_asr_injection::commands::mqtt: Launch with default model: "/usr/share/snips/assistant/custom_asr" and user dir: "/var/lib/snips"
INFO:snips_asr_hermes: Preparing decoder
2018-07-12 22:32:46,026 INFO success: snips-tts entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-07-12 22:32:46,027 INFO success: snips-dialogue entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-07-12 22:32:46,027 INFO success: snips-asr entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-07-12 22:32:46,028 INFO success: snips-nlu entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-07-12 22:32:46,028 INFO success: snips-analytics entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-07-12 22:32:46,029 INFO success: snips-hotword entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-07-12 22:32:46,029 INFO success: snips-audio-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
INFO:snips_asr_hermes: Idle
INFO:snips_nlu_hermes: model loaded in 1687 msI have tried now for 7 hours to get this working with no success.
I installed the Snips.AI addon, set the mqtt_bridge to the values of my existing (and working) Mosquitto broker and set the AudioInput to USB mic.
My Snip.AI settings:
{
"mqtt_bridge": {
"active": true,
"host": "core-mosquitto",
"port": 1883,
"user": "myMQTTuser",
"password": "myMQTTpassword"
},
"assistant": "SnipAssistant.zip",
"language": "de",
"custom_tts": {
"active": false,
"platform": "amazon_polly"
}
}
My Mosqitto settings:
{
"plain": true,
"plain_websockets": false,
"ssl": false,
"ssl_websockets": false,
"anonymous": true,
"logins": [],
"customize": {
"active": false,
"folder": "mosquitto"
},
"certfile": "fullchain.pem",
"keyfile": "privkey.pem"
}
Snips.AI is broking on 9898.
Then I went to snips.ai, created an account, created a new assistant with german language, added the Apps:
Radio - by KINL99
HomeAssistant by HASS
Lichtstuerung by JKBECKER
Wikipedia by CRYSTALMETHOD (now I know that this wont work with HASS)
Wetter by SNIPS
and Musik by TEST123456
Then I deployed the Assistant and downloaded the .zip file.
I created the folder /share on my raspberry Pi 3 and copied the .zip file into it and restarted Snips (with the correct .zip - name in the Snips Addon Config).
When I say “Hey Snips - turn the light on” I can hear the sound that it is listen to me and see that it recocnises my command
INFO:snips_asr_hermes : Cleanup
INFO:snips_asr_hermes : Preparing decoder
[05:06:53] [Asr] was asked to stop listening on site default
[05:06:53] [AudioServer] was asked to play a wav of 93.1 kB with id '374e6f28-aad5-4351-bb76-ff5ca84ec122' on site default
INFO:snips_audio_portaudio: Playing "374e6f28-aad5-4351-bb76-ff5ca84ec122" using output "default", wav spec: WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
INFO:snips_asr_hermes : Idle
INFO:snips_audio_portaudio: Playing of "374e6f28-aad5-4351-bb76-ff5ca84ec122" finished
[05:06:54] [AudioServer] finished playing wav with id '374e6f28-aad5-4351-bb76-ff5ca84ec122'
WARN:snips_tts_hermes: Did received a "play finished" message from audio server with an unknown id '374e6f28-aad5-4351-bb76-ff5ca84ec122'. It could be normal if you have more than one audio server.
[05:06:54] [Nlu] was asked to parse input schalte das licht ein
[05:06:54] [Nlu] detected intent domi:LampenAnSchalten with probability 1.000 for input "schalte das licht ein"
[05:06:54] [Dialogue] New intent detected domi:LampenAnSchalten with probability 1.000
INFO:snips_dialogue_lib::coordinator::coordinator: Session [2619eb32-9b46-4b56-94f9-359d901e15c5]: closing session which was active: Timeout
INFO:snips_dialogue_lib::coordinator::coordinator: Site [default]: reset site -> turn off ASR and turn on Hotword
[05:06:59] [Dialogue] session with id '2619eb32-9b46-4b56-94f9-359d901e15c5' was ended on site default. The session was ended because one of the component didn't respond in a timely manner
[05:06:59] [Asr] was asked to stop listening on site default
[05:06:59] [Hotword] was asked to toggle itself 'on' on site default
INFO:snips_hotword_lib::features: Audio thread for site default running
So it gets the intent ’ domi:LampenAnSchalten’.
I added this to my configuration.yaml:
snips:
intent_script:
TestConversation:
speech:
text: Alles Super!
LampenAnSchalten:
action:
- service: light.turn_on
entity_id: light.devicesEntity_id
conversation:
intents:
TestConversation:
- Alles gut?
I also tried with “domi:LampenAnSchalten” but as I read here: https://www.home-assistant.io/components/snips/
HASS internaly cuts everything before the ‘:’ off…
(tested both anyway… )
I also tried directly to intent it on snip:
snips:
intents:
LampenAnSchalten:
action:
- service: light.turn_on
entity_id: light.devicesEntity_id
But nor the “conversation” neither the lights work.
I hope you can tell my what I did wrong 
I would try to remove the TestConversation: part.
Maybe the text should be between quotes.
I removed the conversation part but its not working either (I intentionally added this to debug if it understands anything).
How exactly to I have to “catch” the intention?
For example
[Dialogue] New intent detected domi:LampenAnSchalten....
does nothing.
Nor with
snips:
intent_script:
LampenAnSchalten:
action:
- service: switch.turn_on
entity_id: light.devicesEntity_id
Neither with
snips:
intent_script:
domi:LampenAnSchalten:
action:
- service: switch.turn_on
entity_id: light.devicesEntity_id
Do I need to add more than this?
EDIT:
I deployed a new Assistant on snips.ai.
This time ONLY with the HomeAssistant “app” by HASS.
Its working now.
So the light app doesnt work with hassio?
strange…
You have also remove intent_scipt, that is incorrect.
Try if it works like this:
snips:
intent_script:
LampenAnSchalten:
action:
- service: light.turn_on
entity_id: light.devicesEntity_id
The snips: is just for activating the component.
The intent_script is actually what does the work. This section can be used by other speech components as well
By the way, what is : light.devicesEntity_id?
Do you have a light entity named “devicesEntity_id”?
If not, changed that to the actual entity id
The HomeAssistant App uses intents which are internal available in HomeAssistant.
Thanks Romkabouter… looks like I made a mistake.
(I corrected the above post to the one I had)
I had:
snips:
intent_script:
LampenAnSchalten:
action:
- service: switch.turn_on
entity_id: light.devicesEntity_id
Good that you ask about the entity ID because everything has to be completly clear.
The entity is the one I found at the dev-tools->states at searching for the device at the entity dropdown.
I just replaced it here for security reasons to “devicesEntity_id”
Maybe there is another place where I can find it (like a overview list), but its the only one I found.
So right now I have just the hassio addon Snips.ai installed (NO Assistant deployed at the snips.ai) and the light turn on/off function does work (yay!)
Also a speech response does work (yayyay!)
The light is congurated like this (my real ID is replaced here with THISID):
light:
- name: ESPURNA-THISID
platform: mqtt
state_topic: ESPURNA-THISID/relay/0
command_topic: ESPURNA-THISID/relay/0/set
payload_on: 1
payload_off: 0
availability_topic: ESPURNA-THISID/status
payload_available: 1
payload_not_available: 0
brightness_state_topic: ESPURNA-THISID/brightness
brightness_command_topic: ESPURNA-THISID/brightness/set
rgb_state_topic: ESPURNA-THISID/rgb
rgb_command_topic: ESPURNA-THISID/rgb/set
color_temp_command_topic: ESPURNA-THISID/mired/set
What does NOT work is change the light color and/or brightness.
My config as it is right now (replaced the device entity id):
snips:
intent_script:
HassTurnOff:
action:
- service: light.turn_off
entity_id: light.devicesEntity_id
speech:
type: plain
text: 'OK, human being'
HassTurnOn:
action:
- service: light.turn_on
entity_id: light.devicesEntity_id
HassLightSet:
action:
- service: light.turn_off
entity_id: light.devicesEntity_id
HassLightSet is catched at talking to snips but it has no function
I get this at the snips log:
detected intent hass:HassLightSet with probability 0.966 for input "stelle das licht auf rot und vierzig prozent ein"
Slots ->
light -> licht
color -> rot
brightness -> 40.00%
I just dont know how to setup these “slots” correctly.
I tried:
HassLightSet:
action:
- service: light.turn_off
entity_id: light.devicesEntity_id
color_name: {{ color }}
brightness: {{ brightness }}
I found this a the light component:
but its not working.
I might have the wrong component befause it drops a error at the developer info log that HassLightSet dont know “color_name”.
looking at the component “snips” give me this:
Which should activate “ActivateLightColor” but at the snips log it never did.
It always just activates the HassLightSet.
So the documentation of this component is wrong, incomplete or maybe for “home assistant” and not for “hassio” - pretty confusing for me.
Im not bad at coding but finding the information I need is almost impossible.
I also found this one:
But it just tells me what intents are build in and not how to use them correctly…
Is there any other thing I need for my LED bulb which uses the ESPurna firmware?
Thanks a lot!
Leon333, did you ever get this working. I am having similar frustration. I would appreciate knowing which snips intents you chose and your final configuration.yaml entries.
Thanks!
Greg