Who has got snips working on Hassio? I got it to recognize the hot word, but it doesn’t pick up any of the words I say after that. If you got it working could you please share your set-up? Don’t know what I’m doing wrong.
Thanks!
I too am wondering about this. I’ve tried running snips as a Hassio addon and on a second Pi running raspbian. In both cases, I can see the messages being sent on the broker, so I know that Snips is picking up the hotword on both systems, but Hassio doesn’t seem to be doing anything with it. I’d really like to see an example config of someone who has Snips working on Hassio.
Can you also see the words you say after the hotword in the broker? Because I can’t, but they should be there according to this site `
I can see the text on the MQTT bus but not in the snips addon log. It processes that I said something, but it doesn’t show what, even though the software parses it correctly.
I’m not using my own MQTT broker but the build in one of snips. However I always get this response after I use the hotword:
[10:17:51.590102] INFO :snips_hotword : Hotword detected
[10:17:51.607924] INFO :snips_dialogue::dialogue: State: Idle, incoming Message: Hotword(Detected)
[10:17:51.608068] INFO :snips_dialogue::services: publish Hotword(Wait)
[10:17:51.608151] INFO :snips_dialogue::services: publish ASR(ToggleOn)
[10:17:51.608220] INFO :snips_dialogue::services: publish AudioServer(PlayFile)
[10:17:51.608279] INFO :snips_dialogue::dialogue: Current State: WaitingQuery
[10:17:51.657784] INFO :snips_asr : Listening
[10:17:51.662036] INFO :snips_audio_server : using output "default" for "/usr/share/snips/dialogue/sound/start_of_input.wav", WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
So it detects the hotword, starts listening and uses “default” as output. So I assume it is not picking up any thing of what I say.
Is it the same with you?
Here’s the output I get from the Snips log in HASSIO. I said “Hey Snips, turn on the light” 3 times and waited for the responses
[22:01:33.583504] ERROR:snips_analytics : Can't parse assistant.json: missing field `name` at line 261 column 1
1504148493: Socket error on client mqttc_1876782751, disconnecting.
2017-08-30 22:01:33,589 INFO exited: snips-analytics (exit status 1; not expected)
[22:01:33.618290] INFO :hermes::mqtt: Connecting to MQTT broker at address localhost:1883
1504148493: New connection from 127.0.0.1 on port 1883.
1504148493: New client connected from 127.0.0.1 as mqttc_2191896867 (c1, k5).
[22:01:33.628574] INFO :rumqtt::connection: $$$ Connected to broker
Can't parse assistant.json: missing field `name` at line 261 column 1
1504148493: Socket error on client mqttc_2191896867, disconnecting.
2017-08-30 22:01:33,685 INFO exited: snips-queries (exit status 1; not expected)
2017-08-30 22:01:36,696 INFO spawned: 'snips-analytics' with pid 122
2017-08-30 22:01:36,709 INFO spawned: 'snips-queries' with pid 123
[22:01:36.746004] INFO :hermes::mqtt: Connecting to MQTT broker at address localhost:1883
1504148496: New connection from 127.0.0.1 on port 1883.
1504148496: New client connected from 127.0.0.1 as mqttc_3372216042 (c1, k5).
[22:01:36.757393] INFO :rumqtt::connection: $$$ Connected to broker
[22:01:36.792176] ERROR:snips_analytics : Can't parse assistant.json: missing field `name` at line 261 column 1
1504148496: New connection from 127.0.0.1 on port 1883.
1504148496: New client connected from 127.0.0.1 as mqttc_1451175279 (c1, k5).
[22:01:36.794270] INFO :hermes::mqtt: Connecting to MQTT broker at address localhost:1883
[22:01:36.802012] INFO :rumqtt::connection: $$$ Connected to broker
1504148496: Socket error on client mqttc_3372216042, disconnecting.
2017-08-30 22:01:36,816 INFO exited: snips-analytics (exit status 1; not expected)
Can't parse assistant.json: missing field `name` at line 261 column 1
2017-08-30 22:01:36,832 INFO gave up: snips-analytics entered FATAL state, too many start retries too quickly
1504148496: Socket error on client mqttc_1451175279, disconnecting.
2017-08-30 22:01:36,841 INFO exited: snips-queries (exit status 1; not expected)
2017-08-30 22:01:37,846 INFO gave up: snips-queries entered FATAL state, too many start retries too quickly
[22:01:48.215458] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:01:48.215632] INFO :snips_dialogue::services: publish Hotword(ToggleOn)
[22:02:08.215555] INFO :snips_dialogue::dialogue: Timeout is reached. Going to Idle mode
[22:02:08.215710] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:02:08.215787] INFO :snips_dialogue::services: publish Hotword(ToggleOn)
[22:02:28.215655] INFO :snips_dialogue::dialogue: Timeout is reached. Going to Idle mode
[22:02:28.215852] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:02:28.215927] INFO :snips_dialogue::services: publish Hotword(ToggleOn)
[22:02:48.215760] INFO :snips_dialogue::dialogue: Timeout is reached. Going to Idle mode
[22:02:48.215911] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:02:48.215988] INFO :snips_dialogue::services: publish Hotword(ToggleOn)
[22:02:48.446383] INFO :snips_hotword_hermes: Hotword detected
[22:02:48.455475] INFO :snips_dialogue::dialogue: State: Idle, incoming Message: Hotword(Detected)
[22:02:48.455629] INFO :snips_dialogue::services: publish Hotword(Wait)
[22:02:48.455699] INFO :snips_dialogue::services: publish Asr(ToggleOn)
[22:02:48.455820] INFO :snips_dialogue::services: publish AudioServer(PlayFile)
[22:02:48.455884] INFO :snips_dialogue::dialogue: Current State: WaitingQuery
[22:02:48.504915] INFO :snips_asr_hermes : Listening
[22:02:48.510538] INFO :audio_server_hermes : Playing "/usr/share/snips/dialogue/sound/start_of_input.wav" using output "default", wav spec : WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
[22:02:50.845909] INFO :snips_asr_lib::asr: Source thread stop on push: "Full(..)"
[22:03:07.162553] INFO :snips_asr_hermes : Cleanup
[22:03:07.163893] INFO :snips_asr_hermes : Idle
[22:03:07.202616] INFO :snips_dialogue::dialogue: State: WaitingQuery, incoming Message: Asr(TextCaptured)
[22:03:07.202707] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:03:07.202751] INFO :snips_dialogue::services: publish AudioServer(PlayFile)
[22:03:07.216578] INFO :snips_dialogue::services: publish Nlu(Query)
[22:03:07.216741] INFO :snips_dialogue::dialogue: Current State: WaitingIntent
[22:03:07.297228] INFO :audio_server_hermes : Playing "/usr/share/snips/dialogue/sound/end_of_input.wav" using output "default", wav spec : WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
[22:03:28.215957] INFO :snips_dialogue::dialogue: Timeout is reached. Going to Idle mode
[22:03:28.216103] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:03:28.216182] INFO :snips_dialogue::services: publish Hotword(ToggleOn)
[22:03:44.298517] INFO :snips_hotword_hermes: Hotword detected
[22:03:44.323979] INFO :snips_dialogue::dialogue: State: Idle, incoming Message: Hotword(Detected)
[22:03:44.324126] INFO :snips_dialogue::services: publish Hotword(Wait)
[22:03:44.324199] INFO :snips_dialogue::services: publish Asr(ToggleOn)
[22:03:44.324264] INFO :snips_dialogue::services: publish AudioServer(PlayFile)
[22:03:44.324324] INFO :snips_dialogue::dialogue: Current State: WaitingQuery
[22:03:44.414304] INFO :snips_asr_hermes : Listening
[22:03:44.418793] INFO :audio_server_hermes : Playing "/usr/share/snips/dialogue/sound/start_of_input.wav" using output "default", wav spec : WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
[22:03:47.597948] INFO :snips_asr_lib::asr: Source thread stop on push: "Full(..)"
[22:03:49.196659] INFO :snips_asr_lib::asr: Endpoint detection.
[22:03:49.494249] INFO :snips_asr_hermes : Cleanup
[22:03:49.494590] INFO :snips_asr_hermes : Idle
[22:03:49.514468] INFO :snips_dialogue::dialogue: State: WaitingQuery, incoming Message: Asr(TextCaptured)
[22:03:49.514691] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:03:49.514795] INFO :snips_dialogue::services: publish AudioServer(PlayFile)
[22:03:49.517614] INFO :snips_dialogue::services: publish Nlu(Query)
[22:03:49.517781] INFO :snips_dialogue::dialogue: Current State: WaitingIntent
[22:03:49.603312] INFO :audio_server_hermes : Playing "/usr/share/snips/dialogue/sound/end_of_input.wav" using output "default", wav spec : WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
[22:04:28.216334] INFO :snips_dialogue::dialogue: Timeout is reached. Going to Idle mode
[22:04:28.216483] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:04:28.216565] INFO :snips_dialogue::services: publish Hotword(ToggleOn)
[22:04:34.465507] INFO :snips_hotword_hermes: Hotword detected
[22:04:34.474390] INFO :snips_dialogue::dialogue: State: Idle, incoming Message: Hotword(Detected)
[22:04:34.474535] INFO :snips_dialogue::services: publish Hotword(Wait)
[22:04:34.474618] INFO :snips_dialogue::services: publish Asr(ToggleOn)
[22:04:34.474686] INFO :snips_dialogue::services: publish AudioServer(PlayFile)
[22:04:34.474745] INFO :snips_dialogue::dialogue: Current State: WaitingQuery
[22:04:34.564373] INFO :snips_asr_hermes : Listening
[22:04:34.565481] INFO :audio_server_hermes : Playing "/usr/share/snips/dialogue/sound/start_of_input.wav" using output "default", wav spec : WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
[22:04:38.016351] INFO :snips_asr_lib::asr: Endpoint detection.
[22:04:38.138296] INFO :snips_asr_hermes : Cleanup
[22:04:38.138392] INFO :snips_asr_hermes : Idle
[22:04:38.172642] INFO :snips_dialogue::dialogue: State: WaitingQuery, incoming Message: Asr(TextCaptured)
[22:04:38.172738] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:04:38.172787] INFO :snips_dialogue::services: publish AudioServer(PlayFile)
[22:04:38.173522] INFO :snips_dialogue::services: publish Nlu(Query)
[22:04:38.173620] INFO :snips_dialogue::dialogue: Current State: WaitingIntent
[22:04:38.224004] INFO :audio_server_hermes : Playing "/usr/share/snips/dialogue/sound/end_of_input.wav" using output "default", wav spec : WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
[22:05:08.216537] INFO :snips_dialogue::dialogue: Timeout is reached. Going to Idle mode
[22:05:08.216684] INFO :snips_dialogue::services: publish Asr(ToggleOff)
[22:05:08.216763] INFO :snips_dialogue::services: publish Hotword(ToggleOn)
Here’s what gets published on the MQTT bus.
{"filePath":"/usr/share/snips/dialogue/sound/start_of_input.wav"}
{"id":"/usr/share/snips/dialogue/sound/start_of_input.wav"}
{"text":"please turn on the light","likelihood":0.0019832603,"seconds":18.656}
{"filePath":"/usr/share/snips/dialogue/sound/end_of_input.wav"}
{"text":"please turn on the light","likelihood":0.0019832603,"seconds":18.656}
{"id":"/usr/share/snips/dialogue/sound/end_of_input.wav"}
{"filePath":"/usr/share/snips/dialogue/sound/start_of_input.wav"}
{"id":"/usr/share/snips/dialogue/sound/start_of_input.wav"}
{"text":"turn on the light","likelihood":0.0021343841,"seconds":5.078}
{"filePath":"/usr/share/snips/dialogue/sound/end_of_input.wav"}
{"text":"turn on the light","likelihood":0.0021343841,"seconds":5.078}
{"id":"/usr/share/snips/dialogue/sound/end_of_input.wav"}
{"filePath":"/usr/share/snips/dialogue/sound/start_of_input.wav"}
{"id":"/usr/share/snips/dialogue/sound/start_of_input.wav"}
{"text":"turn on the light","likelihood":0.002096636,"seconds":3.573}
{"filePath":"/usr/share/snips/dialogue/sound/end_of_input.wav"}
{"text":"turn on the light","likelihood":0.002096636,"seconds":3.573}
{"id":"/usr/share/snips/dialogue/sound/end_of_input.wav"}
When I ran it on another PI, Snips published the ActivateLightColor intent, so I didn’t change anything and added this to my intent_scripts.yaml file:
ActivateLightColor:
action:
- service: switch.turn_on
entity_id: switch.ge_14291_inwall_smart_switch_switch
In addition to there being no action taken when it parses the text, it also seems to be doubling up on the publishing. I’m waiting for my other PI3 to finish reinstalling the AIO so that I can test it on that. I’ll post the output from that as soon as it’s available.
Silly question maybe, but did you put snips: in your configuration file?
I did.
I had some major success last night using Snips on the second Pi. After the AIO finished setting up. I got Snips installed and used the same assistant.zip file that I’m using on the HASSIO server. I used the full Snips launch command to point it at the Mosquitto broker on the HASSIO server and that connected no problem. I could see the speech that it was parsing and the intents it was publishing. After that it was a little bit of tweaking the script, which it turns out I had formatted incorrectly, and eventually I got it to turn on the light and say stuff with the google-tts service. I still need to retry it on the HASSIO server but having it run on a second Pi is gets me most of what I want since I’d like to have microphones in other rooms.
great news! Well done.
Few questions, hope you can help me.
Today I also made some progress. I’m not so familiar with MQTT, so I’m using the Snips broker. And in the logs I can see that is is actually publishing an intent now. Unfortunately I do not know how to check the MQTT bus, so I don’t know if it is processed the good way.
I see you are using google and not the Snips ASR provider. Why did you choose this? Is it because you use another language, or you find it better? Was looking into it to, but you need a business to sign up for it (at lease in Europe).
I created switches in home assistant for all devices, so also for the lights. That way I can say turn the light off, and turn the TV off with the same code. Does that make sense? I trained my Snips assistant so that it recognized if I say on or off, and to pick up a device. But I think it is not correct. As sofar it is not working. What do you think?
snips:
intent_script:
ActivateObject:
action:
- service_template: >
{% if OnOff == "on" -%}
switch.turn_on
{%- else -%}
switch.turn_off
{% endif %}
data_template:
entity_id: "switch.{{ device | string }}"Sorry for the delayed response. After I got this working, I got excited/carried away with adding things that I forgot to check back on this thread.
FYI, I’m still only running Snips on a second Pi, so far, that hasn’t been a limitation.
To check the bus, you’ll need a computer that has the mosquitto-clients package. Once that’s installed, you can just subscribe to the broker. For example:
mosquitto_sub -h 192.168.1.3 -t hermes/intent/#
This will let you see everything that Snips is posting. It’s super helpful since it lets you see when it came up with the wrong text.
As far as my ASR provider, I think I’m using Snips ASR. It’s the default and I haven’t specified anything else.
As far as your script goes, I am certainly not an expert, but the structure would depend on the slots you have on the Snips Assistant. For example, I’m using the built in ActivateLightColor intent to control my lights. It’s very much a work in progress, but here’s how it looks right now:
ActivateLightColor:
speech:
text: Turning {{ objectLocation | replace(" ", "_") }} on
card:
type: simple
title: Turn object off
content: Ask Home Assistant to turn {{ objectLocation | replace(" ", "_") }} on
action:
service_template: >
{%- if objectLocation.lower() in ['testing', 'main'] -%}
light.turn_on
{%- else -%}
homeassistant.turn_on
{%- endif -%}
data_template:
entity_id: >
{%- if objectLocation.lower() in ['kitchen', 'main', 'testing'] -%}
light.
{%- elif objectLocation.lower() in ['living room'] -%}
group.
{%- elif objectLocation.lower() in ['entry', 'porch'] -%}
switch.
{%- endif -%}
{%- if objectLocation.lower() in ['kitchen'] -%}
kitchen_lights
{%- elif objectLocation.lower() in ['entry'] -%}
entry_way_switch
{%- elif objectLocation.lower() in ['testing'] -%}
lifx_1
{%- elif objectLocation.lower() in ['main'] -%}
ge_14294_inwall_smart_dimmer_level
{%- endif -%}
I’ve got a similar script for the DeactivateObject intent:
DeactivateObject:
speech:
text: Turning {{ objectType | replace(" ", "_") }} off
card:
type: simple
title: Turn object off
content: Ask Home Assistant to turn {{ objectType | replace(" ", "_") }} off
action:
service: homeassistant.turn_off
data_template:
entity_id: >
{%- if objectType.lower() in ['kitchen', 'main light', 'testing light'] -%}
light.
{%- elif objectType.lower() in ['living room'] -%}
group.
{%- elif objectType.lower() in ['entry light', 'porch'] -%}
switch.
{%- endif -%}
{%- if objectType.lower() in ['kitchen'] -%}
kitchen_lights
{%- elif objectType.lower() in ['entry light'] -%}
entry_way_switch
{%- elif objectType.lower() in ['testing light'] -%}
lifx_1
{%- elif objectType.lower() in ['main light'] -%}
ge_14294_inwall_smart_dimmer_level
{%- endif -%}
That one is a little easier because turning stuff off doesn’t have as many data fields (like color or brightness). The process of turning things on is proving a little tricky because I’m trying to cram lights and switches into the same script but lights have additional properties like color that my switches don’t.
Anyway, I can say for sure that you can run Snips on a second Pi and get HASSIO to run intent scripts based on the output.
1 Like
Thanks for your help, I can read the client bus now. Using snips on the hass.io and using the MQTT of hass.io
I’m getting the following output:
{“input”:“turn off the table light”,“intent”:{“intentName”:“ActivateObject”,“probability”:1.0},“slots”:[{“rawValue”:“off”,“value”:{“kind”:“Custom”,“value”:“off”},“ran
ge”:{“start”:5,“end”:8},“entity”:“onoff”,“slotName”:“onoff”},{“rawValue”:“table light”,“value”:{“kind”:“Custom”,“value”:“table”},“range”:{“start”:13,“end”:24},“entity”:“device”,“slot
Name”:“device”}]}
And have this Script:
snips:
intent_script:
ActivateObject:
action:
service_template: >
{%- if onoff() == "on" -%}
switch.turn_on
{%- else -%}
switch.turn_off
{%- endif -%}
data_template:
entity_id: switch.{{ device() | string }}
Do you see something wrong with this? Clearly there is, as it is not working. I wanted the same, switches and lights in same script, so I made switches for my lights so that these can be used in the input_intent, could work for you too. However you cannot add extra properties then.
- platform: template
switches:
table:
value_template: "{{ is_state('light.table', 'on') }}"
turn_on:
service: light.turn_on
entity_id: light.table
turn_off:
service: light.turn_off
entity_id: light.tablePlease excuse me for hijacking the thread a little bit but what microphones are you using?
Hi, I’m using a playstation eye.
I’m using a logitech pro9000 webcam at the moment. I plan to switch to a Matrix-Voice when they start shipping as long as they confirm that Snips can use them (currently it can’t).
Try2Fly
One thing that jumps out at me is that you are referencing the slots differently than I am. My ActivateLighColor script has a slot called objectLocation. When I reference that slot in the script, I just type it like a variable inside the JSON formatting like {{ objectLocation }}. You seem to be calling it like you would a function by adding the parentheses i.e. {{ device }} vs {{ device() }}. The same thing is true for your onoff slot.
Also, I don’t know that you need to explicitly convert device to string, but I don’t know that it’s hurting anything; just one more thing to think about.
Hi dbenhart, thanks for answering me! I think I tried everythink, but it is not working. I have 2 slotnames in snips, they are “onoff” and “device”. Does it automatically take the value of these slotnames as input?
This is not working:
intent_script:
ActivateObject:
action:
service_template: >
{%- if onoff == "on" -%}
switch.turn_on
{%- else -%}
switch.turn_off
{%- endif -%}
data_template:
entity_id: switch.{{ device }}
I also tried to make an intent without a template so just like:
intent_script:
ActivateObject:
action:
service: switch.turn_on
entity_id: switch.table
but this isn’t working either. Could it be that HA is not receiving the message that Snips publishes?
I suppose that’s possible, but I’m not sure how to test that explicitly. One thing to clarify, are you running Snips on the same Pi that HASSIO is running on, or are you running it on a separate device? I haven’t done much testing with them running on the same device, so there could be other issues with that.
Another thing to confirm is that Snips ans HASSIO are using the same broker. Snips starts its own by default and it runs on a different port than the default port for the mosquitto broker addon for HASSIO. You’re supposed to be able to bridge the two if they’re on the same machine but, again, I haven’t played around with that enough to know the common pitfalls. On my setup with the second Pi, i start Snips with the full command so that I can specify the mosquitto broker that HASSIO is using:
docker run -t --rm --name snips --log-driver none -v /home/pi/.asoundrc:/root/.asoundrc -v /opt/snips/config:/opt/snips/config --privileged -v /dev/snd:/dev/snd snipsdocker/platform --mqtt MQTT_BROKER_IP:MQTT_BROKER_PORT
exchange MQTT_BROKER_IP:MQTT_BROKER_PORT with the values for your setup. The default port for the mosquitto addon in HASSIO is 1883
I’d double check that you’re using the same broker for Snips and HASSIO by using the mosquitto_sub command with the broker that you think everything is running on. You can subscribe to all messages that Snips will send using:
mosquitto_sub -h YOUR_BROKER_IP -t hermes/intent/#
Then use the built in MQTT developer tool to publish something with the same topic. If you see both sets of messages, then everything should be using the same broker, in which case I’d guess that the problem is with your script formatting.
One last thing, it looks like you have too much indentation in your no-template script
intent_script:
ActivateObject:
action:
service: switch.turn_on
entity_id: switch.table
There should be 2 spaces between ActivateObject and intent_script. That may just be a quirk of how you pasted it, but YAML is sensitive to indents, so if that’s off, it won’t work.
Thanks for your answer. For now I’m using snips on the same raspi as Hassio. I’ll toy with it coming weekend. Else I got another raspi running docker, so should be possible to run snips on this pi.
The indentation is indeed a copy paste error. Without this, this should work if the intent is published, right?
Thanks for your help so far!
Hi dbenhart,
I’ve set up Snips on a seperate Pi, and tested mqtt like you suggested. This is working, however the intents are still not working… I’m completely in the dark now. Found this in my log file. Do you know what this means?
2017-09-09 22:58:29 ERROR (MainThread) [homeassistant.components.snips] Error while handling intent.
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/homeassistant/components/snips.py", line 67, in message_received
hass, DOMAIN, intent_type, slots, request['input'])
File "/usr/lib/python3.6/site-packages/homeassistant/helpers/intent.py", line 41, in async_handle
raise UnknownIntent()
homeassistant.helpers.intent.UnknownIntentHi dbenhart,
is there some other configuration that you have to do to have the response speech play and what does the card section do, thanks
I’m not sure what that error is. It looks like you’ve tried to pass it an intent that you haven’t set up. I don’t suppose that you have a github repo for your configuration?
Also, can you post the message that Snips is sending on the bus?