short version
i don’t know.
long version
I didn’t write my intent_script(s) from scratch; I copied and modified a script from another user and just left that in as it wasn’t causing any problems. I don’t really care what the card section does, but the speech is on my to-do list. I expect that it works like the text to speech service where you call service.say (which I also haven’t played with.)
No haven’t got GitHub working. Looked into it, but I couldn’t find how to do this for Hassio…
So this is what my intent looks like:
intent_script:
user_Yr2dkVnM6__ActivateObject:
action:
service_template: >
{%- if onoff == "on" -%}
switch.turn_on
{%- else -%}
switch.turn_off
{%- endif -%}
data_template:
entity_id: switch.{{device | string}}
and this is published on the bus:
Client mosqsub/2563-a0d7b954-t received PUBLISH (d0, q0, r0, m0, ‘hermes/intent/user_Yr2dkVnM6__ActivateObject’, … (349 bytes)) │·····························
{“input”:“turn on the”,“intent”:{“intentName”:“user_Yr2dkVnM6__ActivateObject”,“probability”:0.9026303},“slots”:[{“rawValue”:“on”,“value”:{“kind”:“Custo│·····························
m”,“value”:“on”},“range”:{“start”:5,“end”:7},“entity”:“onoff”,“slotName”:“onoff”},{“rawValue”:“table”,“value”:{“kind”:“Custom”,“value”:“table”},“range”:│·····························
null,“entity”:“device”,“slotName”:“device”}]}
Also, I find that Snips is recording for a long time (about 40-60 sec) after I say ‘Hey Snips’, which makes the system not as quick as I would like. Is this the same with you?
I’d try simplifying the process a little bit for troubleshooting. For example, the first thing I’d do is make sure that HA is doing something with the script by removing any variables. which is what you did here.
It looks like you’ve changed to using a custom intent. Did you get the ActivateObject script working?
Yes it works for me with default https://console.snips.ai/intents IoT - these 4 predifined intents and the above discribed intent_script: from dbenhart.
But only sometimes and snips often returned false objects, errors etc.
Defining my own intent on https://console.snips.ai/intents ( for example: user_Yr2dkVnM6__ActivateObject) never worked for me.
Snips publishes everthing on mqtt - so why not use mqqt trigger/scripts/sensors in homeassistant.
For testing I created my own intent on console.snips.ai - only one slot linked with some user sentences (turn on movie time etc.).
for my snips test environment i defined 4 sensors to see in time what snips returns:
- platform: mqtt
state_topic: "hermes/asr/textCaptured"
name: "Answer"
value_template: '{{ value_json.text }}'
- platform: mqtt
state_topic: "hermes/nlu/intentParsed"
name: "textinput"
value_template: '{{ value_json.input }}'
- platform: mqtt
state_topic: "hermes/nlu/intentParsed"
name: "intentname"
value_template: '{{ value_json.intent.intentName }}'
- platform: mqtt
state_topic: "hermes/nlu/intentParsed"
name: "probability"
value_template: '{{ value_json.intent.probability }}'
With sensor.probability you could do some math in automations.
simple scripts:
radio538:
alias: Play FFN
sequence:
- service: media_player.volume_set
data:
entity_id: media_player.srszr51
volume_level: '0.35'
- service: media_player.play_media
data:
entity_id: media_player.srszr51
media_content_id: 'http://stream.ffn.de/ffn/mp3-192'
media_content_type: 'audio/mp4'
snips_say_ok:
alias: Play OK Snips
sequence:
- service: mqtt.publish
data:
topic: "hermes/tts/say"
payload: '{"text":"OK master of the universe","lang":"en"}'
snips_say_notok:
alias: Play NotOK Snips
sequence:
- service: mqtt.publish
data:
topic: "hermes/tts/say"
payload: '{"text":"Master please learn english","lang":"en"}'
turnon_movie_light:
alias: Its movie time
sequence:
- alias: Spot 1
service: homeassistant.turn_on
data:
entity_id: light.hue_ambiance_candle_1
- alias: Spot 2
service: homeassistant.turn_on
data:
entity_id: light.hue_ambiance_candle_2
- alias: sayok
service: homeassistant.turn_on
data:
entity_id: script.snips_say_ok
Snips talks to me 
(hermes/tts/say)
simlple automation:
- id: do_some_on
alias: "Do Something"
initial_state: 'on'
trigger:
platform: mqtt
topic: "hermes/nlu/intentParsed"
action:
- service: script.turn_on
data_template:
entity_id: >
{% if (trigger is defined) and ('radio in kitchen' in trigger.payload) %}
script.radio538
{% elif (trigger is defined) and ('movie time' in trigger.payload) %}
script.turnon_movie_light
{%- else -%}
script.snips_say_notok
{% endif %}
For me it works like a charm
Hey snips - turn on radio in kitchen
Hey snips - turn on movie time
etc. etc.
after testing for a while now i think snips has potential for me in future if:
more languages especially german
better voice output
Another tip for all who want to test it:
I first used a cheap microphone plug in raspberry for testing -> bad
My Intent is to control my devices without active internet connection - no amazon or google cloud etc. cause of privacy or simply if internet is down - so snips seems to be perfect.
I’ve been using the custom intent al the time, but left it out to simplefy, as I thought it would not matter. Now reading @kasimabdalla post, this might be the problem why it is not working.
As I do want my custom intent I’m trying Kasimabdalla method to see if I can get that working. Thanks both!
How are your response time’s? My snips still has a long listening period before it sends the intent. And as I’m not native english it sometimes has trouble understanding me. Now I have to wait to long before it will respond, which makes it impractical.
For me the inclusion of Dutch would be great!!
Got it working now! only thing left is the response time!
This is my automation:
- id: Snips
alias: "Snips controle"
initial_state: 'on'
trigger:
platform: mqtt
topic: "hermes/intent/user_Yr2dkVnM6__ActivateObject"
action:
service_template: >
{% if (trigger is defined) and ('OFF' in trigger.payload) %}
switch.turn_off
{%- else -%}
switch.turn_on
{% endif %}
data_template:
entity_id: >
{% if (trigger is defined) and ('screen' in trigger.payload) %}
switch.screen
{% elif (trigger is defined) and ('cinema' in trigger.payload) %}
switch.cinema
{% elif (trigger is defined) and ('beamer' in trigger.payload) %}
switch.beamer
{% elif (trigger is defined) and ('receiver' in trigger.payload) %}
switch.receiver
{% elif (trigger is defined) and ('tv' in trigger.payload) %}
switch.tv
{% elif (trigger is defined) and ('blinds' in trigger.payload) %}
switch.blinds
{% elif (trigger is defined) and ('standing' in trigger.payload) %}
switch.standing
{% elif (trigger is defined) and ('spots' in trigger.payload) %}
switch.spots
{% elif (trigger is defined) and ('table' in trigger.payload) %}
switch.table
{% endif %}I installed snips straight forward from github snips-platform documentation:
1.-Setup-the-Snips-Voice-Platform-on-your-Raspberry-Pi
uploaded intent - start it in docker on my raspberry Raspberry Pi 3 Model B (wired connection)
My response times are very good:
Hey snips (1sec ) -> user input (1-2 sec) -> published on mqtt
yes I did the same. Hey snips (1sec) ->user input (30-35 sec before sound it makes sound) -> published on mqtt
I’m running several programs on the same raspberry that Snips is on. When closing these, it has become much faster, and I think it also understands me better. Don’t know if I want another Pi running in my room. Futhermore I need to be quit close (next to it) to the microphone in order for it to hear wat I’m saying.
The idea is great but for now it does not work sufficient enough for me to use it.
It’s exactly the same for me - I’m using the snips addon on hass.io. The wake word is recognized everytime, also when I’m in another room - and sometimes even when no one said it.
Speech recognition is also pretty bad for me - I have to be very close to the microphone and talk very slow. I have not yet figured out if it is because of the microphone (Playstation Eye), because of the software settings or because of my English (no native speaker).
I haven’t had any issues with processing time but, custom intents may be different in that regard. Regarding the microphone performance, I’d expect there to be issues using cheap USB mics. They are not omnidirectional and they are not designed for far-field use. They’ll work for testing, you can make sure that the scripts work, but I wouldn’t rely on them for everyday use. kasimabdalla mentioned the ReSpeaker from Seeed Studio; that and the Matrix Voice both seem to be good options. You can get a ReSpeaker now, the Matrix Voice is still in progress. The people at Matrix are working with Snips right now to get that up and running by the time they ship, I plan to get one of them, at least for testing.
I am having below error in the log…
Subdevice #0: subdevice #0
[Info] Setup audio device
[Info] Setup internal mqtt bridge
[Info] Start internal mqtt broaker
1509386043: Warning: Mosquitto should not be run as root/administrator.
1509386043: mosquitto version 1.3.4 (build date 2017-05-29 22:25:09+0000) starting
1509386043: Config loaded from /etc/mosquitto.conf.
1509386043: Opening ipv4 listen socket on port 1883.
1509386043: Opening ipv6 listen socket on port 1883.
1509386043: Connecting bridge main-mqtt (192.168.1.27:1883)
Couldn’t find any assistant
The assistant.zip file is placed in the config folder and mqtt and snips is defined in the configuration.yaml file.
Can anyone help me on starting snips on hassio?
Please check https://home-assistant.io/addons/snips/
Connect to the “share” Samba share and copy your training data over. Name the file assistant.zip. => you should place assistant.zip in /share/
I am hoping they release dutch and a custom hotword soon!
Hi,
I set up snips on hassio. It also detect the hot word and the sentence that I speak as I can see on the logs.
But it does not call the intent.
Further to my investigation i found that irrespective of specifying the mqtt server in the add on configuration , it does not connect ti it and start its own mqtt at 127.0.0.1.
Below are the log from the mqtt server addon
1510064306: New connection from 172.30.32.1 on port 1883.
1510064306: Client core-snips.main-mqtt disconnected.
1510064306: New client connected from 172.30.32.1 as core-snips.main-mqtt (c0, k60, u’username’).
1510064390: Socket error on client core-snips.main-mqtt, disconnecting.
1510064401: New connection from 172.30.32.1 on port 1883.
1510064401: Client core-snips.main-mqtt disconnected.
1510064401: New client connected from 172.30.32.1 as core-snips.main-mqtt (c0, k60, u’username’).
1510064676: New connection from 172.30.32.1 on port 1883.
1510064676: New client connected from 172.30.32.1 as mosqsub/1675-core-ssh (c1, k60).
1510064686: Socket error on client core-snips.main-mqtt, disconnecting.
1510064711: New connection from 172.30.32.1 on port 1883.
1510064711: Client core-snips.main-mqtt disconnected.
1510064711: New client connected from 172.30.32.1 as core-snips.main-mqtt (c0, k60, u’username’).
Below are the log from the snips add on:
10:25:22.749053] INFO :hermes::mqtt: Connecting to MQTT broker at address localhost:1883
2017-11-07 10:25:22,750 INFO gave up: snips-audio-server entered FATAL state, too many start retries too quickly
1510064722: New connection from 127.0.0.1 on port 1883.
1510064722: New client connected from 127.0.0.1 as mqttc_326161151 (c1, k5).
[10:25:22.754080] INFO :rumqtt::connection: $$$ Connected to broker
[10:25:22.755285] ERROR:snips_hotword : Error: Cannot assign requested address (os error 99)
1510064722: Socket error on client mqttc_326161151, disconnecting.
2017-11-07 10:25:22,761 INFO exited: snips-hotword (exit status 1; not expected)
Error: Unable to read file "/opt/snips/config/assistant/trained_assistant.json"
Caused by: missing field language at line 1 column 105948
1510064722: Socket error on client mqttc_2747760409, disconnecting.
2017-11-07 10:25:22,821 INFO exited: snips-queries (exit status 1; not expected)
2017-11-07 10:25:25,832 INFO spawned: ‘snips-hotword’ with pid 154
2017-11-07 10:25:25,842 INFO spawned: ‘snips-queries’ with pid 155
[10:25:25.940629] INFO :hermes::mqtt: Connecting to MQTT broker at address localhost:1883
1510064725: New connection from 127.0.0.1 on port 1883.
1510064725: New client connected from 127.0.0.1 as mqttc_1848329584 (c1, k5).
[10:25:25.951717] INFO :rumqtt::connection: $$$ Connected to broker
[10:25:26.011359] INFO :queries_hermes : loading model at path “/opt/snips/config/assistant”
[10:25:26.315970] INFO :hermes::mqtt: Connecting to MQTT broker at address localhost:1883
1510064726: New connection from 127.0.0.1 on port 1883.
1510064726: New client connected from 127.0.0.1 as mqttc_3300695550 (c1, k5).
[10:25:26.321469] INFO :rumqtt::connection: $$$ Connected to broker
[10:25:26.323070] ERROR:snips_hotword : Error: Cannot assign requested address (os error 99)
1510064726: Socket error on client mqttc_3300695550, disconnecting.
Below is the configuration addon:
{
“mqtt_bridge”: {
“active”: true,
“host”: “192.168.1.27”,
“port”: 1883,
“user”: “username”,
“password”: “password”
},
“mic”: “1,0”,
“speaker”: “0,1”,
“assistant”: “assistant.zip”
}
Below is my MQTT from configuration addon file:
mqtt:
broker: 192.168.1.27
port: 1883
client_id: home-assistant-1
username: username
password: password
Can you guys please help me to run snips on the hassio ??
You got this in the logs:
Are you sure the hotword and sentence is detected?
Also, you do not need the MQTT, please paste your snips: configuation part
Hello @kasimabdalla,
Could you please share what turning on a simple switch via snips would look like, script wise?
Here is the code within configuration.yaml file:
switch:
Any help is greatly appreciated.
Antonio
Hi - I need some help. I have managed to configure Speach-To-Text on SNIPS, but I’m struggling with Text-To-Speach section. Below is my config:
{
“mqtt_bridge”: {
“active”: true,
“host”: “192.168.1.11”,
“port”: 1883,
“user”: “homeassitant”,
“password”: “this is not my real password”
},
“mic”: “1,0”,
“speaker”: “0,0”,
“assistant”: “assistant.zip”
}
I’m using speaker connected to Pi3’s jack output. Snips beeps on “HEY SNIPS” (so I presume that the speaker and audio output are OK), but it does not say or send any commands to the speaker.
in my script.yaml file I have the following:
snips_say_ok:
alias: Play OK Snips
sequence:
- service: tts.google_say
data:
message: 'OK master of the universe!'
- service: mqtt.publish
data:
topic: "hermes/tts/say"
payload: '{"text":"Master please learn english","lang":"en"}'
snips_say_notok:
alias: Play NotOK Snips
sequence:
- service: mqtt.publish
data:
topic: "hermes/tts/say"
payload: '{"text":"Master please learn english","lang":"en"}'
Could you be so kind and let me know what am I doing wrong?
Thank you in advance.
hallo, i have the same problem:
Have you solved the problem ?
Alex
[23:33:18.225184] INFO :queries_hermes : loading model at path “/opt/snips/config/assistant”
Error: Unable to read file "/opt/snips/config/assistant/trained_assistant.json"
1511735598: New connection from 127.0.0.1 on port 1883.
1511735598: New client connected from 127.0.0.1 as mqttc_4232104926 (c1, k5).
[23:33:18.214302] INFO :rumqtt::connection: $$$ Connected to broker
[23:33:18.225184] INFO :queries_hermes : loading model at path “/opt/snips/config/assistant”
Error: Unable to read file "/opt/snips/config/assistant/trained_assistant.json"
Caused by: missing field language at line 1 column 1024
1511735598: Socket error on client mqttc_4232104926, disconnecting.
2017-11-26 23:33:18,250 INFO exited: snips-queries (exit status 1; not expected)
2017-11-26 23:33:19,254 INFO gave up: snips-queries entered FATAL state, too many start retries too quickly
[23:33:31.070711] INFO :snips_dialogue_hermes::dialogue: Timeout is reached. Going to Idle mode
snips_stt --use-energy=false --num-mel-bins=40 --num-ceps=40 --low-freq=20 --high-freq=-400 --sample-frequency=16000
[23:33:36.983049] INFO :hermes::mqtt: Connecting to MQTT broker at address localhost:1883
1511735617: New connection from 127.0.0.1 on port 1883.
1511735617: New client connected from 127.0.0.1 as mqttc_1141237573 (c1, k5).
[23:33:37.154045] INFO :rumqtt::connection: $$$ Connected to broker
[23:33:37.199845] INFO :snips_asr_hermes : Idle
[23:33:43.215702] INFO :snips_hotword_hermes: Hotword detected
[23:33:43.300444] INFO :snips_dialogue_hermes::dialogue: State: Idle, incoming Message: HotwordDetected
[23:33:43.308292] INFO :snips_dialogue_hermes::dialogue: Current State: WaitingQuery
[23:33:43.344571] INFO :snips_asr_hermes : Listening
[23:33:43.433092] INFO :snips_audio_portaudio: Playing “/usr/share/snips/dialogue/sound/start_of_input.wav” using output “default”, wav spec : WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
[23:33:46.489202] INFO :snips_asr_lib::asr: Endpoint detection.
[23:33:46.652349] INFO :snips_asr_hermes : Cleanup
[23:33:46.652445] INFO :snips_asr_hermes : Idle
[23:33:46.654974] INFO :snips_dialogue_hermes::dialogue: State: WaitingQuery, incoming Message: AsrTextCaptured(TextCapturedMessage { text: “turn the lights on”, likelihood: 0.0021969997, seconds: 3.307 })
[23:33:46.665126] INFO :snips_audio_portaudio: Playing “/usr/share/snips/dialogue/sound/end_of_input.wav” using output “default”, wav spec : WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
[23:33:46.675829] INFO :snips_dialogue_hermes::dialogue: Current State: WaitingIntent
[23:33:46.771664] INFO :snips_analytics_hermes: Cleanup
[23:33:46.771855] INFO :snips_analytics_hermes: Idle
Can you please share your configuration.yaml file… Need to check the configuration for the mqtt and also can share the details of the description in the add on part of the hassio which looks like this
{
“mqtt_bridge”: {
“active”: true,
“host”: “hostname”,
“port”: port,
“user”: “username”,
“password”: “password”
},
“mic”: “1,0”,
“speaker”: “0,0”,
“assistant”: “assistant.zip”
}