Snips MQTT bridging

Configuration
1

I am trying to integrate Snips with HA but a lot of the Snips docs are pretty out of date. I can’t seem to figure out what topics I should be bridging between HA and the Snips MQTT broker.

If I point Snips at HA’s own MQTT broker, everything works as expected (intents interpreted, results sent back to Snips, calling snips.say TTS works). But it looks like it is sending all it’s binary audio data over MQTT as well which seems unnecessary.

I found a few examples of bridging here on the forums but they do not seem to work with the current Snips, here is my mosquitto.conf currently:

connection bridge-to-snips
address snips.lan:1883
clientid bridge-from-hass
start_type automatic
topic hermes/intent/# in 2 "" ""
topic hermes/tts/say out 2 "" ""

Any current users of Snips have a working bridge config to share? Also curious if anyone has deployed Snips using Docker, or with multiple audio clients, as I cannot find examples of either anywhere.

1 Like
2

Here is my mosquitto.conf, it works on my system with snips. I have a couple of satellites running the snips-audio-server and snips-skill-server (for led lights); the main snips unit runs all of the snips services. My HA is on a Debian server, using python virtualenv not docker.
The bridge setup is configured on the snips mosquitto server, not the HA mosquitto server. I cannot remember if the mosquitto server on the snips main unit was installed with snips or if I installed it separately.
My HA MQTT is pointed at a mosquitto server that is running on yet another Debian server I use for other services. It is the hub for MQTT on my system.
Hope this helps.

# Bridge to HA
connection snips-ha
# bridge_cafile /etc/mosquitto/
bridge_insecure true
address someserver:1883
cleansession false
clientid snips
start_type automatic
username someuser
password somepassword
notifications false
try_private true
topic # both 2 hermes/intent/ hermes/intent/
topic # both 2 hermes/dialogueManager/ hermes/dialogueManager/

The audio packets do not pass over my main MQTT server, but do pass over the snips satellites and main snips server.

1 Like
3

I’ve been trying to figure out how to get the satellites working. Was this just changes to the snips.toml?

Also, does your main snips unit also have a mic and speakers?

4

My main snips pi has a respeaker 2 mic hat, and the satellites are using the respeaker 4 mic hat.
I setup the satellites with help from psychokillers very helpful satconnect script- worked perfectly.


Make sure you have the main unit working before attempting to connect the satellite …
Also, make sure the satellites are connecting to the main snips unit mqtt server- not the ha mqtt server. The satellites only need the snips-audio-server running- all explained in the satconnect readme…
2 Likes
5

Much appreciated, that worked for me bridging HA -> Snips MQTT.

I have another question for you guys, I am getting an error when attempting to use the “HomeAssistant” skill from the Snips Console. Do you use it, or are you making custom intents for everything, or something else? If I define an intent manually in HA it works, but not this Snips-HA integration.

ie “Turn off kitchen light”

hermes/intent/hass:HassLightSet {"sessionId":"aab9d81b-461e-4f2e-82f5-d0538aa81c2b","customData":null,"siteId":"default","input":"turn kitchen light","intent":{"intentName":"hass:HassLightSet","probability":0.77675915},"slots":[{"rawValue":"light","value":{"kind":"Custom","value":"light"},"range":{"start":13,"end":18},"entity":"light","slotName":"light"}]}

in my HA logs…

2018-07-13 03:54:57 WARNING (MainThread) [homeassistant.helpers.intent] Received invalid slot info for HassTurnOn: extra keys not allowed @ data['site_id']
2018-07-13 03:54:57 ERROR (MainThread) [homeassistant.components.snips] Error while handling intent: HassTurnOn.
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/helpers/intent.py", line 59, in async_handle
    result = await handler.async_handle(intent)
  File "/usr/src/app/homeassistant/helpers/intent.py", line 187, in async_handle
    slots = self.async_validate_slots(intent_obj.slots)
  File "/usr/src/app/homeassistant/helpers/intent.py", line 142, in async_validate_slots
    return self._slot_schema(slots)
  File "/usr/local/lib/python3.6/site-packages/voluptuous/schema_builder.py", line 267, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.6/site-packages/voluptuous/schema_builder.py", line 587, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.6/site-packages/voluptuous/schema_builder.py", line 425, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['site_id']
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/components/snips.py", line 139, in message_received
    hass, DOMAIN, intent_type, slots, request['input'])
  File "/usr/src/app/homeassistant/helpers/intent.py", line 65, in async_handle
    'Received invalid slot info for {}'.format(intent_type)) from err
homeassistant.helpers.intent.InvalidSlotInfo: Received invalid slot info for HassTurnOn

Not sure where the problem lies here, my Snips install, HA, or the ‘Skill’

6

You did not specify in your input whether to turn kitchen light on, off or change color, etc. Maybe that is the issue…

If the input is correct, try this, around June 30… especially the comment by dthulke…

might solve your problem too.

2 Likes
7

I don’t have the link handy, but there is a bug in Hass.io. It has to do with Snips adding the site_id. There is a fix already in the dev branch, so I’m hoping it will be included in the .74 beta.

8

So using Psychokiller’s script, I think I have the satellites connected properly, but I’m lacking the “Bridge”.

The mosquitto.conf example that is posted… Is that on the Hass.io instance, the snips hub or the snips satellite?

9

I used mosquitto on the main snips pi3. It is configured as a bridge to the HA mqqt server using the code I posted. The satellites connect with mqtt to the main snips pi, not the ha mqtt. If you can visualize the snips system and the ha system as separate, then the satellites connect to the main snips unit. Then, the main snips unit bridges to the HA system. The bridging is just a component of the MQTT; other than that it will receive and send to anyone else (in this case the satellites). It only bridges a couple of topics with the HA mqtt server. The snips system units are identified by the id@mqtt and is their site-id in the mqtt message.

The snips main unit snips.toml (192.168.3.20):

[snips-tts]

[snips-dialogue]
lambda_timeout = 8

[snips-asr]

[snips-nlu]

[snips-asr-google]

[snips-analytics]

[snips-hotword]
audio = [“default@mqtt”, “snipspisat-1@mqtt”, “snipspisat-2@mqtt”]

[snips-audio-server]
bind = “default@mqtt”

[snips-common]

Snips satellite-1 snips.toml (192.168.3.21):

[snips-tts]

[snips-dialogue]

[snips-asr]

[snips-nlu]

[snips-asr-google]

[snips-analytics]

[snips-hotword]

[snips-audio-server]
bind = “snipspisat-1@mqtt”

[snips-common]
mqtt = “192.168.3.20:1883”

Snips satellite-2 snips.toml (192.168.3.22):

[snips-tts]

[snips-dialogue]

[snips-asr]

[snips-nlu]

[snips-asr-google]

[snips-analytics]

[snips-hotword]

[snips-audio-server]
bind = “snipspisat-2@mqtt”

[snips-common]
mqtt = “192.168.3.20:1883”

mosquitto.conf (on snips-main, 192.168.3.20)

# Bridge to HA
connection snips-ha
# bridge_cafile /etc/mosquitto/
bridge_insecure true
address _your ha mqtt server_:1883 (_or change to your mqtt port if other than 1883_)
cleansession false
clientid snips
start_type automatic
username _your user_
password _your password_
notifications false
try_private true
topic # both 2 hermes/intent/ hermes/intent/
topic # both 2 hermes/dialogueManager/ hermes/dialogueManager/
2 Likes
10

Awesome, thank you!

I didn’t install Mosquitto on my main Snips, the scripts from Psychokiller installed “paho-mqtt”. Since the Mosquitto folder and config file exists I’ll start there.

Have you tried getting the satellites running the hotword detection to reduce the audio traffic?

11

Thank you I was driving myself crazy trying to figure out what was going on I assumed I had it misconfigured… that is exactly what’s it doing. I will wait for .74 I guess.

Here was the MQTT config I adapted from lightmind, my bridge goes HA MQTT->Snips MQTT

connection bridge-to-snips
bridge_insecure true
address snips.lan:1883
cleansession false
clientid snips
start_type automatic
try_private true
topic # both 2 hermes/intent/ hermes/intent/
topic # both 2 hermes/dialogueManager/ hermes/dialogueManager/

Thank you for the satellite configs, that will definitely be my next project.

While you other Snips users are here… have either of you integrated some kind of notification LED, similar to the way Alexa etc light up when the hot word is said? Any idea what would be the best way to integrate that? I just saw these new cheap LED hats from Adafruit, which looks like they’d play nice with a Pi DAC.

12

I managed to get the mic and speaker working on a AIY Voice Kit. I haven’t bothered with the button yet.

That being said, I had another Pi3 so I ordered the “ReSpeaker 2-Mics Pi HAT”. It’s the same hardware Snips is packaging in their developer kit. So with the Snips SAM install, it will ask you if you have the ReSpeaker and will auto magically install everything. The lights work. They turn on when it wakes and blink back and fourth when processing. I haven’t done much to try and control them. It also has a button but I haven’t messed with it yet.

I’m still crossing my fingers I followed everything right in Git and the fix will be in .74. I’m not very familiar with Git or Docker.
This is the bug I was referencing: https://github.com/home-assistant/home-assistant/issues/14918

I might even jump on the beta train just to get this fix a week early.

13

I use the 2-mic respeaker hat and the 4 mic respeaker hat. The pimoroni led shims are nice but the respeaker has the mics and led’s (the 2 mic version has a headphone also) making it a multi-purpose addon to the pi. Also, for the two-mic hat snips has automated install of the drivers and led skill.

1 Like
14

Looks like the beta is out and has the fix. I haven’t installed it yet, but I might give it a shot later.

I did get the MQTT bridging working… I think. Now my satellite is giving me the error that the session has ended because something didn’t respond in a timely manner.

If I “watch” snips on the satellite, it shows it’s watching on my main Snips MQTT. When I “watch” on the main Snips, it watches on the HASS MQTT.

It seems like the last time I had this error, I had to change my MQTT username and password. I’m not sure if there is a cert or something else created that it could be caching? Any ideas?

15

Make sure you are bridging both in an out. Here is my config on my snips mqtt bridging to my HA mosquitto broker. No need to do anything on the HA mqtt, this will bridge all relevant topics in and out.

topic hermes/dialogueManager/# in
topic hermes/asr/# in
topic hermes/hotword/# out

topic hermes/intent/# out
topic hermes/asr/# out
topic hermes/hotword/# out
topic hermes/nlu/# out

Full config here https://raw.githubusercontent.com/tschmidty69/homeassistant-config/master/snips/mosquitto.conf

1 Like
16

Hi tschmidty,

first of all thanks a lot for your work with snips.

I have this error on my snips when I use your config:

Could not start MQTT client on localhost:1883

 -> caused by: Connection refused (os error 111)

I had this before and made fresh install with sam.

I did follow the steps shown in the guide ( https://docs.snips.ai/getting-started/quick-start-raspberry-pi )

Here i just clicked enter for “default”

Checking actions for end-user parameters

i Action snips-skill-respeaker contains end-user parameters that need to be configured 

? **Please enter a value for the parameter [mqtt_host]** localhost

? **Please enter a value for the parameter [mqtt_port]** 1883

? **Please enter a value for the parameter [site_id]** default

? **Please enter a value for the parameter [led_bri]** 64

i You can change these values later by editing the file located at /var/lib/snips/skills/snips-skill-respeaker/config.ini 

Relaunching snips-skill-server

So i do have a very clean installation and than entered your config.
snips ip is 100% correct
homeassistant ip is 100% correct

# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

listener 1883 10.1.1.82
listener 1883 127.0.0.1
connection_messages true
#log_type all
allow_anonymous true
password_file /etc/mosquitto/pwfile

connection snipsmqtt
address 10.1.1.85:1883
#remote_username HASS_MQTT_USER
#remote_password HASS_MQTT_PASSWORD
remote_clientid snips
start_type automatic
topic hermes/dialogueManager/# in
topic hermes/asr/# in
topic hermes/hotword/# out

topic hermes/intent/# out
topic hermes/asr/# out
topic hermes/hotword/# out
topic hermes/nlu/# out

on homeassistant i just added

mqtt:

to the configuration.yaml

17

So you need to set up mosquitto on the HA server. The default mqtt server on the HA server won’t really work.

So set up mosquitto on HA and specify the broker address to tell HA to use that instead of it’s more basic built in one

mqtt:
  broker: YOUR_IP_ADDRESS
18

To clarify you should have mosquitto configured with a basic setup on the HA server, and that config goes on the one on your snips server.

19

Thank you for the quick answer.

That how my config looked like before:

mqtt:
  broker: 10.1.1.85
#  port: 1883
#  username: mqtt-user
#  password: mqtt-123789
#  client_id: homeassistant
#  discovery: true

The stuff behind the # is what i had before. I commented it out to minimize the possibility of mistakes.

Testing now

20

still getting this error on snips server

Toto-PC:~ toto$ sam watch
Could not start MQTT client on localhost:1883
 -> caused by: Connection refused (os error 111)


┌─────────────────────────────────────────────────────────┐
│              snips-sam update check failed              │
│           Try running with sudo or get access           │
│          to the local update config store via           │
│ sudo chown -R $USER:$(id -gn $USER) /Users/toto/.config │
└─────────────────────────────────────────────────────────┘

Do i need to edit the snips.toml? I left it at default…