Snips.ai with a satellite now working

Installation Home Assistant OS
1

So, to start off, I have spent days trying to read every guide (there are no complete ones I could find for setting up Satellites with the Snips.ai Add-on under Hass.io) and FAQ and topic from this forum and many others, but I’m still struggling to get it to work. I have also tried the Snips discord channel, but no one was able to help there either (just saying, not complaining :-), but no-one responds there).

That being so, I have decided to create this topic with my issue detailed for assistance :slight_smile:

So, what’s my set-up?

I have:

  • Hass.io on a Pi 3B+
  • The Snips.ai Add-on installed
  • The Mosquitto Add-on installed
  • A satellite set-up running:
    ** On a Pi0
    ** Respbian Stretch
    ** The snips-satellite service
    ** With a ReSpeaker 2

Let me know if you need more detail then the above, but here’s some of my configs.

Note: I have sanitised all passwords with he string “********”, but these are all set and correct in my actual config.

Snips.ai Add-on:

{
  "assistant": "assistant2.zip",
  "language": "en",
  "custom_tts": {
    "active": false,
    "platform": "amazon_polly"
  }
}

Note: The file /share/assistant2.zip does not exist and so the default assistant is being used with the “Hay Snips” hotword.

Mosquitto Add-on:

{
  "logins": [
    {
      "username": "kitchen",
      "password": "********"
    },
    {
      "username": "bedroom",
      "password": "********"
    },
    {
      "username": "conservatory",
      "password": "********"
    },
    {
      "username": "temp",
      "password": "********"
    },
    {
      "username": "mower",
      "password": "********"
    },
    {
      "username": "snips_bridge",
      "password": "********"
    }
  ],
  "anonymous": false,
  "customize": {
    "active": false,
    "folder": "mosquitto"
  },
  "certfile": "fullchain.pem",
  "keyfile": "privkey.pem"
}

Note: The key files do not exist, so for now, the secure connection stuff is not in use.

The snips.toml file placed in /share/snips.toml (in an attempt to apply various fixes, none of which have worked) for use by the Hass.io Snips.ai Add-on base unit:

[snips-common]
# bus = "mqtt"
mqtt = "localhost:1883"
# audio = ["default@mqtt"]
# assistant = "/usr/share/snips/assistant"
# user_dir = "/var/lib/snips"

## MQTT authentication
mqtt_username = "snips_bridge"
mqtt_password = "********"

[snips-audio-server]
bind = "default@mqtt"

[snips-hotword]
audio=["default@mqtt","bedroom@mqtt"]

[snips-dialogue]
sound_feedback_disabled_default = true

The snips.toml file in use by my satellite:

[snips-common]
# bus = "mqtt"
mqtt = "192.168.178.23:1883"
# audio = ["+@mqtt"]
# assistant = "/usr/share/snips/assistant"
# user_dir = "/var/lib/snips"

## MQTT authentication
mqtt_username = "bedroom"
mqtt_password = "********"

## MQTT TLS configuration
# mqtt_tls_hostname = ""
# mqtt_tls_disable_root_store = false
# mqtt_tls_cafile = ""
# mqtt_tls_capath = ""
# mqtt_tls_client_cert = ""
# mqtt_tls_client_key = ""

[snips-analytics]

[snips-asr]
# no_fst_map = true
# beam_size = 8
# model = "/usr/share/snips/asr"
# audio = ["+@mqtt"]
# partial = false
# partial_period_ms = 250

[snips-asr-google]
# credentials = "/usr/share/snips/googlecredentials.json"
# audio = ["+@mqtt"]

[snips-audio-server]
# frame = 256
bind = "bedroom@mqtt"
alsa_capture = "default"
alsa_playback = "default"
# mike = "Built-in Microphone"
# disable_playback = false
# disable_capture = false

[snips-dialogue]
# resources = "/usr/share/snips/dialogue"
# session_timeout = 15
# lambda_timeout = 5
# retry_count = 3
sound_feedback_disabled_default = true

[snips-hotword]
# model = "/usr/share/snips/assistant_moved/custom_hotword/"
# hotword_id = "default"
# sensitivity = "0.5"
# audio = ["+@mqtt"]
# no_vad_inhibitor = false
# vad_messages = false

[snips-injection]
#asr_model = "/usr/share/snips/asr"
#nlu_engine = "/usr/share/snips/nlu_engine"
#keep_nfirst = 3

[snips-nlu]

[snips-tts]
## Choose one tts provider (defaults to picotts)
# provider = "picotts"
# provider = "makerstts"
# provider = "customtts"
## customtts specific configuration (here configured to use picotts using the en-US language)
## available placeholder variables : %%OUTPUT_FILE%%, %%LANG%%, %%TEXT%%
# customtts = { command = ["pico2wave", "-w", "%%OUTPUT_FILE%%", "-l", "en-US", "%%TEXT%%"] }

[snips-pegasus]
# address = "0.0.0.0:8443"
## Pegasus TLS configuration
## Self signing certificates are generated with /usr/local/bin/snips-pegasus-tls-keygen.sh
# tls_certificate_chain = "/var/lib/snips/pegasus/tls/server.fullchain"
# tls_private_key = "/var/lib/snips/pegasus/tls/server.key"

The asound.conf file on my satellite:

# The IPC key of dmix or dsnoop plugin must be unique
# If 555555 or 666666 is used by other processes, use another one


# use samplerate to resample as speexdsp resample is bad
defaults.pcm.rate_converter "samplerate"

pcm.!default {
    type asym
    playback.pcm "playback"
    capture.pcm "capture"
}

pcm.playback {
    type plug
    slave.pcm "dmixed"
}

pcm.capture {
    type plug
    slave.pcm "array"
}

pcm.dmixed {
    type dmix
    slave {
        pcm "hw:seeed2micvoicec"
        period_time 0
        period_size 1024
        buffer_size 4096
    }
    ipc_key 555555
    ipc_perm 0666 # share the interface for all users.
}

pcm.array {
    type dsnoop
    slave {
        pcm "hw:seeed2micvoicec"
        channels 2
        rate 16000 # drastically reduce the audio-server cpu consumption by avoiding resampling
        # Warning, for piHAT 2, the playback interface will be also forced to 16000 Hz, you must use alsa_playback="default" option or comment this rate configuration.
    }
    ipc_key 666666
    ipc_perm 0666 # share the interface for all users.
}

Note: I have tested the sound on the satellite using “arecord -f cd out.wav” and “aplay out.wav” and it works perfectly.

Other notes: I don’t have SAM as I’m on a Mac and I could not get SAM to work at all on OSX, so please bare this in mind, thanks.

So! What works?

When I try the snips using a mic and speaker on the Pi3B+ that has Hass.io and Snips.ai on it directly, it works fine. It generates the intents and Hass.io responses… basically, it’s all working perfectly.

So what does NOT work?

The satellite will detect the hot-work and seems to communicate with the base unit to trigger a session etc, but for some reason, after the ASR is started, nothing is recognised at all.

Please see the log generated by “snips-watch -vvv” below for a single session where I say “Hay snips” then wait (for the session to have been created) and say “hello” (note: I have a simple intent for bellow in my Hass.io configuration file that works when I do this from the base unit, as described above).

Here is what snips-watch shows:

pi@raspberrypi:~ $ snips-watch -vvv
[11:01:55] Watching on 192.168.178.23:1883 (MQTT)
[11:01:59] [Sound Feedback] was asked to toggle itself 'on' on site bedroom
[11:02:10] [Hotword] detected on site bedroom, for model hey_snips
[11:02:11] [Asr] was asked to stop listening on site bedroom
[11:02:11] [Hotword] was asked to toggle itself 'off' on site bedroom
[11:02:11] [Asr] was asked to listen on site bedroom
[11:02:25] [Asr] was asked to stop listening on site bedroom
[11:02:25] [Hotword] was asked to toggle itself 'on' on site bedroom

Here is what the Snips.ai log itself shows for the same request:

[11:02:10] [Hotword] detected on site bedroom, for model hey_snips
[11:02:10] [Asr] was asked to stop listening on site bedroom
INFO:snips_dialogue_lib::coordinator::coordinator: Site [bedroom]: reset site -> turn off ASR and turn off Hotword
INFO:snips_dialogue_lib::coordinator::coordinator: Session [3b074709-40ed-4a9d-9a3b-29de4fe4686d]: The session has been created because the site "bedroom" is free
[11:02:10] [Hotword] was asked to toggle itself 'off' on site bedroom
[11:02:10] [Dialogue] session with id '3b074709-40ed-4a9d-9a3b-29de4fe4686d' was started on site bedroom
[11:02:10] [Asr] was asked to listen on site bedroom
INFO:snips_dialogue_lib::coordinator::coordinator: Session [3b074709-40ed-4a9d-9a3b-29de4fe4686d]: closing session which was active: Timeout
INFO:snips_dialogue_lib::coordinator::coordinator: Site [bedroom]: reset site -> turn off ASR and turn on Hotword
[11:02:25] [Dialogue] session with id '3b074709-40ed-4a9d-9a3b-29de4fe4686d' was ended on site bedroom. The session was ended because one of the component didn't respond in a timely manner
[11:02:25] [Asr] was asked to stop listening on site bedroom
[11:02:25] [Hotword] was asked to toggle itself 'on' on site bedroom

So, I cannot work out why it’s not working :frowning:

All help is appreciated, so thanks for anyone that can offer some advice. :slight_smile:

2

OK, so (vary) long story short:

The above config works, however there’s some kind of issue with Snips in Hass.io.

If the satellite or HA is restarted, then I must restart the Snips.ai Add-on in HA. Once I restart the Add-on, everything start to work OK (not perfect, but OK).

The entire set-up is slow to respond, hugely complex to configure the intents (the ones you install into your Snips Assistant file are basically ignored and only the ones you explicitly create in HA actually work… slowly) and finally it seems not to even follow the configurations (disabling the notification sound does not work).

I’m looking into having a dedicated Snips base station now, because the HA solution just isn’t up to the job.