How to: Run Wyoming Satellite and OpenWakeWord on Android

Share your Projects!
, ,
82

I believe I have found the issue with the SIGSYS:31 error.

After investigating the stack traces, the error occurs on a syscall to pidfd_open aka syscall __NR_pidfd_open. It appears that the Android kernal’s SECCOMP is denying the syscall due to pidfd_open not being on the SECCOMP whitelist for older versions of Android (mine is v8.1.0, kernel 3.18.14).

Per discussion on how GO dealt with this, it appears that this syscall was properly added to the seccomp allow list in Android 12, so the issue would theoretically affect devices running Android 11 or lower.

My own testing has shown that the kernel DOES expose a pidfd_open syscall, but because Android improperly handles it in the whitelist (it should error with a SECCOMP_RET_TRAP or similar but instead errors with a SECCOMP_RET_KILL.

The relevant syscall is actually performed by the underlying python asyncio library, ironically during the very check to see if it should avoid using pidfd…:man_facepalming:. This appears to be getting called the first time that asyncio.create_subprocess_exec(...) is called, which in this case is when the satellite calls the wyoming library’s mic connect function.

My guess is that (at least part of) the reason @1Drew was able to make their workaround work is that the ubuntu distro allows the code to make that check without hitting the actual Android system SECCOMP check.

With all that in mind, I think so far the workarounds may be:

  • Root the android device and edit the SECCOMP config to reply with an error that doesn’t also kill the calling process (haven’t investigate how easy this is yet)
  • Run the code in a containerized fashion somehow such that this check doesn’t get made against the actual Android kernel.
  • Upgrade the Android version to 12+ (for old devices, this is not possible and thus would have to try rooting and installing something like lineageOS)
1 Like
83

Fantastic and exciting work everyone!

Given that so many people (including myself) use android tablets, which are mostly always on and have the Home Assistant Companion app in the foreground -
wouldn’t it be somewhat easier to add the keyword detection functionality natively into the android app?

Maybe this is a naive view, but in my eyes, it would have several benefits:

  • Easier microphone access (the app already has the permission)
  • TFlite, which the wake word detection is based on, was originally developed to be embedded in android apps anyway and has been done thousands of times before.
  • No need to mess around with Termux/Containers/rooting e.g. This would make voice assistant much more accessible for non-technical people, which is the announced strategy of the foundation anyway: Become more appealing to the masses.
  • As far as I understand, the app already acts as a wyoming satelite, when triggered through a button.

I’m admittedly not very familiar with the wyoming protocol and if it could be spoofed from the app - but it seems to be worth the discussion at least?

As a reference: These people have made both STT and TTS run natively in android using the onnxruntime. It even uses all the piper language models and has precompiled apks.

(Maybe further down the line even the intent modelscould run in the app, given that most tablet CPUs are much faster than a Raspberry Pi anyway… Would be amazing from a UX perspective)

84

Wish I could get this working on android fire 8 tablet
Tried the install command a few times I do get ha to see the open wake word auto discovery but it’s unavailable
When typing in the ip from ifconfig with port 10700 ha is unable to connect.
Anything I should look for or do differently?

85

Yes, having the functionality built into the app would be way more elegant und user-friendly, but I barely have any expertise in regards to Android app development, so this is nothing that I could help with unfortunately. I’m sure it’s possible though.

86

What happens if you use port 10400 instead like mclever suggested?

87

If I’m not mistaken when I did that it finds the open wake word.
However I think the mic was also not working.
I did configure view assist and that worked. I’m not sure if having Wyoming running would be less resources then the rspmic app?

88

Thank you so much @randomuser123!! Your script works amazingly on Lenovo Thinsmart View with Lineage (Android version 8).
The only things missing are wget and git packages (as easy as “pkg install wget” and “pkg install git”).
Was looking around a lot for a solution to get both a dashboard and voice assist running on the device and this is by far the best solution I found! I think many people will find this useful

1 Like
89

i realised that i also needed to do apt upgrade before i can install the wget package

90

i have just did a fresh install of the lineage on thinksmart view 8". ran the script without issue. however, it’s not working and i got these error messages when trying to start the satellite, any idea?

~/wyoming-satellite $ ./script/run   --name 'Thinksmart View Satellite'   --uri 'tcp://0.0.0.0:10400'   --mic-command 'rec -r 16000 -c 1 -b 16 -e signed-integer -t raw --no-show-progress - '   --snd-command ' play -r 22050 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -'   --awake-wav ./sounds/awake.wav   --done-wav ./sounds/done.wav   --timer-finished-wav ./sounds/timer_finished.wav   --timer-finished-wav-repeat 5 0.5   --debug
DEBUG:root:Namespace(mic_uri=None, mic_command='rec -r 16000 -c 1 -b 16 -e signed-integer -t raw --no-show-progress - ', mic_command_rate=16000, mic_command_width=2, mic_command_channels=1, mic_command_samples_per_chunk=1024, mic_volume_multiplier=1.0, mic_noise_suppression=0, mic_auto_gain=0, mic_seconds_to_mute_after_awake_wav=0.5, mic_no_mute_during_awake_wav=False, mic_channel_index=None, snd_uri=None, snd_command=' play -r 22050 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -', snd_command_rate=22050, snd_command_width=2, snd_command_channels=1, snd_volume_multiplier=1.0, wake_uri=None, wake_word_name=[], wake_command=None, wake_command_rate=16000, wake_command_width=2, wake_command_channels=1, wake_refractory_seconds=5.0, vad=False, vad_threshold=0.5, vad_trigger_level=1, vad_buffer_seconds=2, vad_wake_word_timeout=5.0, event_uri=None, startup_command=None, detect_command=None, detection_command=None, transcript_command=None, stt_start_command=None, stt_stop_command=None, synthesize_command=None, tts_start_command=None, tts_stop_command=None, tts_played_command=None, streaming_start_command=None, streaming_stop_command=None, error_command=None, connected_command=None, disconnected_command=None, timer_started_command=None, timer_updated_command=None, timer_cancelled_command=None, timer_finished_command=None, awake_wav='./sounds/awake.wav', done_wav='./sounds/done.wav', timer_finished_wav='./sounds/timer_finished.wav', timer_finished_wav_repeat=[5.0, 0.5], uri='tcp://0.0.0.0:10400', name='Thinksmart View Satellite', area=None, no_zeroconf=False, zeroconf_name=None, zeroconf_host=None, debug_recording_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s')
INFO:root:Ready
DEBUG:root:Detected IP: 10.0.2.252
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/data/data/com.termux/files/home/wyoming-satellite/wyoming_satellite/__main__.py", line 490, in <module>
    run()
  File "/data/data/com.termux/files/home/wyoming-satellite/wyoming_satellite/__main__.py", line 484, in run
    asyncio.run(main())
  File "/data/data/com.termux/files/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.12/asyncio/base_events.py", line 686, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/home/wyoming-satellite/wyoming_satellite/__main__.py", line 457, in main
    await register_server(
  File "/data/data/com.termux/files/home/wyoming-satellite/.venv/lib/python3.12/site-packages/wyoming/zeroconf.py", line 35, in register_server
    await aiozc.async_register_service(service_info)
  File "/data/data/com.termux/files/home/wyoming-satellite/.venv/lib/python3.12/site-packages/zeroconf/asyncio.py", line 207, in async_register_service
    return await self.zeroconf.async_register_service(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/home/wyoming-satellite/.venv/lib/python3.12/site-packages/zeroconf/_core.py", line 342, in async_register_service
    await self.async_check_service(info, allow_name_change, cooperating_responders, strict)
  File "/data/data/com.termux/files/home/wyoming-satellite/.venv/lib/python3.12/site-packages/zeroconf/_core.py", line 508, in async_check_service
    raise NonUniqueNameException
zeroconf._exceptions.NonUniqueNameException
Traceback (most recent call last):
  File "/data/data/com.termux/files/home/wyoming-satellite/./script/run", line 12, in <module>
    subprocess.check_call([context.env_exe, "-m", "wyoming_satellite"] + sys.argv[1:])
  File "/data/data/com.termux/files/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/data/data/com.termux/files/home/wyoming-satellite/.venv/bin/python3', '-m', 'wyoming_satellite', '--name', 'Thinksmart View Satellite', '--uri', 'tcp://0.0.0.0:10400', '--mic-command', 'rec -r 16000 -c 1 -b 16 -e signed-integer -t raw --no-show-progress - ', '--snd-command', ' play -r 22050 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -', '--awake-wav', './sounds/awake.wav', '--done-wav', './sounds/done.wav', '--timer-finished-wav', './sounds/timer_finished.wav', '--timer-finished-wav-repeat', '5', '0.5', '--debug']' returned non-zero exit status 1.
91

This is great! Got it working on a TSV. One question tho. How do you change the wake word when running openwakeword? I presume i can create some startup script and pass in my preferred wake word (alexa currently). I just dont know how or where to do that.

92

NonUniqueNameException sounds like you configured multiple devices with the same name, is this the case? I set up 2 TSVs and manually changed the name of the second one in the boot script

93

You can set it up in the boot script which relative to the home directory (where termux starts) is in .termux/boot/wyoming-satellite (or something like that). There are 2 occurences of ok_nabu, replace both with your preferred wakeword.

1 Like
94

Thanks @eliasm . I got it working. Would be great if that was an argument or prompt during the install script for that to get configured.

1 Like
95

Thanks, im not sure why, this morning it just started working. maybe it needed a reboot somehow.
Appreciate the reply, cheers

1 Like
96

I’ve read through this post and i noticed a few folks have gotten this working on old fire tablets. I am struggling with that though. It gets to the end and then closes without keeping the commands running.

Starting it now...
Wed Dec 11 15:47:21 CST 2024
Ensure sox is available...
Enter wyoming-openwakeword directory...
Enter wyoming-satellite directory...
Ensure module-sles-source is loaded...
17
DEBUG:root:Namespace(uri='tcp://0.0.0.0:10400', models_dir=PosixPath('/data/data/com.termux/files/home/wyoming-openwakeword/wyoming_openwakeword/models'), custom_model_dir=[], preload_model=['ok_nabu'], threshold=0.5, trigger_level=1, output_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s', debug_probability=False, version=False, model=[])
DEBUG:root:Loading ok_nabu_v0.1 from /data/data/com.termux/files/home/wyoming-openwakeword/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
DEBUG:root:Loading /data/data/com.termux/files/home/wyoming-openwakeword/wyoming_openwakeword/models/melspectrogram.tflite
DEBUG:root:Loading /data/data/com.termux/files/home/wyoming-openwakeword/wyoming_openwakeword/models/embedding_model.tflite
INFO:root:Ready
INFO: Initialized TensorFlow Lite runtime.
DEBUG:root:Namespace(mic_uri=None, mic_command='rec -r 16000 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -', mic_command_rate=16000, mic_command_width=2, mic_command_channels=1, mic_command_samples_per_chunk=1024, mic_volume_multiplier=1.0, mic_noise_suppression=0, mic_auto_gain=0, mic_seconds_to_mute_after_awake_wav=0.5, mic_no_mute_during_awake_wav=False, mic_channel_index=None, snd_uri=None, snd_command='play -r 22050 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -', snd_command_rate=22050, snd_command_width=2, snd_command_channels=1, snd_volume_multiplier=1.0, wake_uri='tcp://127.0.0.1:10400', wake_word_name=[['ok_nabu']], wake_command=None, wake_command_rate=16000, wake_command_width=2, wake_command_channels=1, wake_refractory_seconds=5.0, vad=False, vad_threshold=0.5, vad_trigger_level=1, vad_buffer_seconds=2, vad_wake_word_timeout=5.0, event_uri=None, startup_command=None, detect_command=None, detection_command=None, transcript_command=None, stt_start_command=None, stt_stop_command=None, synthesize_command=None, tts_start_command=None, tts_stop_command=None, tts_played_command=None, streaming_start_command=None, streaming_stop_command=None, error_command=None, connected_command=None, disconnected_command=None, timer_started_command=None, timer_updated_command=None, timer_cancelled_command=None, timer_finished_command=None, awake_wav='./sounds/awake.wav', done_wav='./sounds/done.wav', timer_finished_wav='./sounds/timer_finished.wav', timer_finished_wav_repeat=[5.0, 0.5], uri='tcp://0.0.0.0:10700', name='Android Satellite - Amazon KFTRPWI', area=None, no_zeroconf=False, zeroconf_name=None, zeroconf_host=None, debug_recording_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s')
INFO:root:Ready
DEBUG:root:Detected IP: 192.168.1.197
DEBUG:root:Zeroconf discovery enabled (name=930e0d071f44, host=None)
DEBUG:root:Connecting to mic service: ['rec', '-r', '16000', '-c', '1', '-b', '16', '-e', 'signed-integer', '-t', 'raw', '--no-show-progress', '-']
DEBUG:root:Connecting to snd service: ['play', '-r', '22050', '-c', '1', '-b', '16', '-e', 'signed-integer', '-t', 'raw', '--no-show-progress', '-']
DEBUG:root:Connecting to wake service: tcp://127.0.0.1:10400
INFO:root:Connected to services
Traceback (most recent call last):
  File "/data/data/com.termux/files/home/wyoming-satellite/./script/run", line 12, in <module>
    subprocess.check_call([context.env_exe, "-m", "wyoming_satellite"] + sys.argv[1:])
  File "/data/data/com.termux/files/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/data/data/com.termux/files/home/wyoming-satellite/.venv/bin/python3', '-m', 'wyoming_satellite', '--name', 'Android Satellite - Amazon KFTRPWI', '--uri', 'tcp://0.0.0.0:10700', '--mic-command', 'rec -r 16000 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -', '--snd-command', 'play -r 22050 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -', '--awake-wav', './sounds/awake.wav', '--done-wav', './sounds/done.wav', '--timer-finished-wav', './sounds/timer_finished.wav', '--timer-finished-wav-repeat', '5', '0.5', '--wake-uri', 'tcp://127.0.0.1:10400', '--wake-word-name', 'ok_nabu', '--debug']' died with <Signals.SIGSYS: 31>.
~ $
97

is there a way to enable it as a media_player? my household uses LMS (squeezelite) and i believe there’s a squeezelite that uses pulseaudio.

if you are snapcast user, i think you could enhance this following this guide.wyoming-enhancements/snapcast/docs/3_install_snapclient.md at 2d10690022ec54aa64d3de77d95324de7b0116f3 · FutureProofHomes/wyoming-enhancements · GitHub

98

Im gonna give this a shot on an old fire tablet. Will report back. One thing to note is that you have to chmod +x wyoming-microwake-word.sh in order for its to be executable.

99

So this runs fine on my fire hd 10 tablet (HA auto discovered it) but i dont think the microphone is being used as i dont see it detecting anything. I forked it and tried to pass in a different model: udocker_run --entrypoint "bash -c" -p "$PORT:10400" "$CONTAINER_NAME" "--preload-model 'alexa'" but still no luck. Id be great to get a container that has both mww and ws on it to mimic what we have in the OPs script. Anyone have any luck with that?

100

Tried to get this working on an old Samsung tablet, but getting the died with <Signals.SIGSYS: 31> error from wyoming satellite. Openwakeword seems to run OK.

Galaxy Tab A
Android 8.1.0
Kernel 3.18.14

Cheers.

101

The SIGSYS/31 error that you and @janstadt appear to have is a known issue How to: Run Wyoming Satellite and OpenWakeWord on Android - #82 by justicefreed

As far as I remember @wizhack had that same error and was able to fix it by doing this:

I don’t really know why this works and if this would have any side effects. But I’m sure someone with Python experience could enlighten us. My intention with this code actually just was to print a Python traceback. But I guess it catches the SIGSYS signal in a way that prevents the original error from getting thrown.

If someone with actual Python experience would say that this is a reasonable way to fix this, then I could put that into the install script to make it more accessible.

1 Like