How to: Run Wyoming Satellite and OpenWakeWord on Android

Share your Projects!
, ,
82

I believe I have found the issue with the SIGSYS:31 error.

After investigating the stack traces, the error occurs on a syscall to pidfd_open aka syscall __NR_pidfd_open. It appears that the Android kernal’s SECCOMP is denying the syscall due to pidfd_open not being on the SECCOMP whitelist for older versions of Android (mine is v8.1.0, kernel 3.18.14).

Per discussion on how GO dealt with this, it appears that this syscall was properly added to the seccomp allow list in Android 12, so the issue would theoretically affect devices running Android 11 or lower.

My own testing has shown that the kernel DOES expose a pidfd_open syscall, but because Android improperly handles it in the whitelist (it should error with a SECCOMP_RET_TRAP or similar but instead errors with a SECCOMP_RET_KILL.

The relevant syscall is actually performed by the underlying python asyncio library, ironically during the very check to see if it should avoid using pidfd…:man_facepalming:. This appears to be getting called the first time that asyncio.create_subprocess_exec(...) is called, which in this case is when the satellite calls the wyoming library’s mic connect function.

My guess is that (at least part of) the reason @1Drew was able to make their workaround work is that the ubuntu distro allows the code to make that check without hitting the actual Android system SECCOMP check.

With all that in mind, I think so far the workarounds may be:

  • Root the android device and edit the SECCOMP config to reply with an error that doesn’t also kill the calling process (haven’t investigate how easy this is yet)
  • Run the code in a containerized fashion somehow such that this check doesn’t get made against the actual Android kernel.
  • Upgrade the Android version to 12+ (for old devices, this is not possible and thus would have to try rooting and installing something like lineageOS)
1 Like
83

Fantastic and exciting work everyone!

Given that so many people (including myself) use android tablets, which are mostly always on and have the Home Assistant Companion app in the foreground -
wouldn’t it be somewhat easier to add the keyword detection functionality natively into the android app?

Maybe this is a naive view, but in my eyes, it would have several benefits:

  • Easier microphone access (the app already has the permission)
  • TFlite, which the wake word detection is based on, was originally developed to be embedded in android apps anyway and has been done thousands of times before.
  • No need to mess around with Termux/Containers/rooting e.g. This would make voice assistant much more accessible for non-technical people, which is the announced strategy of the foundation anyway: Become more appealing to the masses.
  • As far as I understand, the app already acts as a wyoming satelite, when triggered through a button.

I’m admittedly not very familiar with the wyoming protocol and if it could be spoofed from the app - but it seems to be worth the discussion at least?

As a reference: These people have made both STT and TTS run natively in android using the onnxruntime. It even uses all the piper language models and has precompiled apks.

(Maybe further down the line even the intent modelscould run in the app, given that most tablet CPUs are much faster than a Raspberry Pi anyway… Would be amazing from a UX perspective)

84

Wish I could get this working on android fire 8 tablet
Tried the install command a few times I do get ha to see the open wake word auto discovery but it’s unavailable
When typing in the ip from ifconfig with port 10700 ha is unable to connect.
Anything I should look for or do differently?

85

Yes, having the functionality built into the app would be way more elegant und user-friendly, but I barely have any expertise in regards to Android app development, so this is nothing that I could help with unfortunately. I’m sure it’s possible though.

86

What happens if you use port 10400 instead like mclever suggested?

87

If I’m not mistaken when I did that it finds the open wake word.
However I think the mic was also not working.
I did configure view assist and that worked. I’m not sure if having Wyoming running would be less resources then the rspmic app?

88

Thank you so much @randomuser123!! Your script works amazingly on Lenovo Thinsmart View with Lineage (Android version 8).
The only things missing are wget and git packages (as easy as “pkg install wget” and “pkg install git”).
Was looking around a lot for a solution to get both a dashboard and voice assist running on the device and this is by far the best solution I found! I think many people will find this useful

1 Like
89

i realised that i also needed to do apt upgrade before i can install the wget package

90

i have just did a fresh install of the lineage on thinksmart view 8". ran the script without issue. however, it’s not working and i got these error messages when trying to start the satellite, any idea?

~/wyoming-satellite $ ./script/run   --name 'Thinksmart View Satellite'   --uri 'tcp://0.0.0.0:10400'   --mic-command 'rec -r 16000 -c 1 -b 16 -e signed-integer -t raw --no-show-progress - '   --snd-command ' play -r 22050 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -'   --awake-wav ./sounds/awake.wav   --done-wav ./sounds/done.wav   --timer-finished-wav ./sounds/timer_finished.wav   --timer-finished-wav-repeat 5 0.5   --debug
DEBUG:root:Namespace(mic_uri=None, mic_command='rec -r 16000 -c 1 -b 16 -e signed-integer -t raw --no-show-progress - ', mic_command_rate=16000, mic_command_width=2, mic_command_channels=1, mic_command_samples_per_chunk=1024, mic_volume_multiplier=1.0, mic_noise_suppression=0, mic_auto_gain=0, mic_seconds_to_mute_after_awake_wav=0.5, mic_no_mute_during_awake_wav=False, mic_channel_index=None, snd_uri=None, snd_command=' play -r 22050 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -', snd_command_rate=22050, snd_command_width=2, snd_command_channels=1, snd_volume_multiplier=1.0, wake_uri=None, wake_word_name=[], wake_command=None, wake_command_rate=16000, wake_command_width=2, wake_command_channels=1, wake_refractory_seconds=5.0, vad=False, vad_threshold=0.5, vad_trigger_level=1, vad_buffer_seconds=2, vad_wake_word_timeout=5.0, event_uri=None, startup_command=None, detect_command=None, detection_command=None, transcript_command=None, stt_start_command=None, stt_stop_command=None, synthesize_command=None, tts_start_command=None, tts_stop_command=None, tts_played_command=None, streaming_start_command=None, streaming_stop_command=None, error_command=None, connected_command=None, disconnected_command=None, timer_started_command=None, timer_updated_command=None, timer_cancelled_command=None, timer_finished_command=None, awake_wav='./sounds/awake.wav', done_wav='./sounds/done.wav', timer_finished_wav='./sounds/timer_finished.wav', timer_finished_wav_repeat=[5.0, 0.5], uri='tcp://0.0.0.0:10400', name='Thinksmart View Satellite', area=None, no_zeroconf=False, zeroconf_name=None, zeroconf_host=None, debug_recording_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s')
INFO:root:Ready
DEBUG:root:Detected IP: 10.0.2.252
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/data/data/com.termux/files/home/wyoming-satellite/wyoming_satellite/__main__.py", line 490, in <module>
    run()
  File "/data/data/com.termux/files/home/wyoming-satellite/wyoming_satellite/__main__.py", line 484, in run
    asyncio.run(main())
  File "/data/data/com.termux/files/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.12/asyncio/base_events.py", line 686, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/home/wyoming-satellite/wyoming_satellite/__main__.py", line 457, in main
    await register_server(
  File "/data/data/com.termux/files/home/wyoming-satellite/.venv/lib/python3.12/site-packages/wyoming/zeroconf.py", line 35, in register_server
    await aiozc.async_register_service(service_info)
  File "/data/data/com.termux/files/home/wyoming-satellite/.venv/lib/python3.12/site-packages/zeroconf/asyncio.py", line 207, in async_register_service
    return await self.zeroconf.async_register_service(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/home/wyoming-satellite/.venv/lib/python3.12/site-packages/zeroconf/_core.py", line 342, in async_register_service
    await self.async_check_service(info, allow_name_change, cooperating_responders, strict)
  File "/data/data/com.termux/files/home/wyoming-satellite/.venv/lib/python3.12/site-packages/zeroconf/_core.py", line 508, in async_check_service
    raise NonUniqueNameException
zeroconf._exceptions.NonUniqueNameException
Traceback (most recent call last):
  File "/data/data/com.termux/files/home/wyoming-satellite/./script/run", line 12, in <module>
    subprocess.check_call([context.env_exe, "-m", "wyoming_satellite"] + sys.argv[1:])
  File "/data/data/com.termux/files/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/data/data/com.termux/files/home/wyoming-satellite/.venv/bin/python3', '-m', 'wyoming_satellite', '--name', 'Thinksmart View Satellite', '--uri', 'tcp://0.0.0.0:10400', '--mic-command', 'rec -r 16000 -c 1 -b 16 -e signed-integer -t raw --no-show-progress - ', '--snd-command', ' play -r 22050 -c 1 -b 16 -e signed-integer -t raw --no-show-progress -', '--awake-wav', './sounds/awake.wav', '--done-wav', './sounds/done.wav', '--timer-finished-wav', './sounds/timer_finished.wav', '--timer-finished-wav-repeat', '5', '0.5', '--debug']' returned non-zero exit status 1.
91

This is great! Got it working on a TSV. One question tho. How do you change the wake word when running openwakeword? I presume i can create some startup script and pass in my preferred wake word (alexa currently). I just dont know how or where to do that.

92

NonUniqueNameException sounds like you configured multiple devices with the same name, is this the case? I set up 2 TSVs and manually changed the name of the second one in the boot script

93

You can set it up in the boot script which relative to the home directory (where termux starts) is in .termux/boot/wyoming-satellite (or something like that). There are 2 occurences of ok_nabu, replace both with your preferred wakeword.

1 Like
94

Thanks @eliasm . I got it working. Would be great if that was an argument or prompt during the install script for that to get configured.

95

Thanks, im not sure why, this morning it just started working. maybe it needed a reboot somehow.
Appreciate the reply, cheers