Is this the perfect standalone tablet for HA?

Cactus_J · January 31, 2024, 1:10am

Semi Off Topic, but one thing I can’t figure out is how to actually PLAY music FROM the tablet. Not cast, but play.

The tablet connects via Bluetooth just fine, and if I load the Plex App, i can play music perfectly. But there seems to be no actual way to play the Plex library from inside Home Assistant.

There is a Plex Meets HA integration, but it requires your to cast to a plex play. it cant be the Plex Player itself. I’d like the ability to just play from the tablet, and not have to open multiple apps…

O well, weird… but maybe this is just me…

jaburges · January 31, 2024, 4:56pm

So I’ve successfully set up openwakeword with VAD and Wyoming protocol using a Pi zero W 2 and reespeaker Pi Hat (with mic). The guide was excellent.

So in theory if there was an app that could run and be listening the whole time and offload to openwakeword, if a pi zero W can do it - can this device not do it?

Endlessvoid · January 31, 2024, 7:20pm

You want an app that just streams an open mic to open wake word?

Looks like there are a few apps that might fit the bill, LanMic pops up as a possibility: https://play.google.com/store/apps/details?id=com.portable.lanmic

Interesting idea, I’m not familiar enough with Wyoming and open wake word to try it out, but seems like it could work in theory.

Side note, there are photos of this device running android 11 from the developer who cracked the device to start with. Waiting on more details of how he did it, but if that’s possible then using a 3rd party wake word and the companion app in conjunction becomes possible.

jaburges · January 31, 2024, 8:00pm

sweeeet - I LOVE this community

Here was the guide for the pi zero w 2 (i think the goal is to figure out x86 so it can also be run on a pi zero w 1.1)

github.com

rhasspy/wyoming-satellite/blob/8850100ca658a80708e562e6741739ed27c91f99/docs/tutorial_2mic.md

# Tutorial with 2mic HAT

Create a voice satellite using a [Raspberry Pi Zero 2 W](https://www.raspberrypi.com/products/raspberry-pi-zero-2-w/) and a [ReSpeaker 2Mic HAT](https://wiki.keyestudio.com/Ks0314_keyestudio_ReSpeaker_2-Mic_Pi_HAT_V1.0).

This tutorial should work for almost any Raspberry Pi and USB microphone. Audio enhancements and local wake word detection may require a 64-bit operating system, however.

## Install OS

Follow instructions to [install Raspberry Pi OS](https://www.raspberrypi.com/software/). Under "Choose OS", pick "Raspberry Pi OS (other)" and "Raspberry Pi OS (Legacy, **64-bit**) Lite".

When asking if you'd like to apply customization settings, choose "Edit Settings" and:

* Set a username/password
* Configure the wireless LAN
* Under the Services tab, enable SSH and use password authentication

## Install Software

After flashing and booting the satellite, connect to it over SSH using the username/password you configured during flashing.

This file has been truncated. show original

Re the LAN mic, if that is possible you can technically send ALL audio to the Home Assistant instance, and run openwakeword on that. The neat thing about the pi zero w 2 config as a “Wyoming-satellite” is you can run openwakeword on the actual pi so it only sends audio chunks after the wake word is detected.

Then the other services (LLM, Piper & Whisper) are running on the HA server.

@synesthesiam sorry to cold mention you, but how hard would it be to create an android app that could run like a wyoming-satellite? (ideally with VAD and openwakeword on the device?)

Endlessvoid · February 1, 2024, 4:38am

He’s already working on it according to his feedback on the wakeword thread back in Oct, not sure if that effort has stalled or maybe just focusing on other things.

Endlessvoid · February 1, 2024, 4:48am

Just putting a general progress update in here. Firstly, huge thank you to Deadman96385 for his help getting this far, with a few tips from him I have successfully replicated him by flashing a GSI rom of Android 11 on this device. My steps were as follows:

Flash the manufacturers debug firmware using QFIL or whatever EDL flash tool you prefer. Link to Deadmans original post with various firmwares, I had to use the debug one as none of the other versions would work for me: CD-18781Y (Lenovo ThinkSmart View) Bootloader, Firmware, Zoom/Teams Conversion, Normal Android | XDA Forums
Boot the device up and get into the system settings (See the instructions in the previous link for how to escape the teams setup and access normal settings), enable developer options and check that OEM unlocking is enabled.
Reboot to the bootloader/fastboot
Using a recent copy of android platform tools (I used 34.0.5), use fastboot to unlock the bootloader “fastboot oem unlock-go”, make sure you hold volume down as you enter this command as the device will immediately reboot and we need it to go directly back into fastboot mode.
Flash a GSI image that will fit in the system partition, see this XDA comment by Deadman for some that worked and some that didnt: [GUIDE][ROOT] Unlocking and Improving Your Lenovo ThinkSmart View (CD-18781Y) | Page 2 | XDA Forums
Here’s a link to the GSI repository: Releases · phhusson/treble_experimentations · GitHub
(optional) Flash the vendor package Deadman created to fix wifi issues (note that this did not work for me). See the above link in step 5 for the vendor package.
Relock the bootloader with “fastboot flashing lock”
The device will reboot, give it about a minute to start up, then it should be good to go.

Based on some others feedback it seems like YMMV depending on the manufacture date of the device, fingers crossed it’ll work fine for you.

I’m still working on getting wifi working well, I was able to get it to work by manually pushing the stock wlan firmware into the system partition with ADB, but it was pretty unstable and buggy, still working on this part. I’ll give it another try tomorrow.

nwduncan · February 1, 2024, 11:32am

I haven’t seen this mentioned elsewhere in the thread (maybe I’m an outlier, or did something wrong, or just thick), but for anyone who may be stuck running the adb commands I had to turn on PTP on the device for this to work.

Settings > Connected devices > USB > PTP

Thanks to @mattmon for putting this together, and others for the help.

El_Pollo_Loco · February 1, 2024, 2:43pm

Thanks for this summary, I was running into issues flashing the GSI last night. Fastboot would successfully flash all imgs but I could not boot after locking the bootloader. I was flashing a tablet I had already loaded with the ROM from the OP in the thread, so I will try again tonight with the debug firmware you linked.

mattmon · February 1, 2024, 9:32pm

Thanks for taking the time to figure this out. Can you verify that the audio and screen wake issues are resolved?

Kingston-ha-rom is also based upon the ‘debug’ rom. So it should be possible to directly upgrade for those who have already flashed.

I guess I’ll pivot from fixing up the Oreo ROM to updating OP with the GSI image and @deadman96385’s fixes.

Will make it flashable via EDL so everyone can avoid dealing with the boot loader’s insecure security.

synesthesiam · February 1, 2024, 9:33pm

We’d like to add local wake word and VAD to the HA Android app, but we currently don’t have a full-time developer to work on it.
I know it is technically possible to run openWakeWord and silero VAD on Android, as both can run under the onnx runtime (which can be built for Android). But actually doing this is something I’ve never tried

Endlessvoid · February 1, 2024, 10:05pm

I started from the Kingston-ha-rom and didn’t have any luck, interesting to know it’s based on the debug rom, it seems like there’s something that’s giving everyone a slightly different result when flashing GSI roms, wish I knew enough to track it down.

mattmon · February 2, 2024, 12:05am

I’ve managed to make the gsi work. It’s got some issues:

wifi firmware (as you mentioned)
will not wake from suspend
boot up takes for…ev…er…

Otherwise it does feel pretty snappy.

Also can’t seem to successfully dump the emmc and then flash into another unit though, it just hangs at the first stage splash.

Something is happening during init, but I don’t know what.

Probably, the right way to solve this is to fork the treble repo and build it with all our fixes already inside system.img.

brooksben11 · February 2, 2024, 12:56am

So I’ve been playing with using Fully Kiosk Browser integrated with HA to make the tablet show up as a media player and while it works, I have a repeatable issue where it stops playing audio clips roughly around the 1 minute mark; the audio just stops. Are other folks seeing this?

Just sat down with 2 identical length tracks (1:17 in length each) and confirmed that it stops at the exact same point in each track every time I play the same track, but it’s a different length of time for each (one is almost exactly a minute, the other it makes it about 6 seconds longer). Made me wonder if it’s a data issue, but the inverse seems to be true; it makes it further along the SLIGHTLY larger file.

Also, it perpetually shows the media player and ‘playing’ even after a track has completed. Not a big deal, but still kind of annoying. Is this a general FKB issue?

I will also mention that while I have ‘Fully Remote Admin’ enabled in FKB to allow the HA integration, I’m still using the free version while I test it out (so it has a watermark).

Endlessvoid · February 4, 2024, 5:14am

With the help of deadman and others to get Android 11 working, I just wanted to follow up with a quick video of this device set up with a working voice assistant:

mrhand · February 4, 2024, 6:49am

Nice! Before we get to the voice assistant part, what did you do to fix the wifi?

Endlessvoid · February 4, 2024, 7:21am

Steal these wifi firmware files from the debug roms system partition and after flashing the GSI, push them back onto the system partition with ADB:
system/etc/firmware/ice40.bin
system/etc/firmware/wlan/qca_cld/WCNSS_qcom_cfg.ini
system/etc/firmware/wlan/qca_cld/wlan_mac.bin

The last one might not be needed, it gave me some trouble pulling with adb so I left it off once and it still worked

mattmon · February 4, 2024, 2:24pm

Anybody figured out the suspend issue?

yabdali · February 4, 2024, 5:41pm

Does it run smoothly without problems? I order 3 units and I would like to get the GSI rom flashed…

mattmon · February 5, 2024, 1:54am

GSI should be considered experimental.

It works, but may have issues.

I hope that at some point in the near future we have an official gsi "upgrade’ for kingston-ha-rom.

Endlessvoid · February 5, 2024, 4:51am

There are a few bugs on the Android 11 GSI rom, the most unpleasant is the screen timeout, it wont turn back on without an adb command or a usb cable being plugged in. There are also no A11 or A10 roms with gapps that have been found to fit on the system partition yet, so if you need your paid apps from the play store you’ll have to roll your solution with microg/magisk or something along those lines.

The base firmware on android 8 is pretty rock solid, just don’t expect to get a voice assistant on it.

Edit to say that there is at least 1 dev (deadman, he popped in this thread earlier) actively working on a single GSI image with fixes for most of the issues right now.