Here’s a video on what I call ‘View Assist’. This is a visual feedback device tied into the Assist voice assistant. The project goal was to use cheap/existing hardware to replicate the functionality of the Amazon Echo Show devices we use in our home.
The proof-of-concept was created using a 2016 Amazon Fire7 Android tablet running Android 5 (!!!) that had been sitting in a drawer. The device feeds its audio via an Android webcam app to Stream Assist which passes the audio over to the HA server for wake word detect. You’ll notice in the video that the ‘listening bar’ is a bit delayed in appearing but the actual listening happens before the visual appears. Display control is via the ‘Browser Mod’ extension which allows for changing the view being displayed on the tablet. I am also running Fully Kiosk Android app for the full screen display. Fully Kiosk also exposes the tablet as a media player device. While I used an old tablet, I’m confident that any Android device capable of running the webcam app and Fully Kiosk can be used for a similar project both for visual and audio output or audio only if you choose. A great way to repurpose ejunk!
From here, all commands are being handled by the ‘custom sentences’ via HA automation page. This allows for making specialty sentences which call a wide range of services. I am able to tie into things like my chores list, shopping list, Wikipedia, launch Waze maps, send broadcast messages to other media devices, play music from Music Assistant, etc. You are really only limited by your imagination. Unfortunately I am limited to only custom sentences for actions as the underlying service calls for Assist are not yet exposed so I can’t detect when those core sentences (eg turn on light) happen so I can’t create a view that shows the entity that just turned on. Hopefully this will be exposed as well as the assist device calling it so that I can better tailor the display to only the device being used for the voice command. This will also be useful for extending this to audio only satellite voice assistants as well.
The display portion is running a single dashboard with one view per display screen (eg clock). I will admit that I knew next to nothing in regards to CSS and still know next to nothing about graphical layouts and colors. You’ll see that what I’ve done leans heavy on the Amazon look of the Echo Show devices I’m looking to replace.
I could not have done any of this without the support of the HA community both here on the forum as well as on Discord. I know I tried a lot of people’s patience with my questions but they were all extremely helpful in getting me this far. I can only imagine what this could look like in capable hands.
At any rate, if anyone is interested I’ll gladly share the details of how I did this in the ‘Share my project’ section of the forum. My hope is that this might inspire folks to expand on this idea and to do it the ‘right way’.
Thanks for watching. If you like it please click the Vote button next to the title of the post at the top.
EDIT I’ve started to document this and have a bit on the Wiki page. The majority of the information has been provided and I will work to add supplemental information and examples.
