Thoughts/guesses Lenovo Smart Display with Google Assistant

Can I ask for everyone who reads this thread to help me get a response from the Google team?

If you also want to open Home Assistant on a Google Home Smart display can you also add a comment to this thread saying you want custom URLs such as home assistant to work on the smart display.

Googlers tell me they will respond to people power, but atm I’m the only one asking!

Please support me here https://support.google.com/assistant/thread/232545?msgid=232545 9

Looks like I’m the only one trying to get Home Assistant to work on the Google Smart Display. Hopefully persistence pays off.

Google isn’t responding and Dialogflow isn’t possible either. The only path seems to Spoof a food recipe so that home assistant will show up on the smart display (pretending to be a food recipe). According to this reviewer the browser seems only to work for food recipes https://variety.com/2018/digital/news/lenovo-smart-display-1202886406/

So my plan is as below. Let me know if you’re interested and I’ll write up the detailed steps on what I do to get this working.

  1. Identify a food recipe on the smart display to Spoof and get its full Web address.
  2. Install and configure Dnsmasq on Hassio https://www.home-assistant.io/addons/dnsmasq/
  3. Create a HTML webpage to reroute traffic back to the home assistant URL, use DNSMASQ as my dns server
  4. Configure my home router to use DNSMASQ and the raspberry pi as my primary dns server.
  5. HOPEFULLY, when I open the spoofed recipe on the smart display, The home assistant UI will show up instead.
  6. If all’s working I’ll create a google Routine/Shortcut to open the spoofed recipe with “ok google, open home assistant”.
1 Like

Would love to hear how this goes

I’ve just setup my webserver, however I now see an issue with my approach.

All the recipe pages that the Smart Display links to are signed HTTPS locations. So spoofing the URL will work but i can’t spoof the certificate… I think this may not work, but I’ll keep trying.

As for the Smart Display, there’s no onscreen keyboard for input. So when navigating a recipe you can click through links etc… but if you click on a search dialog box there’s no ability to enter any values.

Google are very much wanting the smart display to be a “voice forward” device (i.e. you use your voice as the primary input, as compared to a tablet where it’s a “touch forward” device). Google Smart Displays do not currently have an onscreen keyboard, so it begins to make sense why they don’t allow you to open URLs.

But still… I just want to use my Google Smart Display to access Home Assistant! Must this be so hard!!!

Conclusion as at 9th Sept 2018:

  • You can’t open a URL on the Google Smart Display (only food recipes are allowed). This could be by design as the Google Smart Display is a “voice forward” device and does not have an onscreen keyboard
  • Trying to expose a URL on the Smart Display with dialog-flow does not work (URL suggestion chips do not appear)
  • Trying to spoof a recipe page with DNS will not work. All the recipes accessed on the Google Smart Display are HTTPS with signed certificates.

The only way to open a custom URL is to be the owner of one of the signed recipe pages and put a hyperlink on that page which will have embedded details of your HASS login and password (as there is no keyboard input… seems like we have to pester Google to make this capability available.

just got one of these, and trying to figure out how to cast my ring doorbell camera when the doorbell is rung (also testing with an nvidia shield).

watching what happens to the thread (happy to test things too)

Just bought one of these bad boys, excited to integrate it with Home Assistant. I know it will take an ungodly amount of time to get the home assistant UI on here but I’m sure it will be worth it. Three initial questions

  1. You can cast to this thing, right? What happens if you cast a browser tab with your home assistant UI pulled up? Can you interact with it or is it view only?

  2. I know you don’t have routines in Australia, but if you did-- do you think this would solve the problem? Can you interact with other websites that are launched on the screen?

  3. If both of these approaches fail, would creating an “Android Things” app (just a webview) potentially work?

I guess I’m trying to suss out why Google is trying so hard to disallow website interactions on this device.

All 3 of the options you’ve listed I’ve tried and failed. The last one, even though it’s an android things device you can install any custom software to it. You can however build a “google actions” app but the tool set provided for Google Actions will neither let you build your own browser or use the inbuilt one.

My guess is that Google wants the UI to be a Voice Forward device rather than a tablet which is a Touch forward device.

Hopefully Google adds this capability in the future. Recent leaks point to a new HOME HUB tablet attached to a google home speaker dock. This suggests to me touch controls and even an IoT touch based UI with widgets for home control is in the horizon.

Can you “cast” to these devices from Home Assistant?

Would any of you be able to do a system dump, or is that possible with these devices. I’m trying to build a raspberry pi driven Android Things smart display. Any thoughts or suggestions welcome.

I’ve been watching Google’s smart display, and it seems they didn’t use Android Things on their own display.

I know this is not a solution for Google users but the new Amazon Echo Show Gen 2 (or an upgraded G1) can be used with HA as it now includes both a FireFox and Silk Browser. It appears to work well although the browser does time out after 10 mins and revert to the home screen. Haven’t found a workaround for that.

Is there any way to integrate camera feeds (from Home Assistant)

What I don’t understand about these new smart displays is how they can compete with a simple android tablet? These smart displays need a power supply vs. tablet has battery, they are fairly UI locked vs. tablet can do most anything, and they both support Google Assistant… Id rather a cheap tablet and have the flexibility and portability…

The Google Assistant is aiming to be an omnipresent computer interface much like what we see in science fiction space ships. With plugged in device you can be confident that when you’re in the same room it’ll work the same way every time. With a tablet, it being mobile breaks that experience slightly. Google wants you to be confident that everytime you’re in that room the Google Assistant will be ready to respond consistently each time.

It’s not trying to compete with tablets, and it’s deliberate that google doesn’t put the play store on these devices.

theoretically from a HW point of view: better microphones, better speakers

And from a software point of view a OS with: room awareness, voice activated apps only, IOT focused … and so on

Its a different device class from a portable tablet

yes you can - a little convoluted as cast uses an m3u stream.
Here is a thread:

and here it the git:

i ran all my cameras through the docker (which consumed almost no compute, surprisingly, on my NAS) - then could create automations (well you get it)… it did take a few seconds to come up though.

the problem seems to be the type of stream

there was a conversation by the dev team about the current video implementation and a desire to change it, but i imagine it falls WAYYYY down the list of other priorities right now.

The much-awaited display from Lenovo is finally here well know that and i was and still curious to purchase this device. But along with this display, we also get other good options too, like Amazon’s Echo Show 2nd Gen LG’s Smart display and many others. I was searching for few best smart displays and found a source saying that Echo’s 2nd gen is the best value for money so i am still confused that which one i should get.

I was thinking of putting a raspi behind a Dell 24 Touch Monitor P2418HT. No linux drivers though.

What about using an Intel Compute Stick in place of the rPi? A bit more expensive but since it runs Windows, drivers shouldn’t be an issue. Just a thought.

1 Like

Has anyone managed to get any further with this? The Lenovo ZA4R0040AU Smart Clock is quite cheap now.

1 Like