Google Assistant Local SDK - Local Fulfillment

ok this makes some sense, I assume NAT loopback on a router doesn’t make a difference here?

No Unfortunately. Personally I want local HTTPS and wasn’t happy having to expose a non-HTTPS port internally and move to a proxy. The way it is written, it grabs the MDNS name and attempts connections from the home device to the MDNS name. In theory, you could have proper certs tied to the MDNS entries and then it would connect fine but the code would need to be rewritten to 1) allow it again (its specifically disabled now) and 2) to use a specific url as a config entry rather than use the baseUrl/external/internal to connect to and I’m sure other things.

1 Like

This is more in line with what you were looking for.

1 Like

That was indeed a heavy discussion :slight_smile:

1 Like

Yes and he’s not incorrect (both aren’t incorrect). In the current code and the way mDNS is derived, it can’t function. I really hope at some point enhancements are made to this to allow it to function securely. There should never be a reason to disable security. But its a pretty complicated fix.

3 Likes

Nice, I got an answer from @balloob which confirm query intents are made on the cloud.
So lets wait for Google :slight_smile: and hop they will review soon and approve.

Actually quite frustrating, we started troubleshooting more then a month ago, and now the feedback is that it still not activated :slight_smile:

More precisely, the issue is everything is not activated:

  • It is activated for execute intent as switch bedroom light on or activate night scene
  • It is not activated for query intent as is bedroom light on or what is the temperature in the living room.

If I remind well, your issue was it was not working at all, which stays strange as it should have worked at least for instructions I wrote above :confused:

i think with the manual integration , everything is working, its the NC edition that doesnt work

my state querys and also onoff commands are both local, using the manual GA integrtaion
Before when i used NC, i never got it too work

I seem to be having issues getting this to work and I believe it’s because I’m using Adguard Home which is screwing up mDNS stuff. For instance, I cannot use http://hassio.local/ or http://homeassistant.local/ to access my instance. I don’t really know enough to debug this though. Anyone else use Adguard Home (+ NGINX proxy manager) and have local fulfillment working?

EDIT: AdGuard seems to be unrelated. After disabling it, I still run into DEVICE_VERIFICATION_FAILED. I have added Name: .*\._home-assistant\._tcp\.local, updated the app.js to the pending pull request one, and checked Support local query. I’m using NGINX Proxy manager and can access HA through http://ha-ip:8123 and https://doman.duckdns.org. I’ve also been restarting my google homes and home assistant…

Think it’s time to give up :slight_smile:

I have the verification error too… I deleted the name field , restarted HA and Google devices… Readded name field , but not with the * at front, but the real HA name of my instance… Rebooted again HA and Google devices… then error was gone…
Then added back the one with * at front, again restarted all, and now it’s fine

Spoke to soon, just restarted my HA? and device verification issue is back, seems restarting google devices are not helping

i have unregistered webhooks now on my log
seems this stuff is not ready yet for production

2022-02-11 10:23:48 WARNING (MainThread) [homeassistant.components.webhook] Received message for unregistered webhook xxxx from 192.168.0.209

i see now this back , no idea why, worked perfect yesterday

According to this, Google’s Local Fulfillment API itself doesn’t support HTTPS:

Closest I can find to an official word on this from Google’s side.

dont get it, yesterday it was working, and now it doesnt anymore
if i reboot the Google devices, i get invalid webhooks now and also verification failed messages

2022-02-11 14:33:15 WARNING (MainThread) [homeassistant.components.webhook] Received message for unregistered webhook xxxxxxxxxxxxxxxxxxxxxxxxxfrom 192.168.0.209

Well. That was specifically around self-signed certs. I wouldn’t expect it to work with self-signed as they will want to verify a cert. That said, you can’t create regular certs for a .local domain so I am not sure how you are supposed to use a local connection with certs. Ever. I don’t know enough about mDNS to know if you can advertise a regular domain name that resolves locally via NAT-LOOPBACK but that would be the only way I could see it function with HTTPS.

Probably the best approach to using local is via regular TCP connections with inbuilt encryption which isn’t something HASS will ever do for valid reasons.

Do me a favor. Restart all your google homes. Then tell googe “hey google sync devices”. Then report back. When I had this occur it fixed itself through magic and this was the ONLY thing I did that day and it worked after. Might be a coincidence. Also if that doesn’t work, wait 24 hours and check again. The second time this happened and fixed itself it was also ‘magic’ and I’m not clear if google is doing something in the background.

I didn’t. however, see unregistered webhooks. You should see if those hooks are related to the google_assistant. You can search the registry in .storage to see what they are related to.

yeah, those hooks are coming from my google devices, my mini and max

verry strange
deleted now the java file, deleted name/disovery, uploaded again, restarted HA , restarted google devices

and now its working again
verry strange

I’ve seen some very strange this happen with that js file. At one point, it was parsed incorrectly by google (it stuck a ‘:’ randomly in the script. I could see it when I downloaded it) and I had to physically change the file by adding a comment to the script and re-upload it). It was getting parsing errors.

So yes I think there are bugs here on the google side.