Google Assistant + CF Tunnels, VERY inconsistent behavior

I’ve got a really weird problem with the Google Assistant integration running over a Cloudflare tunnel where it intermittently stops working-- and even two commands in a row, one will work and the other won’t. No problems with the original linking, only afterwards. “Hey Google sync my devices” also fails but also-- only intermittently. Sometimes it works, sometimes not. Bizarre!

I painstakingly went over the HASS documentation and triple-checked every item. I’m aware it’s complex and long, so I really invested time into it in an effort to not be embarrassed when I finally resort to asking for help and it ends up being something super stupid. For example, everything is IP addresses no hostnames (other than the CF tunnel host of course), so no exposure to DNS shenanigans. IPv6 is disabled too. I did my homework, best I could.

I also went to Cloudflare and tried a bunch of page rules and WAF rules to eliminate caching and ensure Google is permitted to access the host. Running a mtr from outside my network I show zero packets being lost to the Cloudflare tunnel hostname. It’s definitely internet-accessible, I tried with a VPN.

I also checked my LAN’s firewall (a UDM Pro) just in case, although I’m using Cloudflare tunnels so no port is opened or anything. Nothing in there to account for this behavior.

Reading through the Google cloud console action logs, it gives “BACKEND_FAILURE_URL_ERROR” which looks like Google can’t reach my CF URL. I can’t seem to figure out why!

I’m running the very latest version of HASS OS in a Proxmox VM. Config YAML is all bog-standard, nothing unusual, straight from the documentation, including the SECRET_ACCOUNT.JSON file which I also triple-checked was correct. No issues with perms either, everything is readable by root.

Are incoming webhook calls logged anywhere in HASS? I can’t seem to find them.

Anyone have any ideas, other than just paying nabu casa to do it for me?

Happy to post any config files people need, although again they’re very standard. Here’s the Google actions log error:

{
  "textPayload": "requestId <<removed>> failed with code: BACKEND_FAILURE_URL_ERROR",
  "insertId": "28bvi2frnk33g",
  "resource": {
    "type": "assistant_action",
    "labels": {
      "action_id": "SMART_HOME_EXECUTE",
      "project_id": "<<removed but it's correct>>",
      "version_id": ""
    }
  },
  "timestamp": "2023-04-24T18:15:51.351253610Z",
  "severity": "ERROR",
  "logName": "projects/<<removed>>/logs/actions.googleapis.com%2Factions",
  "receiveTimestamp": "2023-04-24T18:15:51.362590945Z"
}
1 Like