Google Assistant (Manual Setup) Firewall Issues

Hi guys. I have been using the Google Assistant integration through the manuals setup for quite some time now and it has for the most part been working great.

However, I recently set up a pfSense with pfBlockerNG on my network and it seems that the Google servers (which I assume is related to communication between my Google devices and my HA server through the Google API we create in the manual setup) are being blocked by the Geographical IP filtering in pfBlockerNG.

I have tried adding some of the server IPs to a Whitelist, but the IP of the Google proxy keeps changing and even the ones I have added still seems to keep being blocked.

Does anyone else have a similar setup and/or have any experience with letting the Google servers through the firewall or pfSense/pfBlockerNG specifically?

Is there a listing or similar of the Google Proxy IP addresses that are in use for this available somewhere. Like a range of IPs that could be entered in the Whitelist?