No luck connecting wifi matter device

Just got my first device that supports matter and I can’t get it connected.

My network consists of a router running OPNSense + Unifi Access points. My Home Assistant box is on my default vlan and I have an IoT vlan.

I have tried the following.

  1. No changes to the network in any way phone connected to default vlan wifi attempted to pair matter device using iPhone.
  2. Updated network with IPv6 static interface on default vlan + DHCP server paired with iPhone
  3. Added the mdns opnsense plugin and added the default vlan in the config paired with iPhone
  4. Manually added firewall rules to allow ipv6 to talk to anything
  5. Tried all of the above but with the phone on the IoT vlan
  6. Tried all of the above but using an Android device

I manage to get to the end where it says “verifying network connection” or something like that and it never completes and I get a generic error “Something went wrong”

I’m sure it has something to do with my network but its not clear what.