HA not routing from LAN to WG

Configuration
1

Hello.

Tried asking about this in the Wireguard Add-on thread, but got no response. As far as I, as a total layman, can deduce, it very possibly isn’t even the problem with the Add-on, but with the HA.

I’ve got HAOS installed on two identical (pretty old) Intel NUCs (Pentium Silver j5005, 16GB RAM) on two different locations. Also have installed the Wireguard Add-on on both of them, configured to use them/it as a site-to-site VPN solution.

For about a year everything worked fine, the LOCAL_LAN clients could reach the REMOTE_LAN clients, LOCAL_LAN clients could reach the REMOTE_HA, REMOTE_LAN clients could reach the LOCAL_LAN clients and LOCAL_HA instance.

Suddenly, about a month (maybe a bit more) ago, the setup partially stopped working. The part, of course, that’s most important for me. REMOTE_HA can reach both LOCAL_HA and LOCAL_LAN clients, LOCAL_HA can reach both REMOTE_HA and REMOTE_LAN clients, but the clients themselves can’t reach each other nor the HA instance remote to them via the VPN. When doing traceroute from a client, the traffic reaches the HA instalnce local to the client, but just times out there. The ipables -S in the underlying OS shows that forward is enabled, the iproute in both HA instances shows traffic is to be routed to/through the add-on.

Please help. Thanks in advance.

EDIT: lans, pings, traceroutes

slika
slika2387×1606 294 KB

EDIT: iptables -S from HAOS_1

# iptables -S
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-BRIDGE
-N DOCKER-CT
-N DOCKER-FORWARD
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-FORWARD
-A FORWARD -d 10.3.0.0/24 -j ACCEPT
-A FORWARD -d 192.168.1.0/24 -j ACCEPT
-A DOCKER -d 172.30.33.7/32 ! -i hassio -o hassio -p udp -m udp --dport 51820 -j ACCEPT
-A DOCKER -d 172.30.33.4/32 ! -i hassio -o hassio -p tcp -m tcp --dport 5050 -j ACCEPT
-A DOCKER -d 172.30.33.2/32 ! -i hassio -o hassio -p tcp -m tcp --dport 9080 -j ACCEPT
-A DOCKER -d 172.30.33.2/32 ! -i hassio -o hassio -p tcp -m tcp --dport 9050 -j ACCEPT
-A DOCKER -d 172.30.33.1/32 ! -i hassio -o hassio -p tcp -m tcp --dport 8884 -j ACCEPT
-A DOCKER -d 172.30.33.1/32 ! -i hassio -o hassio -p tcp -m tcp --dport 8883 -j ACCEPT
-A DOCKER -d 172.30.33.1/32 ! -i hassio -o hassio -p tcp -m tcp --dport 1884 -j ACCEPT
-A DOCKER -d 172.30.33.1/32 ! -i hassio -o hassio -p tcp -m tcp --dport 1883 -j ACCEPT
-A DOCKER -d 172.30.32.6/32 ! -i hassio -o hassio -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER ! -i hassio -o hassio -j DROP
-A DOCKER ! -i docker0 -o docker0 -j DROP
-A DOCKER-BRIDGE -o hassio -j DOCKER
-A DOCKER-BRIDGE -o docker0 -j DOCKER
-A DOCKER-CT -o hassio -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-FORWARD -j DOCKER-CT
-A DOCKER-FORWARD -j DOCKER-ISOLATION-STAGE-1
-A DOCKER-FORWARD -j DOCKER-BRIDGE
-A DOCKER-FORWARD -i hassio -j ACCEPT
-A DOCKER-FORWARD -i docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i hassio ! -o hassio -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o hassio -j DROP
-A DOCKER-USER -j RETURN
2

I don’t use WireGuard, so not probably won’t help you much.

Case#1: From looking at the successful traceroute starting from HA_1, it is going through what appears to be the Docker network inside HA_1 to get to what I think is the WireGuard container on 172.30.33.7 which probably forwards over its VPN (which I think is the 10.11.0.3/10.11.0.1) and later gets routed by the remote WireGuard on HA_2 onto the 10.3.0.x network.

Case#2 By comparison, looking at the traceroute from a client on say VLAN40, it looks like some router (10.0.40.1) is forwarding it to HA_1. It arrives at HA_1 (linux kernel) which should be doing a route lookup, and not sure, but seems it should be a route to 172.30.33.7 (the WireGuard container in HA_1). And from there it would be forwarded as in case#1.

My guess is that there may not be a route, but if there does happen to be a route in the HA_1 linux kernel to forward to 172.30.33.7, that the iptable rule -A DOCKER -d 172.30.33.7/32 ! -i hassio -o hassio -p udp -m udp --dport 51820 -j ACCEPT is blocking it.

3

Hey! Thank you very much for taking an interest, investing your time. And sorry it took me this long to answer.

Case#1: exactly as you concluded.

Case#2: the local router has an ip of 1.0.VLAN_ID.1 on every vlan it routes; the traceroute was done from a pc in the “trusted vlan” 40 that has the appropriate rules set in the firewall to be able to even access the remote VLANs (among other stuff). everything as you concluded yourself.

I don’t know how/where exactly I can check for route(s) you mention - in HA_1 for route HA_1_IP (local HA) → 172.30.33.7 (Wireguard Addon/Docker IP) → HA_2_WG_IP (remote ha over WG) → REMOTE_LAN client(s) / HA_2.

If I check by ssh-ing to HA - both rules for appropriate VLANs are here

➜  ~ iproute
default via 10.0.10.1 dev eno1  src 10.0.10.20  metric 100
default via 10.0.60.1 dev wlo2  src 10.0.60.20  metric 20600
10.0.10.0/24 dev eno1 scope link  src 10.0.10.20  metric 100
10.0.60.0/24 dev wlo2 scope link  src 10.0.60.20  metric 600
10.3.0.0/24 via 172.30.33.7 dev hassio  src 10.0.10.20
172.30.32.0/23 dev hassio scope link  src 172.30.32.1
172.30.232.0/23 dev docker0 scope link  src 172.30.232.1
192.168.1.0/24 via 172.30.33.7 dev hassio  src 10.0.10.20

If I check by ssh-ing to HAOS (root) - both rules for appropriate VLANs are here

# ip r
default via 10.0.10.1 dev eno1 proto dhcp src 10.0.10.20 metric 100
default via 10.0.60.1 dev wlo2 proto dhcp src 10.0.60.20 metric 20600
10.0.10.0/24 dev eno1 proto kernel scope link src 10.0.10.20 metric 100
10.0.60.0/24 dev wlo2 proto kernel scope link src 10.0.60.20 metric 600
10.3.0.0/24 via 172.30.33.7 dev hassio src 10.0.10.20
172.30.32.0/23 dev hassio proto kernel scope link src 172.30.32.1
172.30.232.0/23 dev docker0 proto kernel scope link src 172.30.232.1
192.168.1.0/24 via 172.30.33.7 dev hassio src 10.0.10.20

If I check from inside WG Addon/Docker - both rules for appropriate VLANs are here

# docker exec 13493bfbacb3 ip r
default via 172.30.32.1 dev eth0
10.3.0.0/24 dev wg0 scope link
10.11.0.3 dev wg0 scope link
10.11.0.5 dev wg0 scope link
10.11.0.6 dev wg0 scope link
172.30.32.0/23 dev eth0 proto kernel scope link src 172.30.33.7
192.168.1.0/24 dev wg0 scope link

So it could be what you propose in the end. Unfortunately I’m a total noob in (linux) networking so I’ll try and decipher what exactly the rule you mention does. If you have any more words of wisdom/help and a surplus of time, please do chime in.

Thanks once again for your time… and happy Easter if you’re celebrating, a great Sunday if you’re not

4

Just writing here in case anyone has the same problem, maybe it helps. Anyway, seems that inserting the needed FORWARD rules in HAOS iptables before other forward rules did the trick.

I’ve substituted (removed those)

-A FORWARD -d 192.168.1.0/24 -j ACCEPT
-A FORWARD -d 10.3.0.0/24 -j ACCEPT

I’ve had in iptables, with the following ( by iptables -I FORWARD -i eno1...)

-A FORWARD -s 10.0.0.0/16 -d 192.168.1.0/24 -i eno1 -j ACCEPT
-A FORWARD -s 10.0.0.0/16 -d 10.3.0.0/24 -i eno1 -j ACCEPT

In case you see anything I might have done wrong / insecure, please do chime in.

Thanks again!