Setup VLAN and HA tutorial

Just forget it.
Obviously there are people knowing it better, while I have not seen any useful guide, only written words about theory…

Looking forward to see a post of implementing the vlan correctly.
But be prepared of a lot critical questions, we really want to see it implemented as by its “purpose” without any bypass :smiley:

And of course no solution without network sniffing.

Not sure if it is woth that effort, but happy to see it…

Well, by “purpose of VLAN” I mean that VLANs shoulnd’t be able to talk to each other - at all. The default in Unifi is the opposite, which to me is kind of crazy.

The easiest way to solve it would be to put HA in the IoT network and consider it dangerous, but that’s not a good idea either I’ve read. Maybe I’m over thinking it.

image

I don’t use any Unify equipment, but by default, switches (I use netgear, hp and tp-link) come with one vlan setup to all ports, the pvid, which is normally vlan with id 1. It is only when you add more vlans and set different pvid’s to the ports that segregation starts happening.

When you put an additional network interface to HA, attached to the insecure vlan, HA is not meant to be the gateway for this vlan (vlan could be even sealed from accessing internet by not having a default gw configured), HA should be configured to allow only what you want to allow in that vlan (port wise and service wise via HA and even linux host firewall, depending on installation you have) period.

It’s not overthinking, it’s just minimizing your attack surface, if any should exist. This secure vlan would have no internet access, but HA would update itself via the other interface, that is in other more open vlan and you could “monitor” HA via firewall rules, if you believe it might turn rogue on you) :slight_smile:

I work in IT (Systems Administrator - Linux/Windows), and have been doing so for a few years. You are right that that regular consumer grade products (routers) doesn’t get you the possibility to add different VLANs - Unifi does, which is one of the reasons I got it (even though their implementation isn’t “best practice” IMHO).

I don’t know if you or @autoX are using consumer grade products, or if you actually are network masters, but telling from your reasoning I tend to believe you are not able to easily divide your network into VLANs (as VLANs should be setup - no traffic in between). That’s also why I understand your way of making HA the “gateway” (even if it’s not) for the IoT network.

The whole point for me is to totally separate network to be 100% sure that no traffic can sip through. I even setup a separate local DNS server on the IoT network because I don’t want to mix traffic from either one of my secure networks. A good example of how I’m thinking in my setup above is that rule is that I allow HA → IoT, and then established/related traffic to talk back, so in other words I’ll only allow IoT to talk back if HA initiated the traffic. But again, maybe I’m over-thinking it…

My end goal at least is to be rest assured that no matter what gets hacked on the IoT-LAN, nothing can make it’s way through my LAN, or any other network for that sake. Therefore, putting HA on my LAN, and then connect a direct cable to it from my IoT would make HA the weak point. HA will also talk to my other secure LANs and what stops a hacked IoT device to sniff traffic and make attacks on my other LANs as well if HA are connected to them? Nothing really.

So, in that aspect, what’s the point of making a separate VLAN if it doesn’t protect me anyway? What really bothers me is that it seems that I don’t have any choice than to do something similar to what you have done since broadcasting traffic can’t be sent over separate VLANs, or maybe Unifi is a shitty product - I don’t know since it’s my first Unifi device. I usually buy real network servers and install OPNsense/PfSense which I find more capable.

Regarding internet or not for IoT - I will have (already implemented in the setup above) a rule in FW which I add devices to that shouldn’t talk to the internet so called “NoT”.

That is not the primary purpose of a VLAN, and it’s not crazy for UniFi not to implement strict separation by default.
If your personal network design involves implementing VLANs for security purposes and you want the networks to be segregated then you have to use tools like firewalls or ACLs to achieve that. That’s not crazy at all and just your tools of trade.

Well, that’s what VLANs do, they form separate broadcast domains and that’s why the broadcast traffic does not flow between VLANs.

Are you sure you are concerned about broadcast traffic, or is it multicast traffic for example for auto discovery? If the latter, you could install a multicast relay and then allow specific devices to send their multicast traffic into the network where Home Assistant is located.

Looking at your network design that you posted earlier, I think it’s pretty good and reasonable. I have implemented a very similar design and it has been working quite well over the years. However, it also requires constant supervision, testing and modifications to suite new devices and their unique behaviours.

I would not say that vlan support is what separates consumer grade from enterprise grade switches :slight_smile:
The HA is never a gateway in my setup, it is only another member/client of the vlans/subnets that it belongs to.
The way HA works, as far as I believe, it searches the network for IoT devices/clients that already exist and are ready to talk to it. I don’t have/remember any device at all that I had to do the reverse (give the ip of HA to the client or that it “knew” that HA was there - yes, sometimes you might specify an mqtt server ip, that could or not be HA, but that’s already part of how you configure your HA and what services you make available on it)
You can set up different dns/dhcp/whatever servers in different vlans/subnets and everything will work just fine :slight_smile:
In a contained vlan where the client has controlled/no internet access, it should be really difficult to hack into HA (read firewall rules for allowed clients and ports) and still jump from there to the LAN. I guess there is a much greater possibility that someone will hack a device (pc/phone/laptop) that already lives in your internet enabled LAN and do something bad from there.
Remember that HA is a container that lives on a linux host, so securing the host might be a good first step. Second this is not a howto on network security. I am not a network security expert!
Anyway the hacked IoT device would not sniff any more that what already goes on that vlan/subnet (unless it has magic powers :stuck_out_tongue: ) because it would still have to take over HA to get to sniff traffic on the other networks HA belongs to.
Below is a 5 min. diagram/sketch of how I try to segregate things (hint: this is not really my network :wink:):
VLAN1 (green) is a “normal” vlan with internet access.
VLAN2 (orange) is a light switch vlan with internet access enabled only when it’s time to update firmware
VLAN3 (blue) is a temperature sensors vlan with internet access enabled only when it’s time to update firmware
VLAN4 (red) is a cam vlan with no connection to any other vlan.

Off course the firewall controls/blocks network flows between vlans, but vlan4 is not even configured on the firewall. These are all VMs (FW and HA) so assume that joint cable colors are tagged vlans.

HA belongs to all these vlans/subnetworks, so it can talk to these devices directly/freely to avoid all kinds of subnet crossing problems. This also makes HA a very important/central device in your network design, so it should be well secured and maintained! If you believe HA is really a danger to your LAN (I would say the opposite is more likely), then you should have yet another vlan just to allow HA to talk to the internet, not common to the one you use with pc/laptops/phones, and exclude HA from VLAN1.

2 Likes

I think you got that wrong. :wink:

Stating the obvious here.

Actually, I’m not sure. I haven’t got the devices in place yet to test it, but I’m thinking I will connect all smart devices like vacuum cleaners, lawn robot, my Schnieder Wiser system, printers, etc etc… I don’t know it the any of those needs broadcasting, but time will tell.

One thing I’m certain of, is that I will not connect any smart locks or anything like that. That would be taking a step too far.

You mean something like this?

image

Does the Docker act differently you mean?

Not a problem, I won’t replace devices once everything is up and running. Monitoring and fixing is part of the fun. :slight_smile:

It depends on what you compare. I mean a standard ASUS-RT68U doesn’t have VLANs for example. If you talk switches, that’s another topic. :slight_smile:

Other than that, everything you just wrote makes me lean towards putting the HA in my main LAN instead and let IoT talk back if contacted, and only then. Another alternative would be to put up several HAs in different networks. Since they will be virtualized it doesn’t really matter, HA won’t be reachable from the outside anyway, only from my LAN/WLAN. We’ll see what happens once I move into the new house and have all the devices in place, but your points here have helped me towards a decision at least.

Appreciated! :+1:

1 Like

No, mDNS is just a subset of multicast traffic. Relaying multicast traffic from devices between subnets may help in cases where devices only use multicast traffic to communicate with a mobile app or Home Assistant.

1 Like

I mean a standard ASUS-RT68U doesn’t have VLANs for example

Because Asus doesn’t want to take the time to put it in consumer grade hardware, obviously. But actually, I do have one and with Merlin’s firmware I have it setup with vlans and HA. Does the job wonderfully :wink:

1 Like

I think all assumptions have been clarified already.
And no, I am not a network master but I at least qualified to get one, some years ago.

Nevertheless.
I do use mikrotik, which is far away from being a consumer product.

My HA is just connect as a client to two different vlans to overcome the multicast story

I don’t intend to have an enterprise network at home.
As I am running a lot of iot devices and having my door lock connected to wifi, I want to make sure I am preventing acces to it.
And the issue is not my IT but the non IT persons around me.
While all that was fine, I encountered issue with HA as e.g. mdns was not working anymore.
Therefor I set my HA into 2 vlans, but not as a router, just a a client in each.

Hope this helps.

OK, we’ll see what I end up with.Right now the house isn’t even built. Prepped a lot of outlets though. :slight_smile:

Hi, I recently replaced the standard firmware for Merlin in an 88 and trued to figure out how doing this but I seem not to be skilled enough and missed the trick. Can you give a little hint about how you did this in your 68 ?
(and thanks for all those valuable posts)

Absolutely. Sorry to others for the off-topic post. I will give basic guidelines, although longish, and you should be able to get there on your own. If not, just tell me here or via private message, so I can help you further. (I just did these procedures on a second rt-ac68u, so everything is fresh)

  1. Enable AP mode on the router, enable the number of virtual ssids you wish to have in guest wifi settings and lastly enable the jffs partition setting
  2. ssh into the router
  3. First test all the commands for vlan creation/config and bridge creation/config in sequence and copy them one by one to a notepad, so you can slowly but steadily create your own script (and if something goes wrong, you can always reboot the router and start over without any problems)
  4. When all commands are prepared, create/edit a file named “services-start” under /jffs/scripts/ and “chmod a+x services-start” after, so it can be executed by the system (I again recommended doing all rules and commands first in notepad or equivalent because any commands saved to this file will be executed during boot. Any wrong configuration could leave your router with no access/not working, and you’ll have to reset and restart (happened to me, and it’s not a tragedy anyway)
  5. To configure vlan’s you should use a tool called “robocfg”

robocfg show

will output your current vlan config, each number is a port on the router + cpu port

This is a default config output without any configuration:
1: vlan1: 0 1 2 3 4 5t
2: vlan2: 5t

This is mine after adjusting the router to my needs:
1: vlan1: 0t 5t
2: vlan2: 5t
10: vlan10: 0t 5t
20: vlan20: 0t 1 2 3 4 5t

Important notes:

  • 0 is the WAN port used as tagged uplink to switch
  • 1,2,3,4 are LAN ports
  • 5 is the CPU internal port (newer router versions might have CPU port as number 8, check your model)
  • vlan1 and vlan2 are internal to how the system works LAN and WAN respectively, so try to either adjust your vlanning to them, or just leave them alone and use higher vlan numbers like I did
  • the “t” indicates a tagged port

So as you can see, in my settings, all traffic is tagged before leaving the router to the switch.
Next, you’ll need to create vlans and bridges to attach the network interfaces to them. Here goes my example of configuration:

/usr/sbin/robocfg vlan 1 ports "0t 5t"
/usr/sbin/robocfg vlan 2 ports "5t"
/usr/sbin/robocfg vlan 10 ports "0t 5t"
/usr/sbin/robocfg vlan 20 ports "0t 1 2 3 4 5t"

Next, create the nonexistent vlans (adjust numbers) and up them:

/sbin/vconfig add eth0 10
/sbin/vconfig add eth0 20
/sbin/ifconfig vlan10 up
/sbin/ifconfig vlan20 up
  1. Next we create/adjust the bridges.

First check the status:

brctl show

bridge name	bridge id		STP enabled	interfaces
br0		8000.3c7c3f728380	yes	vlan1 #vlan1 interface
				                eth1 #eth1 wireless 2.4ghz interface
				                eth2 #eth2 wireless 5ghz interface
				                wl0.1 #fist guest/virtual wireless 2.4ghz
				                wl1.1 #first guest/virtual wireless 5.ghz

Now an example on how to set up the bridges

brctl addbr br20
brctl stp br20 on
ifconfig br20 192.168.20.20 netmask 255.255.255.0 #config br20 ip
brctl addif br20 vlan20 #add vlan20 interface to br20
brctl delif br0 eth1 #remove eth1 from br0
brctl delif br0 eth2 #remove eth2 from br0
brctl delif br0 wl0.1 #remove wl0.1 from br0
brctl delif br0 wl1.1 #remove wl1.1 from br0
brctl addif br20 eth1 #add eth1 interface to br20
brctl addif br20 eth2 #add eth2 interface to br20
brctl addif br20 wl0.1 #add wl0.1 interface to br20
brctl addif br20 wl1.1 #add wl1.1 interface to br20
ip link set br20 up

Just use the above to create additional bridges, like br10 or your own.
It should produce this:

brctl show 
bridge name	bridge id		STP enabled	interfaces
br0		    8000.3c7c3f728380	yes		vlan1
br10		8000.3c7c3f728380	yes		vlan10
br20		8000.3c7c3f728380	yes		vlan20
				                        eth1
				                        eth2
                                        wl0.1
                                        wl1.1

Important notes:

  • bridges are needed to join interfaces (physical and virtual) into one network
  • each bridge should belong to a separate network
  • each guest/vssid created in the webui creates a new interface that needs to be added to a bridge
  • we are using the same rules for ssid separation, there are plenty of examples/tutorials out there:
    https://gist.github.com/b1tman1ac/3d2cad0094e78a587f217a0720c9c11c
    is a good starting point
  1. Wrap up the internal mechanisms
    Check the last link in point 6 to create good formatting and to create the nvram rules necessary for your new bridges, lans, etc
    Down goes a possible example for what we have above.
nvram set lan_ifnames="vlan1"
nvram set lan_ifname="br0"
nvram set lan1_ifnames="vlan20 eth1 eth2 wl0.1 wl1.1"
nvram set lan1_ifname="br20"
nvram set lan2_ifnames="vlan10"
nvram set lan2_ifname="br10"
nvram set br0_ifnames="vlan1"
nvram set br0_ifname="br0"
nvram set br20_ifnames="vlan20 eth1 eth2 wl0.1 wl1.1"
nvram set br20_ifname="br20"
nvram set br10_ifnames="vlan10"
nvram set br10_ifname="br10"
nvram commit
killall eapd
eapd
  1. copy rules/commands to services-start, save and restart after proper formatting and checking!

Good luck! :slight_smile:
p.s. - another possible helpful link for ac86u

1 Like

Side question - is it possible to limit which network interface(s) HASS Management interface listens on?

I have just had to give HASS an interface in my “IOT” VLan (which I consider semi-trusted, and therefore want to segregate it from my main LAN; but at the same time I want to add some security).

I have two NICs on my HASS VM (HASSOS, running on Proxmox) - enp6s18 and enp6s19. The first is my LAN, the second is my IoT VLAN. How can I make HASS only listen on enp6s18 for HTTPS TCP 8123, but not at all on enp6s19?

Edit - Ok - managed to somewhat solve this. I added the below in my HASS config

http:
  server_host: 10.1.2.50

Where 10.1.2.50 is the IP of my HASS LAN in the “Trusted” network.

Edit 2: Doing this makes my Mi devices on my IoT Lan not respond. I guess they need the web server to communicate with HASS? If this is infact the case how do I safely lockdown the HASS WebUI (and any other relevant ports)?

I have never dug too much into this, so I don’t know what ports are necessary for what (I guess it might depend on add-ons installed (samba and so), but you can always check with nmap from the outside or Wireshark. If you read the documentation about http integration, that you just referred to, at HTTP - Home Assistant you’ll see that most interactions from HA and IoT equipment uses APIs on top of http integration (it’s more or less a yes to your question). The NO part to your question would be that the IoT “client” would have to be aware of standard ports to connect to them (think of browsers and web pages) and by changing the listening port from 8123 you’d break everything and this isn’t true :slight_smile: So that leads me to believe that configuration in Home Assistant integrations is mostly done via discovery of devices that make themselves announced/available or explicit configuration in HA and not the contrary, which to me speaks volumes about what device is more open/visible. There are many ways to protect Home Assistant 8123 port: http integration definitions, reverse proxy with nginx, firewall filtering, etc. Or even using mqtt as an intermediary between HA and IoT equipment.
Check your options and context, so you can choose adequate solutions :slight_smile:

Found this post while searching for similar issue. I have been running HA in Proxmox as a VM for at least a year now. Have quite similar setup with 3 networks. 2 of the networks are vlans for IoT and NoT devices. Reason why I am doing this is due to me using Xiaomi devices and the gateway uses mDNS/multicast to communicate, something along that line.

All was working very well until I am not sure which release of HA or could it be Unifi…the exact behavior was seen as described by @nikipore. Upon further investigation and troubleshooting, I realized that all my other systems in the network were also having almost 100% CPU usage. The minute I pulled the network cable, the CPU usage went down to <10%. This is only affecting one CPU core so you might not catch it if you do not turn on the logical processor view in task manager. Weird thing is it does not affect the system if using wifi. The moment you plug the network cable back, the CPU usage of CPU 0 will shoot up again.

I initially thought it could be the VM corrupted and I setup another new Proxmox system. All was looking good initially till I added the necessary network cards to talk to the other 2 vlans. Then I am back with the same problem. :cry:

Could anyone else confirm if they are seeing such behaviors? Or better still how to solve it.

Edit: I was searching around and found this other thread (Multicast container causing feedback loops ¡ Issue #1 ¡ home-assistant/plugin-multicast ¡ GitHub) which looks like the issue here. Will troubleshoot further. Hopefully someone also has some ideas how to solve.

Update: I was testing out a few things and I believed I have solved the issue. What I did was to turn off the mDNS option in the vlans and only have that option turned on in the main network. Now there is no more 100% CPU usage and every sensors is still working in HA. Will continue to monitor for a day and see.

Firewalla gold all the way!

Thank you Raymond for reactivating my interest with your post. However, it haven’t had to change my mDNS repeater settings (my router is also Ubiquiti, but not UniFi, but ER-X). Instead, I found in your cited thread that the multicast plugin has been updated very recently s.t. the plugin no longer acts as an mDNS repeater between the connected NICs, cf. this post. It works now and the plugin has been merged into the stable channel of supervisor.

Yes…I indeed also found some issues in the last few days with the mDNS repeater settings turned off for the 2 vlans in Unifi. Even though everything works fine in HA, I was not able to find my Denon receiver using the Denon app. The same also happens when I use the Google app to connect to the Google Home speakers. Funny thing is Spotify is able to find all the GH and allow casting to them. As I don’t usually use the app so much, I temporarily connect to the particular ssid if I need to control them. I will definitely try updating to the beta plugin over the weekend to see if that resolves everything.

Thanks @nikipore for confirming on your Ubnt setup. :slight_smile:

I just wanted to jump in and reply with a thank you as well for your post. There’s a couple of other details people may want to consider if they are following this. Hopefully “future me” also finds this helpful.

The parent interface configuration:
It is likely that if you are using VLANs that you don’t want your host sending untagged traffic, or basically doing anything on the untagged interface at all. This can be achieved by setting the parent interface to not autoconnect on boot. I did this with the following commands:

nmcli con edit Supervisor\ eth0
set connection.autoconnect no
save
quit

This has the effect that the “Supervisor eth0” connection will show as down when you do a nmcli con show but the VLAN sub interfaces will remain operational.

Note: it is not sufficient to just delete the “Supervisor eth0” connection, HA will automatically recreate it on reboot with default settings including autoconnect enabled.

Route Metrics:
If you want to control routing so that your preferred network is used as your default route, you can set the route metrics on your non-preferred networks to a high value. For example, I run two IOT VLANs, one which does not have Internet access (VLAN 100), and one that does (VLAN 101), plus I have a management/system network (VLAN 1) with Internet access. I want HA to use VLAN 1 as its default route. If all route metrics are equal, HA may just decide to use VLAN 101 for its preferred default route which I don’t want. To fix this I used the following commands:

nmcli con edit eth0@vlan100
set ipv4.route-metric 2000
set ipv6.route-metric 2000
save
quit
nmcli con edit eth0@vlan101
set ipv4.route-metric 2000
set ipv6.route-metric 2000
save
quit

All my interfaces use DHCP configuration (and DHCP reservations on my router), and the route metrics on routes advertised through my eth0.1 interface get a metric value of 400 (as set by HA). Setting the other interfaces to use a metric of 2000 means the eth0.1 interface routes are preferred (lowest metric number wins).

2 Likes