To be clear, I’m not exposing 1883/1884 to the internet, only to another private network layer.
I recently decided to change my network setup by adding a separate network for IOT devices. To do that I added a separate physical router for IOT devices, which is connected directly to the internet. My private router is then connected to the IOT router and gets to the internet through it.
HA is running inside the private network, while most of my MQTT clients are on the IOT network.
I thought connecting the MQTT clients would be as easy as forwarding TCP for port 1883 into the private network from the IOT network but something is wrong and I’m struggling to diagnose the issue. The behavior is inconsistent but usually the client will get in a loop where it says it’s connected then immediately disconnects then repeats. As this cycle is happening Mosquitto’s logs look like this:
New connection from 192.168.1.1 on port 1883.
Client <unknown> disconnected due to protocol error.
New connection from 192.168.1.1 on port 1883.
Client <unknown> disconnected due to protocol error.
....
I’ve made sure everything is updated, restarted everything, and have fully reinstalled Mosquitto from the add on store and this still happens.
I don’t think it’s a network config issue since I do see logging happening and I can ping port 1883 from a computer in the IOT network. At first I thought it was an issue caused by Mosquitto rejecting unsecured connections from outside the local network but I’ve had messages come through in the brief period a client shows as connected so I don’t think it’s that. I also thought it could be an issue caused by all connections showing as 192.168.1.1 (the gateway to the IOT network) but turning off all but 1 device doesn’t fix the issue.
Those are the only 2 things I could think of that might be causing the issues. Am I missing something obvious here?
This wouldn’t be called a ping, but whatever.
Could you try using the clients mosquitto_pub and mosquitto_sub from the mosquitto-clients package on that computer?
From how you described your network, I don’t think this should be happening.
Is the following description of your network correct?
You have 2 networks:
IoT network 192.168.2.0/24
private network 192.168.1.0/24
Your IoT network’s router has the WAN port connected to the Internet and is configured to do srcnat masquerade for all traffic going through the WAN interface.
Your private network’s router’s WAN port is connected to one of the IoT router’s LAN ports and also does srcnat masquerade for all traffic going through the WAN interface. The private router has a dstnat rule forwarding all connections made to it’s WAN interface’s IP address (say 192.168.2.2) on port 1883 to your MQTT server (e.g. 192.168.1.11).
You are telling your IoT devices to connect to 192.168.2.2 on port 1883.
Is this correct?
If so, I would eliminate the double NAT situation altogether if I were you. I would use a single configurable router (I tend to use Mikrotik, but use whatever suits your needs), take one of it’s “LAN” port out of the LAN bridge and assign it to the IoT network. I’d then just use firewall rules to restrict access from one network to another, with no NAT in between.
However, you don’t have to do this. Your setup should work, too. Please post more information about your network. I suspect your private network router masquerades connections coming from the IoT network to your private network. That alone should not cause problems, but it’s completely unnecessary.
I’m not sure exactly what you mean by “srcnat masquerade”. Each router is running it’s own DHCP server. The private network is accessible through the gateway/router at 10.0.0.100 and is set up to forward ports in from 1883 (TCP) to the Home Assistant server.
I know very little about networking but this seemed like the way to split up the networks and spread the wireless connection load across routers
You are correct about the WAN setup though:
internet → IoT router wan
IoT lan → private wan
Sure, I tried this same command both from the private network and the IoT network, it only worked on the private network. mosquitto_sub failed with the same error
There was no output from the mosquitto server when I ran that ^
That makes it seem like there’s no port forwarding setup from IoT to Private but I have forwarding rules for the HA UI, plex, and SSH that work fine and are set up the same.
I would have done this for the simplicity of setup but one of the things that pushed me to move to this setup (besides the obvious privacy benefits of segregated networks) is issues with Wifi reliability. It seemed to be caused by too many devices being connected to the one router.
So you can e.g. connect from the IoT network to your SSH server in your private network without issues, but similar port forwarding configuration for MQTT fails? If this is the case, double check that your port forwarding (dstnat) rule for mqtt points to the correct host and port.
Try specifying log_type in your mosquitto.conf. A haven’t tested it myself, but it seems like setting it to all should provide quite a lot of debug information.