As part of my performance tuning of Home Assistant, and my desire to use MQTT for other things, I'm moving from the built-in Add-In to a separate Mosquitto VM. Migrating off of Home Assistant is actually not very intuitive, and as a result, I decided to write my notes for future people who come along. I asked about this in this thread but ended up solving it and wanted to get my learning down on paper for others to benefit from.
First, you'll need to enter the advanced terminal so you can get to the real docker commands; in this case I used the HACS app "Advanced SSH & Web Terminal" and turned protected mode off. This isn't totally required, but it was helpful for me in my discovery. Once you have that, you can enter the mosquitto docker container with the following command:
docker exec -it $(docker ps | grep mosquitto | awk '{print $1}') /bin/bash
From here, we can get the config out of /etc/mosquitto and discover that it's using the go-auth plugin, which isn't great as the author has end-of-lifed it, so it's NOT where I want to target: GitHub - iegomez/mosquitto-go-auth: Auth plugin for mosquitto. ยท GitHub
Let's capture the current state so we can look through it:
user root
log_dest stdout
log_type error
log_type warning
log_type notice
log_type information
log_timestamp_format %Y-%m-%d %H:%M:%S
persistence true
persistence_location /data/
# Limits
# max_queued_messages is effectively the upper limit of
# the number of entities on Home Assistant if startup
# is busy and cannot read messages fast enough
max_queued_messages 8192
# Authentication plugin
auth_plugin /usr/share/mosquitto/go-auth.so
# Restore pre-7.0 behaviour: allow '/' in client ids and usernames.
# The addon's ACL file does not use pattern substitution (%c/%u), so
# mosquitto 2.1's broker-side rejection of +, #, / is not needed here.
auth_plugin_deny_special_chars false
auth_opt_backends files,http
auth_opt_hasher pbkdf2
auth_opt_cache true
auth_opt_auth_cache_seconds 300
auth_opt_auth_jitter_seconds 30
auth_opt_acl_cache_seconds 300
auth_opt_acl_jitter_seconds 30
auth_opt_log_level error
# HTTP backend for the authentication plugin
auth_opt_files_password_path /etc/mosquitto/pw
auth_opt_files_acl_path /etc/mosquitto/acl
# HTTP backend for the authentication plugin
auth_opt_http_host 127.0.0.1
auth_opt_http_port 80
auth_opt_http_getuser_uri /authentication
auth_opt_http_superuser_uri /superuser
auth_opt_http_aclcheck_uri /acl
listener 1883
protocol mqtt
listener 1884
protocol websockets
So, it's running as root and not the mosquitto user, which isn't great. Also, it's logging to stdout, but I want to log to a file. This is the configuration I settled on:
# Plain MQTT Protocol
listener 1883
protocol mqtt
# Websockets Protocol
listener 1884
protocol websockets
# Prevent unidentified users
allow_anonymous false
# Define where to store passwords
password_file /etc/mosquitto/passwd
# Define the acl file
acl_file /etc/mosquitto/acl
# PID File Information
pid_file /var/run/mosquitto/mosquitto.pid
# Logging
log_dest file /var/log/mosquitto/mosquitto.log
log_type error
log_type warning
log_type notice
log_type information
log_timestamp_format %Y-%m-%d %H:%M:%S
# Persistence
persistence true
persistence_location /var/lib/mosquitto
# Limits
max_queued_messages 8192
The ACL and password path are the biggest issue here. Let's look at the pw file:
homeassistant:###==
addons:###==
mqtt-admin:###==
And here's the acl file:
user homeassistant
user addons
user mqtt-admin
This confused me. First, all my clients connect via "mqtt-user" and that's not in here, turns out when I set this up YEARS ago, I made a home assistant user for mqtt-user, and it gets access to Mosquitto. I tested this by logging in with my normal personal home assistant user using MQTT Explorer, and it worked. The go-auth plugin is allowing authentication with home assistant users, and that will probably need to go. Also, the ACL file does NOT define the ACLs, and by default, none of the users on the new server can access MQTT via MQTT-Explorer pointing at the new VM. Lastly, there's the "homeassistant" and "addons" users with hashed passwords, and I have no idea what they are OR where I need to change them. I know in mosquitto I can set passwords, but unless I change them in homeassistant, it'll break any connections. The guide here has some clues as to where to change things:
A lot of search results point to /share/mosquitto/acl.conf or /share/mosquitto/accesscontrollist but those don't exist on HomeAssistant OR in the docker container, so I'm SOL. I believe this is a legacy configuration, or a way to pass configuration from the host to the container. The current documentation for the plugin here is helpful. The above mentioned acl files must be manually created, and as a result, we now see that all users are wildcard allowed readwrite to all topics. As a result, here's my final acl file, and my plan was to move different services to their own users with their own narrow ACL controls, but my initial experiments just broke things so I have left that as a pin to evaluate later:
user mqtt-admin
topic readwrite #
user mqtt-hass
topic readwrite #
user mqtt-z2b
topic readwrite #
user mqtt-tasmota
topic readwrite #
#topic readwrite tele/#
#topic readwrite tasmota/#
#topic read cmnd/#
user mqtt-zigbeehub
topic readwrite #
user mqtt-ratgdo
topic readwrite #
When we create a user, or update a password, we use the following credentials (pointing them to the desired passwd file, which can be renamed/moved/changed as needed so long as it's updated in the config):
mosquitto_passwd /etc/mosquitto/passwd mqtt-admin
If the file doesn't exist, then make sure to add the -c flag to create it.
Now we get to pointing things in Home Assistant at the new MQTT broker. Locate the MQTT integration and do "Reconfigure." You will point at the new server, and if you changed the credentials you can update them here.
We also need to reconfigure Zigbee2MQTT, which you should NOT do in the Z2M web interface, but instead from Home Assistant's Apps screen, where the config variables will get injected to the docker container. Click the App, go to the "Configuration" tab, expand "mqtt" and set your server, user, and password. Then restart the container.
Lastly, I had to go to all my smarthome devices (tasmota devices, ratgdo, sonoff ifans, shellys, smart plugs, and zigbee bridge) and update MQTT there as well, which was tedious but ultimately fine.
MQTT explorer as an AppImage was instrumental in testing the config and credentials. One other quirk; on ubuntu the mosquitto package has issues with the mosquitto user accessing the run folder to store the pid file. Because of this, I made a modified mosquitto.service file in /etc/systemd/system with the following contents:
[Unit]
Description=Mosquitto MQTT message broker
Documentation=https://mosquitto.org/man/mosquitto-8.html
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service
[Service]
Restart=on-failure
RestartSec=2
User=mosquitto
Group=mosquitto
# Here's the important change
ExecStartPre=+/bin/mkdir -m 740 -p /var/log/mosquitto
ExecStartPre=+/bin/chown mosquitto:mosquitto /var/log/mosquitto
ExecStartPre=+/bin/mkdir -m 740 -p /var/run/mosquitto
ExecStartPre=+/bin/chown mosquitto:mosquitto /var/run/mosquitto
ExecStart=/usr/sbin/mosquitto --config-file /etc/mosquitto/mosquitto.conf
ExecReload=/bin/kill -HUP $MAINPID
PrivateTmp=true
PrivateDevices=true
ProtectHome=true
ProtectSystem=full
[Install]
WantedBy=multi-user.target