Mosquitto problems on HA manual venv install on Pi

Configuration
,
1

Migrating from Hassbian, I decided to go the venv route with Raspbian Lite and install HA v0.101.3 (not Hassio). HA is working fine, however, after setting up Mosquitto MQTT locally I hit a wall.

For info, I installed Mosquitto using the following method:

sudo apt-get install mosquitto -y
sudo apt-get install mosquitto-clients -y

Edited the configuration file:
sudo nano /etc/mosquitto/mosquitto.conf

Removed the whole ‘include_dir’ line, and replaced with:

allow_anonymous false
password_file /etc/mosquito/pwfile
listener 1883

Created the password file:
sudo mosquitto_passwd -c /etc/mosquitto/pwfile UserMQTT
and gave a password.

Then I edited the auto-start script for HA to add the service in:
sudo nano /etc/systemd/system/[email protected]

In the [Unit} section changed the second line to read as:
After=network-online.target mosquitto.service

Saved and ran:
systemctl status mosquitto.service
And at first, it worked -said it was running.

Then I ‘sudo rebooted’ and the problems started. After logging in I tested the install with:
sudo systemctl status mosquitto

And it came up with an error:

image
image869×418 22.2 KB

I tried:
sudo systemctl start mosquitto
and got:
Job for mosquitto.service failed because the control process exited with error code.
See “systemctl status mosquitto.service” and “journalctl -xe” for details.

When I tried the log:

mosquitto -c /etc/mosquitto/mosquitto.conf
1573511140: Error: Unable to open log file /var/log/mosquitto/mosquitto.log for writing.

journalctl -u mosquitto

image
image909×594 59.5 KB

I also tried to reinstall it over itself but this didn’t change the conf. So the service won’t run from either an auto start or manually. I’m not sure what I’ve done wrong?

Is it that the broker must be installed inside the venv? :thinking:
Is it a permissions thing?
Or have I missed something obvious?

Any help is much appreciated :slightly_smiling_face:

2

I haven’t installed mosquitto in a while but back when I originally set up my HA in a venv on the RPi, and even later on when i re-installed in both a venv and docker on a mini-pc, I followed the following procedure to install it and everything ran just fine with no need to mess with any services configuration.

$ sudo apt-get install mosquitto 
$ sudo apt-get install mosquitto-clients 

Edit the config file:

$ sudo nano /etc/mosquitto/mosquitto.conf 

Add the following:

allow_anonymous false
password_file /etc/mosquitto/pwfile
listener 1883 

comment out the existing line:

# include_dir /etc/mosquitto/conf.d

- save

Create a username:

$ sudo mosquitto_passwd -c /etc/mosquitto/pwfile <username>

It will then ask for a password
3

You certainly do not need to install in the venv. The apt-get packages are fine.

4

Thanks. I repeated the above steps but the problem remains. But it’s also reassuring to know that installing it outside the venv was the right thing to do.

I’ve looked into the mosquitto.service itself:
sudo nano /lib/systemd/system/mosquitto.service

and this is the result:

[Unit]
Description=Mosquitto MQTT v3.1/v3.1.1 Broker
Documentation=man:mosquitto.conf(5) man:mosquitto(8)
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
NotifyAccess=main
ExecStart=/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
Restart=on-failure

[Install]
WantedBy=multi-user.target

Does this look right?

I also wondered if an embedded broker in HA or Raspian could be active somehow and causing a conflict, so I used Fing on my Android phone to check my IP of the Pi for open ports but 1883 wasn’t open, so that rules that one out.

1 Like
5

Did you read all the instructions, including this

If you are running a Mosquitto instance on the same server as Home Assistant then you must ensure that the Mosquitto service starts before Home Assistant. For a Linux instance running Systemd (Raspberry Pi, Debian, Ubuntu and others) then you should edit the file /etc/systemd/system/[email protected] as root (e.g., sudo nano /etc/systemd/system/[email protected] ) and add the Mosquitto service:

[Unit]
Description=Home Assistant
After=network.target mosquitto.service
6

I’m not saying that nobody needs to do that but I never needed to do that on any system I’ve installed mosqiutto on with HA running in a venv.

And that would (should…) only be a concern after a reboot of the host machine. But after just a restart of HA (without rebooting) then everything should work even without that line since the mosqiutto service will already be running by that time. If it’s not then it’s a problem with Mosquitto not a problem with the HA service directive.

I personally wouldn’t put that line in the HA service call because if the mosquitto service fails to run for some reason then HA won’t start either.

Figure out why mosquitto isn’t running first.

7

I did put that line in, and HA still starts only it takes a bit longer because of the error.

8

Omg, it turns out I did miss something obvious - a typo. And I was so sure I doubled-check it. In the config file I put ‘mosquito’ for the pwfile - so when I set the user up obviously it would fail. I was totally at a loss over this, all the advice and research pointed to this IS the correct method without really any variation and so should’ve (would’ve) worked. A muppetation on my part, sorry. :face_with_hand_over_mouth:

Fixing the .conf file and re-doing the:
sudo mosquitto_passwd -c /etc/mosquitto/pwfile username
and
sudo systemctl start mosquitto
…did the trick. A test reboot and I’m pleased to report it’s up and running nicely now.

Thank you for the support guys :+1:

2 Likes
9

Hi Folks,
I have exactly the same problem as described, and the same configuration (Home Assistant on venv on Pi4, MQTT installed via console with the same method as above. I have also zigbee2mqtt configured. Everything was fine and working great, till the first Pi reboot. After that - device entities in HA not available, zigbee2mqtt cant connect to mqtt server, and mqtt gives me identical errors like above… The difference is I do not have any typo… that’s already checked… Where to look please ?

This is what I’m getting right after reboot of Pi.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Tue Nov  3 14:36:29 2020
pi@raspberrypi:~ $ sudo systemctl status mosquitto
● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
   Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2020-11-03 14:36:24 CET; 4min 10s ago
     Docs: man:mosquitto.conf(5)
           man:mosquitto(8)
  Process: 559 ExecStart=/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf (code=exited, status=3)
 Main PID: 559 (code=exited, status=3)

lis 03 14:36:24 raspberrypi systemd[1]: Failed to start Mosquitto MQTT v3.1/v3.1.1 Broker.
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Service RestartSec=100ms expired, scheduling restart.
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Scheduled restart job, restart counter is at 5.
lis 03 14:36:24 raspberrypi systemd[1]: Stopped Mosquitto MQTT v3.1/v3.1.1 Broker.
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Start request repeated too quickly.
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Failed with result 'exit-code'.
lis 03 14:36:24 raspberrypi systemd[1]: Failed to start Mosquitto MQTT v3.1/v3.1.1 Broker.
pi@raspberrypi:~ $ sudo journalctl -u mosquitto.service -f
-- Logs begin at Tue 2020-11-03 14:36:18 CET. --
lis 03 14:36:24 raspberrypi mosquitto[559]: Error: Unable to open configuration file.
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Failed with result 'exit-code'.
lis 03 14:36:24 raspberrypi systemd[1]: Failed to start Mosquitto MQTT v3.1/v3.1.1 Broker.
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Service RestartSec=100ms expired, scheduling restart.
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Scheduled restart job, restart counter is at 5.
lis 03 14:36:24 raspberrypi systemd[1]: Stopped Mosquitto MQTT v3.1/v3.1.1 Broker.
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Start request repeated too quickly.
lis 03 14:36:24 raspberrypi systemd[1]: mosquitto.service: Failed with result 'exit-code'.
lis 03 14:36:24 raspberrypi systemd[1]: Failed to start Mosquitto MQTT v3.1/v3.1.1 Broker.

What is wierd, when I do manuall from the console mosquitto -v , then everything suddenly starts working fine. So something is messed up with starting it as a service… But where to look ?

10

Ok so somehow I managed to get the status look a bit better:

pi@raspberrypi:/ $ sudo systemctl status mosquitto.service
● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
   Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2020-11-03 19:05:30 CET; 20min ago
     Docs: man:mosquitto.conf(5)
           man:mosquitto(8)
 Main PID: 1622 (mosquitto)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/mosquitto.service
           └─1622 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

lis 03 19:05:30 raspberrypi systemd[1]: Starting Mosquitto MQTT v3.1/v3.1.1 Broker...
lis 03 19:05:30 raspberrypi systemd[1]: Started Mosquitto MQTT v3.1/v3.1.1 Broker.
pi@raspberrypi:/ $

But still no joy.If i run mqtt by hand by:
mosquitto -v
mosquitto -c /etc/mosquitto/mosquitto.conf
mosquitto -c /etc/mosquitto/conf.d/default.conf

It starts, runs and connects with Home Assistant and zigbee2mqtt, and entities are avaliable (sensors)
However if I do this by
sudo systemctl start mosquito.service
or simply by restart a whole Pi server, it shows what I put above. It looks like running, but something is wrong as it can’t connect.
However 2 - if in HA I go to configure, integrations, mqtt then configure - it finds broker with no problems… but zigbee and entities doesnt.