MQTT Problems with 0.101 release

Configuration
1

I am having serious problems with mqtt component when upgrading to 0.101.3 from 0.100.3.
Is this configuration correct related to the breaking changes listed in the 0.101 release?

mqtt:
  broker: !secret mqtt_broker
  username: !secret mqtt_username
  password: !secret mqtt_password
  discovery: true
  discovery_prefix: homeassistant

and about lights:

  - platform: mqtt
    name: "Faretti Ingresso"
    command_topic: "cmnd/sonoff-01/power"
    state_topic: "stat/sonoff-01/POWER"
    qos: 1
    payload_on: "ON"
    payload_off: "OFF"
    payload_available: "Online"
    payload_not_available: "Offline"
    retain: true
2

I donā€™t see anything wrong. What do you see in the log ?

3

I already opened a thread about this problem:

There there are the logsā€¦ actually downgraded to 0.100.3 to run HA.

4

Ok, I have over 25 tasmota devices and did not have any problems going to 0.101. But I run a mosquitto on bare metal, not a hassio add-on. And autodiscovery for all of them. But that does not help you. Sorry I canā€™t help better.

5

Ok thanks the sameā€¦ i run HA on venv and not hass.io

6

Then you must have a regular mosquitto install (as I do) . You donā€™t see anything in the mosquitto log ?

7

which is the right link to the log?

8

Open up a ssh connection

find / -name mosquitto.conf

(it is /etc/mosquitto/mosquitto.conf in my case)

nano /etc/mosquitto/mosquitto.conf

the log file be seen in the line
log_dest file /var/log/mosquitto/mosquitto.log

9

Strangeā€¦ i am running mosquitto since the beginning of HA but in this log i have only this:

1526335661: mosquitto version 1.4.15 (build date Sat, 07 Apr 2018 11:13:41 +0100) starting
1526335661: Config loaded from /etc/mosquitto/mosquitto.conf.
1526335661: Opening ipv4 listen socket on port 1883.
1526335661: Error: Address already in use

Whatā€™s the meaning ?

10

You might have to edit your mosquitto.conf file and add ā€˜log_type allā€™

11

done, and about username?

actually in the .conf i see user: mosquitto which is not the username i configured into configuration.yaml in the mqtt section

12

The line ā€˜address already in useā€™ could be that the built-in mqtt broker (deprecated) is starting before mosquitto does, and that could be the source of your problems.

13

Ok step by step i want to solve thisā€¦
What to do now?

following the docs i added this into /etc/systemd/system/[email protected]

[Unit]

Description=Home Assistant

After=network.target mosquitto.service

So in this way mosquitto must start before home assistant and also the built-in mqtt, or not?

14

Try this command, and see if mosquitto is listed:
systemctl --failed

15

no mosquitto listed here

If i check the status of the service i get:

ā— mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2019-11-11 17:41:22 CET; 19h ago
Docs: man:mosquitto.conf(5)
man:mosquitto(8)
Main PID: 492 (mosquitto)
Tasks: 1 (limit: 2200)
Memory: 2.0M
CGroup: /system.slice/mosquitto.service
ā””ā”€492 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

nov 12 12:09:43 hassbian mosquitto[492]: 1573556983: New client connected from ::1 as mosq/3aM0FPtsgV4P0J73td (p2, c1, k60, uā€™homeassistantā€™).
nov 12 12:12:39 hassbian mosquitto[492]: 1573557159: Socket error on client mosq/3aM0FPtsgV4P0J73td, disconnecting.
nov 12 12:13:01 hassbian mosquitto[492]: 1573557181: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
nov 12 12:25:34 hassbian mosquitto[492]: 1573557934: Client DVES_A76D14 has exceeded timeout, disconnecting.
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: New connection from 192.168.1.12 on port 1883.
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: New client connected from 192.168.1.12 as DVES_A76D14 (p2, c1, k15, uā€™homeassistantā€™).
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: Client DVES_A86610 has exceeded timeout, disconnecting.
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: New connection from 192.168.1.15 on port 1883.
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: New client connected from 192.168.1.15 as DVES_A86610 (p2, c1, k15, uā€™homeassistantā€™).
nov 12 12:43:02 hassbian mosquitto[492]: 1573558982: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.

16

I noticed that actually i have two users inside the /etc/mosquitto/pwfile and if i do:
ps -aux | grep mosquitto
i get:

mosquit+ 492 0.0 0.5 9508 5368 ? Ss nov11 0:46 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
pi 10603 0.0 0.0 7368 528 pts/1 S+ 12:59 0:00 grep mosquitto

Maybe is this that make the error ā€˜already in useā€™ ?
Do i need the user pi inside mosquitto.conf as in configuration.yaml i have the user homeassistant only?

17

Hi
I normal Debian/Ubuntu bare metal Mosquitto should have
Config in /etc/mosquitto/mosquitto.conf

This file will normally only define the location of PID file and log file and then it contains an include to /etc/mosquitto/conf.d

And inside conf.d you will find a file called default.conf

And inside of this you will have a path to the password file which is very likely to be /etc/mosquitto/passwd

This file also contains the ports that Mosquitto listens to for mqtt and websockets

Now there are two usernames in play. The username that mosquitto runs under in the Linux world. This is by default mosquitto. This is just a user with limited priviledges so we avoid running the daemon as root. It has nothing to do with clients connecting

It is in the passwd file you have the user that mqtt clients incl Home Assistant needs to login with.

The passwd file is with encrypted password but you can see which users have been setup (name before the colon)

This password file is created with the command

mosquitto_passwd -c passwordfile user

You can add more users with

mosquitto_passwd -b passwordfile user password

The commands create the file in the current working directory but you can move it afterwards if you create it in a wrong directory.

You need to restart Mosquitto to pick up the username/password changes

For you with the trouble - it sounds more like your Mosquitto stopped running independently of the HA update.

One thing I have not played with is to have TLS encryption between Mosquitto and clients. This require certificates. They can expire!
I see no reason to use TLS between my Sonoffs and Mosquitto as I do not expose my Mosquitto to the wild internet and it complicates debuggins and the ESP8266 based clients really have no resources to do this. So I assume you do not have TLS enabled.

Kenneth

18

Note for debugging Mosquitto

I encourage to learn to use the command line clients for Mosquotto

sudo apt-get install mosquitto-clients

The most useful command is

mosquitto_sub -h 192.168.1.15 -u USERNAME -P PASSWORD -v -t "#"

This subscribes to Mosquitto to all topics (replace USERNAME and PASSWORD and use correct IP host)

19

I am using mqtt component only for two lights and, running actually on 0.100.3, i hav no problems at all with mqtt. Problems begin when i try to upgrade directly to 0.101.3 as i get tons of errors in all the components starting with mqtt. So, being sure that mqtt is running ok, i am asked what changed in 0.101 release to get all those errors!
My first question was if i was running mosquitto broker other than the embedded one and finally i was sure itā€™s the mosquitto running.
Second was about username and password, but as i see also them are set corretly.
So i continue to ask me what changedā€¦

20

I run with Mosquitto on bare metal just like you.

But I run with HassIO in Docker using the standard alternative installation script for installing Hassio on Ubuntu Linux

But I do not run Mosquitto in a Home Assistant Addon. It is bare metal and I chose that from the start because I wanted future freedom to move the Mosquitto to alternative servers. Same as I have chosen to do with Deconz

I hope this can help you.

In Home Assistant I have this in configuration.yaml

mqtt:
  broker: localhost
  username: !secret mqtt_user
  password: !secret mqtt_password

And in secrets.yaml

mqtt_user: USERNAME
mqtt_password: PASSWORD

If the broker is not on same machine as Home Assistant or you run some strange VLANs etc then the syntax for the broker is to put the IP address. Example

broker: 192.168.1.5

I am on latest version of Home Assistant and have upgraded to every release since January 2019 when I started with HA. I have never seen a problem with this configuration.

Make sure you have a broker defined - otherwise you run a deprecated internal MQTT server.

Kenneth