MQTT Problems with 0.101 release

You might have to edit your mosquitto.conf file and add ‘log_type all’

done, and about username?

actually in the .conf i see user: mosquitto which is not the username i configured into configuration.yaml in the mqtt section

The line ‘address already in use’ could be that the built-in mqtt broker (deprecated) is starting before mosquitto does, and that could be the source of your problems.

Ok step by step i want to solve this…
What to do now?

following the docs i added this into /etc/systemd/system/[email protected]

[Unit]

Description=Home Assistant

After=network.target mosquitto.service

So in this way mosquitto must start before home assistant and also the built-in mqtt, or not?

Try this command, and see if mosquitto is listed:
systemctl --failed

no mosquitto listed here

If i check the status of the service i get:

● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2019-11-11 17:41:22 CET; 19h ago
Docs: man:mosquitto.conf(5)
man:mosquitto(8)
Main PID: 492 (mosquitto)
Tasks: 1 (limit: 2200)
Memory: 2.0M
CGroup: /system.slice/mosquitto.service
└─492 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

nov 12 12:09:43 hassbian mosquitto[492]: 1573556983: New client connected from ::1 as mosq/3aM0FPtsgV4P0J73td (p2, c1, k60, u’homeassistant’).
nov 12 12:12:39 hassbian mosquitto[492]: 1573557159: Socket error on client mosq/3aM0FPtsgV4P0J73td, disconnecting.
nov 12 12:13:01 hassbian mosquitto[492]: 1573557181: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
nov 12 12:25:34 hassbian mosquitto[492]: 1573557934: Client DVES_A76D14 has exceeded timeout, disconnecting.
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: New connection from 192.168.1.12 on port 1883.
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: New client connected from 192.168.1.12 as DVES_A76D14 (p2, c1, k15, u’homeassistant’).
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: Client DVES_A86610 has exceeded timeout, disconnecting.
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: New connection from 192.168.1.15 on port 1883.
nov 12 12:25:35 hassbian mosquitto[492]: 1573557935: New client connected from 192.168.1.15 as DVES_A86610 (p2, c1, k15, u’homeassistant’).
nov 12 12:43:02 hassbian mosquitto[492]: 1573558982: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.

I noticed that actually i have two users inside the /etc/mosquitto/pwfile and if i do:
ps -aux | grep mosquitto
i get:

mosquit+ 492 0.0 0.5 9508 5368 ? Ss nov11 0:46 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
pi 10603 0.0 0.0 7368 528 pts/1 S+ 12:59 0:00 grep mosquitto

Maybe is this that make the error ‘already in use’ ?
Do i need the user pi inside mosquitto.conf as in configuration.yaml i have the user homeassistant only?

Hi
I normal Debian/Ubuntu bare metal Mosquitto should have
Config in /etc/mosquitto/mosquitto.conf

This file will normally only define the location of PID file and log file and then it contains an include to /etc/mosquitto/conf.d

And inside conf.d you will find a file called default.conf

And inside of this you will have a path to the password file which is very likely to be /etc/mosquitto/passwd

This file also contains the ports that Mosquitto listens to for mqtt and websockets

Now there are two usernames in play. The username that mosquitto runs under in the Linux world. This is by default mosquitto. This is just a user with limited priviledges so we avoid running the daemon as root. It has nothing to do with clients connecting

It is in the passwd file you have the user that mqtt clients incl Home Assistant needs to login with.

The passwd file is with encrypted password but you can see which users have been setup (name before the colon)

This password file is created with the command

mosquitto_passwd -c passwordfile user

You can add more users with

mosquitto_passwd -b passwordfile user password

The commands create the file in the current working directory but you can move it afterwards if you create it in a wrong directory.

You need to restart Mosquitto to pick up the username/password changes

For you with the trouble - it sounds more like your Mosquitto stopped running independently of the HA update.

One thing I have not played with is to have TLS encryption between Mosquitto and clients. This require certificates. They can expire!
I see no reason to use TLS between my Sonoffs and Mosquitto as I do not expose my Mosquitto to the wild internet and it complicates debuggins and the ESP8266 based clients really have no resources to do this. So I assume you do not have TLS enabled.

Kenneth

Note for debugging Mosquitto

I encourage to learn to use the command line clients for Mosquotto

sudo apt-get install mosquitto-clients

The most useful command is

mosquitto_sub -h 192.168.1.15 -u USERNAME -P PASSWORD -v -t "#"

This subscribes to Mosquitto to all topics (replace USERNAME and PASSWORD and use correct IP host)

I am using mqtt component only for two lights and, running actually on 0.100.3, i hav no problems at all with mqtt. Problems begin when i try to upgrade directly to 0.101.3 as i get tons of errors in all the components starting with mqtt. So, being sure that mqtt is running ok, i am asked what changed in 0.101 release to get all those errors!
My first question was if i was running mosquitto broker other than the embedded one and finally i was sure it’s the mosquitto running.
Second was about username and password, but as i see also them are set corretly.
So i continue to ask me what changed…

I run with Mosquitto on bare metal just like you.

But I run with HassIO in Docker using the standard alternative installation script for installing Hassio on Ubuntu Linux

But I do not run Mosquitto in a Home Assistant Addon. It is bare metal and I chose that from the start because I wanted future freedom to move the Mosquitto to alternative servers. Same as I have chosen to do with Deconz

I hope this can help you.

In Home Assistant I have this in configuration.yaml

mqtt:
  broker: localhost
  username: !secret mqtt_user
  password: !secret mqtt_password

And in secrets.yaml

mqtt_user: USERNAME
mqtt_password: PASSWORD

If the broker is not on same machine as Home Assistant or you run some strange VLANs etc then the syntax for the broker is to put the IP address. Example

broker: 192.168.1.5

I am on latest version of Home Assistant and have upgraded to every release since January 2019 when I started with HA. I have never seen a problem with this configuration.

Make sure you have a broker defined - otherwise you run a deprecated internal MQTT server.

Kenneth

Actually i run mosquitto in the same machine where it lies home assistant and my configs are:

In configuration.yaml i have

mqtt:
  broker: !secret mqtt_broker
  username: !secret mqtt_username
  password: !secret mqtt_password
  discovery: true
  discovery_prefix: homeassistant

And about lights i have:

  - platform: mqtt
    name: "Faretti Ingresso"
    command_topic: "cmnd/sonoff-01/power"
    state_topic: "stat/sonoff-01/POWER"
    qos: 1
    payload_on: "ON"
    payload_off: "OFF"
    payload_available: "Online"
    payload_not_available: "Offline"
    retain: true

So it means i have a wrong configuration as the secret broker is the IP address of my raspberry?

The problem here is that you are asking the same/similar question in many different places and as a result you are getting many different disparate answers because no one knows about the other conversations going on in 4 different threads.

I really think you need to just stick to using one thread to try to help you figure stuff out and things will be far less confusing for others who are trying to help you and especially for you trying to follow suggestions from 4 different threads at the same time.

You are right… Sorry… I’ll continue, eventually, only the thread about 0.101 release hoping all my problems will be solved.

Is the log in this post from the working .100 or the non working .101 setup MQTT Problems with 0.101 release

Non working .101 setup

@maurizio53
I was reading another thread related to MQTT and I noticed this detail

If you look at https://www.home-assistant.io/addons/mosquitto/ then it says

This add-on is attached to the Home Assistant user system, so mqtt clients can make use of these credentials. Local users may also still be set independently within the configuration options for the add-on. For the internal Hass.io ecosystem we register ‘homeassistant’ and ‘addons’ , so these may not be used as user names.

I can see in your logs that indeed you have chosen the username “homeassistant”

Maybe that creates your trouble. I do not know if this constraint was added recently. I am not a dev. I just noted this and remember that you used that user name in your logs

The link provided relates to using the Mosquitto Addon and not Mosquitto on bare metal. I just wonder if the integration code with external could have similar issue. It is worth a try.

To change user name use the mosquitto_passwd application. You cannot just hack the user name in the password file

can you show me an example about how to change username into mosquitto?

I already had this in previous post

Note this is related to running Mosquitto on bare metal. The Addon is doing it for you via a UI

The command is

sudo mosquitto_passwd -b passwordfile user password

You have to be in the same directory as the existing password file. See my earlier posts.

The command adds a new user and leaves the existing homeassistant user

You can remove the old by just deleting the line from the file.

And remember that all your MQTT devices need to be setup to use the new username if you delete the homeassistant user

You have to restart Mosquitto to activate the new user

sudo systemctl restart mosquitto

Kenneth

He’s not using an add-on. He is running HA in a venv, not Hassio.