No detection of new mqtt devices

zumheulen · July 25, 2022, 1:47pm

Hello guys, since I have not needed any mqtt device integration so far, I have only now become aware of the problem. As a result, I imported a year-old backup and even with a new one I don’t get any devices in the mqtt integration while the broker receives the devices. I don’t know what my problem is…

The broker runs in HASSIO with a username and password (because without it you get an error message!) - No certificates and customize standard (active false + folder mosquitto) - Network ports are all unchanged.

The MQTT integration has the settings:
Broker: core-mosquitto (tested unsuccessfully with HASSIO IP)
Port: 1883
User: MQTT_user (Same as Broker and everywhere in MQTT devices)
Password: (As in brokers and devices, consisting of upper & lower case letters)
MQTT options:
All standard, except for the topic: ha/status instead of homeassistant/status.
Detection is of course activated.

I’m at a loss. Could it be because of the duckdns access in connection with https://IP?
I found some on the net with this problem, but no solution.

stevemann · July 25, 2022, 5:08pm

Do you have discovery: true in your configuration?

This is from my mqtt.yaml:

#mqtt:  
  broker: !secret mqtt_broker_ip
  port: 1883
  client_id: home-assistant-1
  discovery: true
  discovery_prefix: homeassistant

And in mt configuration.yaml:

mqtt: !include mqtt.yaml

zumheulen · July 25, 2022, 7:35pm

I just went to the trouble of creating a Mosquitto Broker via Docker on my Synology and the same thing here too:
Broker is running and devices are busy sending BUT this damn MQTT integration on HomeAssistant neither finds the devices nor did it delete the old ones before mine…

integration1
integratin2

stevemann · July 26, 2022, 1:39am

Discovery is finding old entities?
If your broker has retained messages for the old devices, HA will always “discover” them.

My broker is on a separate Raspberry Pi3 (my home assistant is on an Intel NUC). Send a null message to the topics that you want to remove the retained messages for. The sure fire way that I have found is the delete the persistence file. Deleting the persistence file can be easily done with just a few commands:

sudo service mosquitto stop
sudo rm /var/lib/mosquitto/mosquitto.db
sudo service mosquitto start

I don’t know if this would work on the Home Assistant hosted Mosquitto broker.

nickrout · July 26, 2022, 6:17am

Are your devices actually sending discovery messages?

zumheulen · July 26, 2022, 8:55am

I just reinstalled the broker on HA now. Curiously, I was able to start it in advance without a password. It didn’t work yesterday.
The IP 192.168.1.65 is my HASSIO where the broker runs on it.
Here is the excerpt from the broker:
logins:

username: MQTT_user (2 spaces dash spaces username:)
password: myPassword
require_certificate: false
certfile: fullchain.pem
keyfile: privkey.pem
customize:
active: false
folder: mosquitto
anonymous: false

[s6-init] making user provided files available at /var/run/s6/etc…exited 0.
[s6-init] ensuring user provided files have correct perms…exited 0.
[fix-attrs.d] applying ownership & permissions fixes…
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts…
[cont-init.d] mosquitto.sh: executing…
[10:57:59] INFO: Setting up user MQTT_user
[10:58:00] INFO: Certificates found: SSL is available
[10:58:02] INFO: Starting NGINX for authentication handling…
[cont-init.d] mosquitto.sh: exited 0.
[cont-init.d] nginx.sh: executing…
[cont-init.d] nginx.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[10:58:03] INFO: Starting mosquitto MQTT broker…
1658825883: Warning: Mosquitto should not be run as root/administrator.
[10:58:05] INFO: Successfully send discovery information to Home Assistant.
[10:58:06] INFO: Successfully send service information to the Supervisor.
1658825883: mosquitto version 2.0.11 starting
1658825883: Config loaded from /etc/mosquitto/mosquitto.conf.
1658825883: Loading plugin: /usr/share/mosquitto/go-auth.so
1658825883: ├── Username/password checking enabled.
1658825883: ├── TLS-PSK checking enabled.
1658825883: └── Extended authentication not enabled.
1658825883: Opening ipv4 listen socket on port 1883.
1658825883: Opening ipv6 listen socket on port 1883.
1658825883: Opening websockets listen socket on port 1884.
1658825883: Opening ipv4 listen socket on port 8883.
1658825883: Opening ipv6 listen socket on port 8883.
1658825883: Opening websockets listen socket on port 8884.
1658825883: mosquitto version 2.0.11 running
1658825884: New connection from 127.0.0.1:41046 on port 1883.
1658825884: Client closed its connection.
1658825888: New connection from 192.168.3.7:51375 on port 1883.
1658825888: New connection from 192.168.3.8:64023 on port 1883.
1658825888: New client connected from 192.168.3.7:51375 as esp8266-913F2B (p2, c1, k30, u’MQTT_user’).
1658825888: New client connected from 192.168.3.8:64023 as esp8266-913BB0 (p2, c1, k30, u’MQTT_user’).
1658825893: New connection from 192.168.1.65:53844 on port 1883.
1658825893: New client connected from 192.168.1.65:53844 as nodered-mqtt (p2, c1, k15, u’MQTT_user’).
1658825895: New connection from 192.168.1.65:56035 on port 1883.
1658825895: New connection from 192.168.3.69:55420 on port 1883.
1658825895: New client connected from 192.168.3.69:55420 as Buegelstation (p2, c1, k30, u’MQTT_user’).
1658825895: New client connected from 192.168.1.65:56035 as 38kPBckDn4P7HnpzKM2pkf (p2, c1, k60, u’MQTT_user’).
1658825895: New connection from 192.168.3.103:64564 on port 1883.
1658825895: New client connected from 192.168.3.103:64564 as Hauswasserwerk (p2, c1, k30, u’MQTT_user’).
1658825895: New connection from 192.168.3.66:55071 on port 1883.
1658825895: New client connected from 192.168.3.66:55071 as Raspberry (p2, c1, k30, u’MQTT_user’).
1658825895: New connection from 192.168.3.106:63609 on port 1883.
1658825895: New client connected from 192.168.3.106:63609 as Warmwasserpumpe (p2, c1, k30, u’MQTT_user’).
1658825896: New connection from 192.168.3.148:63331 on port 1883.
1658825896: New client connected from 192.168.3.148:63331 as Poolpumpe (p2, c1, k30, u’MQTT_user’).
1658825896: New connection from 192.168.3.81:53113 on port 1883.
1658825896: New client connected from 192.168.3.81:53113 as DeWalt-Acculader (p2, c1, k30, u’MQTT_user’).
1658825896: New connection from 192.168.3.102:51558 on port 1883.
1658825896: New client connected from 192.168.3.102:51558 as Multimedia (p2, c1, k30, u’MQTT_user’).
1658825896: New connection from 192.168.3.68:49260 on port 1883.
1658825896: New client connected from 192.168.3.68:49260 as Reserve2 (p2, c1, k30, u’MQTT_user’).
1658825896: New connection from 192.168.3.60:63413 on port 1883.
1658825896: New client connected from 192.168.3.60:63413 as Bikelader (p2, c1, k30, u’MQTT_user’).
1658825939: New connection from 172.30.32.2:49414 on port 1883.
1658825939: Client closed its connection.
1658825996: New connection from 192.168.3.131:51604 on port 1883.
1658825996: New connection from 192.168.3.189:61581 on port 1883.
1658825996: New connection from 192.168.3.235:63291 on port 1883.
1658825996: Client disconnected, not authorised.
1658825996: Client disconnected, not authorised.
1658825996: Client disconnected, not authorised.
1658825999: New connection from 192.168.3.6:56375 on port 1883.
1658825999: Client disconnected, not authorised.
1658826059: New connection from 172.30.32.2:49444 on port 1883.
1658826059: Client closed its connection.
1658826083: New connection from 192.168.3.6:64367 on port 1883.
1658826083: New client connected from 192.168.3.6:64367 as Belegtanzeige-Garage-links (p2, c1, k30, u’MQTT_user’).
1658826120: New connection from 192.168.3.189:64487 on port 1883.
1658826120: New connection from 192.168.3.131:56035 on port 1883.

nickrout · July 26, 2022, 11:17am

If you are using the mosquitto addon, then do not create the user or passwd in the addon. Create a home assistant user and use that username and password to access the broker.

Also, please answer this

zumheulen · July 26, 2022, 1:43pm

I thought you meant what reports to the broker…
Ok, if I enter # in the addon and listen, a lot of data comes in.

So you mean e.g. create the user MQTT_user with password xyz like with the broker BUT leave out the user and password in the addon??

I’ll test that right away…

zumheulen · July 26, 2022, 1:55pm

In the addon you have to enter a username and password. Did you mean in the broker? :-/
Although a lot of data is coming in, I still don’t see any devices/entities such as the smartthings example below.
They don’t appear in the overview either so that I could work with them. mqtt geraete

zumheulen · July 26, 2022, 1:57pm

stevemann:

Do you have discovery: true in your configuration?

This is from my mqtt.yaml:
#mqtt:  
  broker: !secret mqtt_broker_ip
  port: 1883
  client_id: home-assistant-1
  discovery: true
  discovery_prefix: homeassistant
And in mt configuration.yaml:
mqtt: !include mqtt.yaml

Oops… do you need that? :-o

stevemann · July 26, 2022, 2:10pm

I AM NOT AN EXPERT!
But this code works here and all new MQTT devices are discovered.

zumheulen · July 26, 2022, 3:59pm

I have now taken the trouble to set up a new instance of HASSIO.

Broker installed WITHOUT user and password
MQTT addon like HASSIO suggests with a different username and password and data is already coming from the broker…
Then username and password as well as IP of HASSIO.
So everything as described on the official HomeAssistant website and shown in many YT videos.

Regardless, no entities appear.
I know I found this earlier on the overview and was able to include it. Adding to mqtt.yaml as suggested by you didn’t change anything either. That’s why I’m now assuming an error on the part of HomeAssistant.

stevemann · July 26, 2022, 7:55pm

For what it’s worth.

My MQTT broker is on a stand-alone Raspberry Pi3. Mosquitto is its only purpose. (This arrangement predates Home Assistant here). No usernames or passwords. Any topic that I publish to the broker’s IP is accepted by the broker.
I do not have the MQTT add-on in my Home Assistant. I don’t even know what it does- my configuration just works.
If I create a new MQTT device in mqtt.yaml and restart Home Assistant, it gets discovered as an entity.

Try posting your configuration and mqtt.yaml if you have split your mqtt stuff to an include file.

zumheulen · July 26, 2022, 10:23pm

The MQTT addon provides the respective entities of the devices. So if a sensor measures temperature and humidity, these two entities appear in this addon and you can work with them immediately like with a light, switch or similar.
I don’t have anything in my mqtt.yaml. except as a test of this access data. Also for what? If I don’t know what I’m getting from the broker? I’m not a Stone Age programmer. But I have neither the desire nor the time to type any line-long programs. There was another way and if these entities don’t appear again in the foreseeable future, the topic is over, I don’t use something like that before, sorry :-))
Ah… that reminds me, maybe the data will appear in NodeRed?

By the way, in this video you can see how the MQTT Etities are listed, that’s how it should be
MQTT Addon entities in HASSIO

nickrout · July 26, 2022, 10:36pm

I have asked you before whether your devices are providing discovery data, you never replied. So I’ll be more specific:

Please show the output of the following, not a picture, cut and paste it please.

mosquitto_sub -h yourhassipaddress -t 'homeassistant/#' -v

nickrout · July 26, 2022, 10:39pm

By the way, you do not configure the mqtt integration in yaml unless there is a specific special reason to do so

It’s right there in the docs

Some broker configuration options can’t be set via the user interface, but require changes of your configuration.yaml file. This includes configuring SSL certificate options.

I haven’t seen any need to do a yaml config for the mqtt integration in your case.

zumheulen · July 26, 2022, 10:43pm

nickrout:

I have asked you before whether your devices are providing discovery data, you never replied. So I’ll be more specific:

Please show the output of the following, not a picture, cut and paste it please.
mosquitto_sub -h yourhassipaddress -t 'homeassistant/#' -v

Sorry, but I have absolutely no idea what to do with these lines… :-/
For me it would be mosquitto_sub -h 192.168.1.65 -t ‘homeassistant/#’ -v
But where should I enter that?

nickrout · July 26, 2022, 10:45pm

In the ssh client connected to your HA instance, or in fact in any (linux) computer on your lan.

zumheulen · July 26, 2022, 10:50pm

I almost thought…
In the terminal comes the connection error: connectin Refused: Not authorized.
On my LinuxMint PC comes:

The command ‘mosquitto_sub’ was not found but can be installed with:

sudo snap install mosquitto # version 2.0.14, or
sudo apt install mosquitto-clients # version 1.6.9-1

Information about new versions can be found with ‘snap info mosquitto’.

nickrout · July 26, 2022, 11:01pm

Easiest in terminal then.

Add the username and password you set up in HA

mosquitto_sub -h yourhassipaddress -u username -P password -t 'homeassistant/#' -v