MQTT Devices not visible as devices

phreaq · July 4, 2020, 11:38pm

I was having issues with some new MQTT devices showing up in HA, so I deleted the integration, restarted, HA prompted me set it up, and now it’s not finding any devices, including the ones that were found previously. I use the auto discovery option when configuring the integration

They are all sonoff devices, flashed to Tasmota, and I have setoption19 set to 1.

There’s no MQTT connection errors within the Tasmota Console (if I change the password, I see a failure to the MQTT broker. Fixing the password connects it back to the mqtt broker.

Here’s my mqtt broker log

[18:19:16] INFO: Setup mosquitto configuration
[18:19:16] INFO: No local user available
[18:19:17] INFO: Initialize Hass.io Add-on services
[18:19:17] INFO: Initialize Home Assistant discovery
[18:19:17] INFO: Start Mosquitto daemon
1593904757: mosquitto version 1.6.3 starting
1593904757: Config loaded from /etc/mosquitto.conf.
1593904757: Loading plugin: /usr/share/mosquitto/auth-plug.so
1593904757: |-- *** auth-plug: startup
1593904757: ├── Username/password checking enabled.
1593904757: ├── TLS-PSK checking enabled.
1593904757: └── Extended authentication not enabled.
1593904757: Opening ipv4 listen socket on port 1883.
1593904757: Opening ipv6 listen socket on port 1883.
1593904757: Opening websockets listen socket on port 1884.
1593904757: Opening ipv4 listen socket on port 8883.
1593904757: Opening ipv6 listen socket on port 8883.
1593904757: Opening websockets listen socket on port 8884.
1593904757: Warning: Mosquitto should not be run as root/administrator.
1593904764: New connection from 192.168.nn.nn on port 1883.
[INFO] found mqtt on Home Assistant
1593904765: New client connected from 192.168.nn.nn as PowerMeter1 (p2, c1, k30, u’mqtt’).
1593904765: New connection from 192.168.nn.nn on port 1883.
1593904765: New connection from 192.168.nn.nn on port 1883.
1593904765: New connection from 192.168.nn.nn on port 1883.
1593904765: New connection from 192.168.nn.nn on port 1883.
1593904765: New connection from 192.168.nn.nn on port 1883.
1593904765: New connection from 192.168.nn.nn on port 1883.
1593904765: New client connected from 192.168.nn.nn as PowerMeter2 (p2, c1, k30, u’mqtt’).
1593904765: New client connected from 192.168.nn.nn as PowerMeter3 (p2, c1, k30, u’mqtt’).
1593904765: New client connected from 192.168.8nn.nn as DVES_93219F (p2, c1, k30, u’mqtt’).
1593904765: New client connected from 192.168.nn.nn as DVES_335314 (p2, c1, k30, u’mqtt’).
1593904765: New client connected from 192.168.nn.nn as DVES_C70FDF (p2, c1, k30, u’mqtt’).
1593904765: New client connected from 192.168.nn.nn as PowerMeter4 (p2, c1, k30, u’mqtt’).

any thoughts or direction?

123 · July 5, 2020, 1:00am

Tell us more about your system. Are you running Home Assistant OS or Home Assistant Supervised? Is it the latest version (0.112.2)?

Did it stop working when you upgraded? In other words, what was the setup when it worked and what recently changed to cause it to stop working?

In the configuration of your Mosquitto MQTT Broker Add-on, is anonymous set to its default value false?

phreaq · July 5, 2020, 1:00am

Just to add, my Node-RED flows that reference the missing devices are working, so there’s communication between node-RED and my devices at least (NR is running on HA)

phreaq · July 5, 2020, 1:04am

I am running 112.2 on HA OS. I had everything working previously, with three devices.

Today, I added four more devices, but a restart did not discover them. I deleted the existing mqtt integration, rebooted, and the mqtt broker was discovered. I configured it (setting up auto discovery), and a reboot later, no devices at all, including the ones that previously shown.

123 · July 5, 2020, 1:09am

MQTT discovery works like this:

The device that wants to be discovered by Home Assistant must connect to your Mosquitto MQTT Broker Add-on and then publish a payload (describing the device) to a discovery topic.

So if a switch wants to be discovered, it must publish a JSON payload (which contains the switch’s name, the state_topic, etc) to a discovery topic that look something like this:

homeassistant/switch/my_switch/config

The first hurdle this switch must overcome is to login to the Mosquitto MQTT Broker. The broker’s default configuration disallows anonymous connections. So the device must supply a valid username and password.

Are these new devices configured to supply a valid username and password?

phreaq · July 5, 2020, 1:22am

but doesn’t that last part of my log show that the mqtt broker is seeing the devices? the ones listed are the ones that aren’t showing as devices.

and I did add a mqtt user name and password in the tasmota setup

123 · July 5, 2020, 2:02am

You never shared the names of the three new devices so whatever I or anyone sees in that log is unknown.

If you are certain they are connecting then, if Home Assistant is not automatically creating entities, it may mean those devices are not publishing their discovery topics (or publishing them incorrectly).

Is this username and password one of the Users you created in Home Assistant?

Configuration > Users

If it is then you should be able to use it with an MQTT client like MQTT Explorer to connect to the broker. MQTT Explorer will then display all of the topics the broker is currently managing. Among those topics you should find all the discovery topics for your devices. If they are not displayed, then it means the devices are either not connecting or not publishing to those topics. This is the simplest and surest way of confirming the user account works, the devices are connecting, and they are publishing to discovery topics properly.

Personally, I rely on MQTT Explorer to serve as an independent observer of the broker’s activities. It’s an excellent debugging tool.

nickrout · July 5, 2020, 6:25am

There are some firmwares that don’t send the discovery messages if they know that they have been discovered before.

So your tasmota devices may need

setoption19 0
setoption19 1

to force them to discover again.

phreaq · July 5, 2020, 2:15pm

DING! DING! DING! We have a winner!

setoption19 0
setoption19 1

The devices were showing up in MQTT Explorer, but flipping that setting made them show up in HA.

Thanks nickrout and Taras!

123 · July 5, 2020, 2:34pm

You’re welcome, I’m glad to hear it is working but … what you wrote makes no sense.

When you say:

The devices were showing up in MQTT Explorer

I assume you mean their discovery topics were shown in MQTT Explorer. If you can see their discovery topics using MQTT Explorer, so can Home Assistant.

It would make more sense if you had reported that you did not see the discovery topics until you toggled setoption19 and that caused them to appear. If they were present before you toggled the setoption19 then they are already accessible to Home Assistant (and can be used by it to create entities).

The last thing you should check is if the discovery topics contain retained messages (you can use MQTT Explorer to confirm it). If they do not, it means that whenever either Home Assistant or the MQTT Broker is restarted, the discovery topics will disappear. This will cause Home Assistant to either report the recently discovered entities as unavailable or simply remove them.

It’s highly likely they are being published as retained messages but, to avoid surprises later on, it’s best to confirm it now.

phreaq · July 5, 2020, 3:42pm

Thanks Taras, I’ll look into the retained messages aspect.

Now that I’ve re-added my devices, I also renamed some, and am having one last issue, not sure if it’s connected or not.

The ‘fan’ aspect of my ifan03 is not showing up correctly, as it’s still showing the old name. It’s also listed as a ‘fan’ integration, but I don’t have a reference to that in my configuration.yaml.

Where can I change the name for it?

123 · July 5, 2020, 4:31pm

When you click the red exclamation icon, what does it tell you?

phreaq · July 5, 2020, 4:34pm

“This entity does not have a unique ID, therefore its settings cannot be managed from the UI.”

123 · July 5, 2020, 4:51pm

If it doesn’t exist in your configuration.yaml file then it must exist in the entity registry file (in the hidden .storage directory) but without a unique_id. The lack of a unique_id makes it inaccessible to be managed by the frontend. That implies you will have to remove it manually by editing the entity registry. However, before you resort to that step, use MQTT Explorer to determine if there is a discovery topic dedicated to the creation of fan.sonoff_ir_fan. If there is, then you can use MQTT Explorer to purge that topic. If you don’t do this step, any attempt to edit the entity registry file will be wasted effort because Home Assistant will simply re-discover the the device.

phreaq · July 5, 2020, 5:03pm

I’m not too familiar with MQTT (just learning), let alone MQTT Explorer.

Here’s a screenshot of my ‘fan’ items within MQTT Explorer. it shows the sonoff fan as being offline.

123 · July 5, 2020, 6:08pm

All topics that begin with homeassistant represents “MQTT discovery topics” used by Home Assistant to create new entities. You’ll notice that under homeassistant/switch there are 4 sub-topics, one for each switch.

I see no homeassistant/fan topic so that means there is no discovery topic on the broker telling Home Assistant to create a fan (like fan.sonoff_ir_fan). Sorry, I’m not a mindreader so I have no idea what you may have done in the past to cause Home Assistant to create the entity fan.sonoff_ir_fan. If it’s not defined in configuration.yaml then the only other place it can be is in the entity registry … which is unusual if the entity has no unique_id (because the entity registry typically only records entities that have unique identifiers).

If you are absolutely certain that entity doesn’t exist in configuration.yaml then you will need to check if it is in the entity registry. If it is there then you can edit it out BUT the problem is the file’s contents are in JSON, it’s best to shutdown Home Assistant before editing the file, and if you make a mistake and corrupt the JSON structure, Home Assistant will be unhappy to say the least (and so will you if on restart it loses all your existing entities).

__
NOTE
Given the consequences of a mistake, I hesitate to explain how to modify the entity registry, especially to anyone who has no experience with modifying data in JSON format.

phreaq · July 5, 2020, 6:36pm

found it!

I found reference to it in /config/packages/fan_package.yaml, a custom front end. I was able to rename it there, and all is good, thanks.

123 · July 5, 2020, 7:05pm

Glad to hear you found it.

The alternative would have been messy.

fistowutini · April 25, 2021, 10:18pm

This was my problem. I apparently started with a lite build, and then ran into more problems trying to get it upgraded (had to flash to minimal then to full build). Switching from lite to the full build resulted in the device immediately appearing in HA integrations under Tasmota.

mcfly76 · October 20, 2021, 3:01pm

Hi all, sorry to resume this old thread, but I think I have a similar issue, with the difference that all the suggestions mentioned above didn’t work for me.
I have a remote Zigbee2Tasmota device, with a NodeMCU connected to a CC2530+CC2591 zigbee module, alongside a more traditional Zigbee2mqtt setup in my house.
I have problems with the former, which has a motion sensor connected to it, which results not available in HA.

Here is a my binary sensor configuration in HA:

- platform: mqtt
  name: "Motion Scala Garage"
  off_delay: 20
  state_topic: "tele/tasmota_garage/SENSOR"
  availability_topic: "tele/tasmota_garage/LWT"
  value_template: "{{ value_json.ZbReceived.0x82DA.Occupancy }}"
  device_class: "motion"
  qos: 1

And here is the problem with that binary sensor:

I see the correct status messages and updates in MQTT Explorer:

I also checked in the entity_registry file, but there’s no reference to that binary sensor.
I tried the SetOption19 in Tasmota, but basically made the entire device disappear in HA.

What else could I try?
Thank you!