MQTT not working?

Hi

Discovery is set “on” under MQTT (Mosquitto) and i can see it is actively receiving messages, as shown above. (MQTT, three dots “Activate recently added entities” + “Activate automatic retrieval of updates”) both set to on.

Full disclosure.

I had this working but after updating Mosquitto i (And a lot of others) experienced that the Solar Assistant (And other MQTT integrations) stopped working.

I then put it on the backburner for a while.

My SD Card failed me, i did a clean install of HA, running off of a SSD instead.

Loaded the latest snapshot.

MQTT was apparantly not included in the snapshot, even though it was installed at the time i took the snapshot

Installed MQTT and made the exact same setup as the first time (Solar Assistant has an extensive setup guide).

I am getting the messages into Mosquitto, and if memory serves, last time, as soon as these messages started appearing, the entities were automatically added to HA, but this time nothing happens, i dont see the sensors for the various stuff that it receives via MQTT ion the entities list and any of the previously made cards with these sensors shown still says that the sensors are not there.

Right, that’s a good clue. Which version are you running?

I hope I’m recalling this correctly now, but I saw a post about a bug in 2.0.14 which was fixed in 2.0.15. I’m running 2.0.11.

EDIT: Bug is in 2.0.15. Stick with 2.0.14.

1 Like

Hi

I am confused as to what these version numbers refer to.

As far as i can see these are the software versions used

Solar Assistant : 2023-02-28

Mosquitto broker - Current version: 6.2.1

Could you please tell me if this version 2.0.15 is some other software or if i am just being not very bright :slight_smile:

The number of the add-on has nothing to do with the mosquitto version packaged.

1 Like

Aah i see, it seems i must downgrade to HA version 6.13, as they state in 6.20 changelog “Updated to 2.0.15”

So now i only need to figure out how to “downgrade” these in HA…

Also…Wow, 2.0.15 has a bug, but was released almost a year ago.

I wonder if that bug will ever get fixed…

Apologies, but you figured it out (was in a rush when I replied). :slight_smile:

1 Like

No worries, i am not expert here, so i am greatful for all help i can get.

My only remaining problem is, it seems the only way to “downgrade” the MQTT integration is to load it from before it was upgraded, but since my install is new, i do not have this in the backup list.

Just waiting for a fix for the Mosquitto MQTT does not seem to be an option either, it seems it was almost a year since they last updated this MQTT.

Do you have any advice on what i might do.

My installation is Home Assistant OS, directly on a Raspberry Pi.

It might work:

Im a novice, this sounds a bit out of my league

I probably have snapshots that are from when the earlier version was installed.

Is it an option to open up this tar.gz backup, and find the partial backup from back when it was updated, or does a full backup not include these partial backups as part of the full backup ??

If yes, where does it store those in the tar.gz file

You don’t have to open the tar, just do a restore, choose partial restore, just check mqtt

1 Like

Thanks to everyone, this seems to be a roundabout problem.

I found an older version of Mosquitto inside a previous full backup.

Rolled back, and now i am running 6.1.13 which contains the .11 Mosquitto.

But now the problem is, Mosquitto refuses to start and throws an error

“Error: Unable to create websockets listener on port 1884”

Did a bit of googling and found that the reason they upgraded Mosquitto in the first place was because something inside of HA from version 10 broke the integration.

So now i can either choose to also downgrade HA to the latest version working with the .11 version of Mosquitto, which also makes any updating of HA, until they fix this impossible or i can upgrade to .15 that will run with the latest versions of HA, but will not actually work.

Not being mad at you guys, you have been fantastic, but honestly, it seems silly, no one has addressed the problem that was created in .15, it has been out for about a year.

Is there anything i can realistically do, to get the Mosquitto people to fix the error, that prevents it from working as it should with the latest versions of HA.

I think it was a security concern where the default wouldn’t be to listen on an unsecured port, forcing you to use TLS. Fortunately, it’s not that big a deal, as you can change your mosquitto.conf to allow unencrypted connections. I’m fairly sure that’s it.

TL;DR: Enable port 1883 in your mosquitto conf and tell your HA integration to connect to it.

Thanks for all the help and suggestions, i dont think it is this simple, it is listening til 1883, and Solar Assistant is set to 1883, the problem is it does not want to start websockets on 1884, i am unable to disable that.

I have found a lot of other people that say, that the only option is to stay on HAOS 9.5 and the .11 MQQT broker.

I tried rolling back to the latest i had, with both of those software versions, but i have made a ton of new stuff since, so i am going to just put on the May 2023 snapshot i have, repair the stuff that got broken from being backed up and restored (Unifi Controller, need to start completely from scratch, not that big of a deal, only 2 hotspots and a few switches).

I will just live without the MQTT working and hope they fix it in the future, allthough i do not have high hopes.

Might end up trying out OpenHAB, from what i have heard it should be a little more stable, i have had more than a few instances where an update in HA, broke something, and was not patched very quickly.

Also looked at Homey, looks to be a LOT more polished, but sadly there are atleasts 2 integrations that i have now, that has no Homey support. :frowning:

Once again…Thanks for trying to help a novice like me :slight_smile:

I do not know how your domestic IT situation looks like.

But maybe you have an additional computer running with something. Or a raspberry pi in a drawer.

I personally want to have full control over which 3rd part I ran and their versions so I moved everything away from the HA addons to a computer where I control things. I only run the SSH and file editor addons on the HA box where it makes sense.

So you can install the version of Mosquitto you want on another box. HA is happy to connect to any MQTT server.

I have a Debian NUC that runs Mosquitto, Deconz, Unifi, Zwave-JS, zigbee2mqtt, and a couple of other things and that has been wonderful as it all became much more stable and I can mix and match revisions that I know work. It sounds difficult but it is actually easy

Mosquitto on Debian is currently 2.0.11 so apt install mosquitto and quick edit of the mosquitto config and “Bob’s your uncle”

2 Likes

Well i have a Rpi 3b that runs PiHole, i could install Mosquitto onto that one maybe.

But to be honest i think it is really poor, that i need to do that, if HA wants to be seen as a serious alternative it needs to become a lot more stable and less fiddley.

I dont blame those who goes out and buys a homey box instead, you almost need a computer science degree and an intimate knowledge of Linux to get this working just marginally ok.

I have reinstalled 3-4 times now, trying different things, used the exact same method every single time, but every time i had a different result.

First time all seemed to be okay, except the Unifi controller that i also have on the HA, after doing another reinstall the controller worked, but something else was broken, now i tried again, controller works fine, but this time the Zigbee stick (SkyConnect) has decided to refuse to work, stating something is wrong with the integration.

I must say, that i am at that point, that if i got everything working i would be VERY sceptical of ever doing another upgrade, because every new update seems to break something that was working before.

I feel really defeated, this is the same, every time i have to deal with something linux…I always feel like walking in quick-sand, there is always something else not quite working lurking at the end of every small gain.

I fully understand your frustration.

My story is … I came from Homey. At that time, Homey sucked even more. “All in one box” solutions have a higher probability to suck somewhere even though they are brilliant most places.

By splitting your smarthome up in smaller chunks you get more freedom to cherrypick.

Right now I run Deconz zigbee and zigbee2mqtt zigbee - in Parallel. Reason being that Deconz is really stable for lights and switches but often suck at sensors. After I moved my battery powered Aqara sensors to their own small Zigbee2MQTT network it is all stable. The cost was a dongle that cost 5 dollars and a few hours of my time. It is not ideal but you have to remember that a smarthome is built from a huge number of independent bits and bobs.
Home Assistant tries to put it all in same box and does a remarkable good job. But Mosquitto is a 3rd party component and the bug in question is one that harms few. And if you downgrade Mosquitto to an earlier version to avoid one bug, you go back to 20 old bugs that may not affect you but affected others.
The maintainer of the Mosquitto Addon is doomed no matter what he does.

Mosquitto uses very little resources. A Raspberry Pi 3 will be just fine.

Here is my /etc/mosquitto/mosquitto.conf file

# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /run/mosquitto/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

bind_address 0.0.0.0
password_file /etc/mosquitto/passwdfile
include_dir /etc/mosquitto/conf.d

I have no additional configurations in the conf.d directory. This is it.

And then you define the mosquitto password and make sure it is added to the file in the location given by the config file. in my case /etc/mosquitto/passwd

To create the password for Mosquitto - cd to the /etc/mosquitto directory and type the command

sudo mosquitto_passwd -c passwdfile username

First you authenticate the sudo command

Then it promps for a password.

See also Mosquitto Username and Password Authentication -Configuration and Testing

If you installed mosquitto on a Debian based box (Raspberry Pi OS, Ubuntu etc) with apt install mosquitto you enable the service with the normal systemctl service commands

sudo systemctl enable mosquitto
sudo systemctl start mosquitto

To restart if it already runs to pick up config

sudo system restart mosquitto

Mosquitto is one of these simple services where it is not even worth running it in Docker. It runs on bare metal using no CPU resources at all for a small normal home installation.

Note that I do not use encryption with Mosquitto. It is a nightmare to make that work with DIY IOT devices and much more difficult to setup.

I don’t agree with this. Why do you blame HA for the specific add-on? I don’t think HA bundles or maintains any add-ons — only the capability.

Disclaimer: I use HA Core so I have no need for add-ons, so I stand corrected.

I dont directly blame them for the failures in the plugins but it would have been nice if they then acknowledged that sometimes plugins go wrong and given us an easy way of reverting.

Right now MQTT does not work, and i have no easy way of reverting to a version of HAOS and Mosquitto where everything works unless i load a very old backup.

Also i just got everything up and running, my SkyConnect dongle was giving me headaches and would not work, seems reinstalling bricked it, and i needed to reflash it.

Then there is the Unifi plugin, the latest version is 2.7.0 but that wont work because of an error and then i tried going back to 2.6.0 that i had just updated from, but after 1 hour it was just doing its spinny spinny so i tried doing a reboot.

After the reboot, the 2.6.0 version was back on, but now the Home Assistant Supervisor is in a “Unable to configure” state so now a lot is not working.

In the end i will probably have to just start over for the umpteens time and then chose to ignore the update for Unifi from 2.6.0 to 2.7.0.

My only problem then is…Will i dare to upgrade to 2.8.0 when it is released. Maybe it fixes the problem, maybe it is still there and i have to reinstall the whole lot again, because i can not downgrade without HA getting stuck and Supervisor throwing a hissyfit.

I really feel this OS is so fragile that any upgrade risks the whole system going down, which is really bad when it is essentially what controls heating, lights, Solar panels, EV Charger and so on.

I never worry when doing a windows update on my computer, but every time there is an update to HA or some plugin i really have to weigh in the pros and cons of trying to upgrade to get new features, vs the very likely risk of the whole system crashing on me and that is pretty bad.

I was the one who raised the alert about the regression bug in the mosquito add-on.

I am lucky that I have a background in IT and extra resources. I run HA in a VM on Proxmox.
To get around the problem with unreliable bridges, I installed a dedicated mosquito broker (at 2.0.11 as this is the current version in Ubuntu) in an Ubuntu LXC. I then configured bridges to my two HA nodes.

This was sufficient to totally cure the instability I was experiencing.

As and when time permits, I’m modifying the configs of the main HA node (which runs on the same Proxmox server as the new broker) to use the new broker rather than the add-on. My objective is remove the need to use the add-on.

2 Likes

Hi

Funny you should say that.

I just got my integration to actually work, even with the new version of Mosquitto and latest version of HA.

Had done a lot of different stuff and restarted a lot and nothing worked.

Restarting in the menu and power off / power on did not work.

I then began looking at a different problem (All of the addon programs were grey and was shown as stopped, even if they were actually running…I could log on to Unifi controller even though it looked to be stopped)

Someone mentioned that he also had this problem and just like me, after reinstalling from a backup.

He mentioned that when selecting “Restart” it would pop up with 2 options “Quick restart - Only reload YAML” and “restart home assistant”

But below you have “Advanced options”

If you press that, it presents 2 extra options, one to shut down and one to “Restart system”.

Restart system is apparanly different than just “restart Home Assistant” and after i did the “Restart system” all of my addons acted like they should in the menu.

AND…My MQTT started working and all of my previously found stuff started reporting the correct data from the Solar inverter.

I have not dared update anything or restart the HA box since so i dont know if this is a “cure” or a “Temporary fix that needs to be used every time i restart the Raspberry PI”

1 Like