MQTT discovery after migration from hass.io to docker

mmcr · May 29, 2021, 11:41pm

I had a perfectly working hass.io installation using the add-on mqtt broker and about 27 auto-discovered entities from this.

From mqtt-explorer, this is the topic tree:

In order to migrate to standalone HA in docker, I first created a MQTT-Eclipse (2.0 SSL) container, and pointed my devices and hass.io at it.

I didn’t change any configs for the devices (so discovery settings etc unchanged) or hass.io, other than pointing them at the new broker.

However with the external (docker) broker, the topic tree doesn’t include the ‘homeassistant’ prefix:

Despite this, the previously working entities in hass.io still work with this topic tree under hass.io

Now when I copy everything (including .storage and the home-assistant_v2.db) to the new standalone HA docker container and fire it up, it will connect to the broker, but won’t discover any MQTT entities.

It’s definitely receiving all messages (I can see them all when I type ‘#’ under listen in configuration), and I can also publish from HA to the broker.

I don’t understand why
a) the external broker doesn’t add the ‘homeassistant’ topic prefix when none of the MQTT device configs have changed
b) the original hass.io installation doesn’t seem to care when pointed to the new external broker without the discovery prefix in the topic tree… is there some other setting in hass.io that I’m missing ?

I can’t see any way to force-add a topic prefix in mosquitto.conf, or to simply get HA to incorporate all topics for discovery

How can I get the new HA docker installation to discovery my devices ?!

nickrout · May 30, 2021, 12:00am

You can see exactly what is posted with mosquitto_sub.

Have turned discovery on in the docker instance (done in integrations UI)

mmcr · May 30, 2021, 1:51am

Yes discovery in the docker instance is on - first from integrations UI

In docker HA, I then tried deleting the MQTT integration, restart, add MQTT through yaml (with discovery: true), but no difference

nickrout · May 30, 2021, 3:47am

Also, how often do your devices send discovery data? On startup (in which case, have you restarted them?)

francisp · May 30, 2021, 3:56am

Most devices only send discovery messages at startup, so indeed restarting them might help.

mmcr · May 30, 2021, 4:03am

I’ve restarted the devices, broker and HA each time I’ve tried something new (so many, many times !)

francisp · May 30, 2021, 4:07am

Well, at least for Tasmota devices, running (in the tasmota console)

setoption19 0
setoption19 1

resends the discovery message.

For Zigbee2MQTT, restarting Zigbee2MQTT re-sends the discovery messages

For other devices I don’t know

mmcr · May 30, 2021, 4:49am

I just created a brand new docker instance of HA without copying my old config/db over

Same issue - can connect to the docker broker, can listen to all topics, but no entities found

I’m stumped (and frustrated !)

I guess the alternative would be to run HA OS in a VM, but I was hoping to avoid that…

nickrout · May 30, 2021, 4:52am

What does mosquitto_sub tell you?

mmcr · May 30, 2021, 5:23am

Same thing as the mqtt_explorer screenshots above - nothing with a homeassistant prefix

nickrout · May 30, 2021, 5:28am

What devices are these?

mmcr · May 30, 2021, 5:31am

Paradox alarm interface and Actron Que

nickrout · May 30, 2021, 5:35am

This is possibly an obvious question, but did you set up the new mqtt broker details in the devices?

mmcr · May 30, 2021, 5:37am

Yes I did

Their logs show connection to the new broker.

Listening in HA shows their topics coming through. Just without the homeassistant prefix (even though they are both configured to do so - unchanged from when they did this in hass.io)

nickrout · May 30, 2021, 5:43am

I guess that includes setting

# MQTT_HOMEASSISTANT_DISCOVERY_PREFIX = 'homeassistant'

Although it looks like that is the default.

nickrout · May 30, 2021, 5:46am

You’re not trying to use anonymous MQTT are you? A recent change to the mosquitto addon makes anonymous unsupported.

Dunno, clutching at straws here.

mmcr · May 30, 2021, 6:00am

Yes re discovery prefix
And no, mqtt is set up with usernames

Interesting (to me!) update:

Reverting to hass.io and mqtt add-on, I’ve found that:

the mqtt devices (each in their own container with their own docker network), have to refer to the docker add-on by IP address (host) rather than ‘core-mosquito’ etc namespacing in order to connect. This makes sense re how docker bridge networking works, but does this imply that all mqtt containers, the broker and HA need to be in bridge mode and on the same bridge network for discovery to work ?
I could do that, but then wouldn’t having HA in a bridge network (rather than host network) break SSL for the HA web interface (I’m using lets encrypt, but no reverse proxy) ?
When I initially had the mqtt device containers set up to connect with the add-on broker with core-mosquitto (rather than IP address), the entities all showed up correctly in HA. However there was no connection in the add-on broker logs, and the device status was obviously wrong. Stopping the containers made the entities unavailable. So even though they weren’t connecting to the broker, HA somehow registered the entities - how does this work ??!

nickrout · May 30, 2021, 6:10am

Are you using home assistant core in docker or a supervised install?

I guess it must be supervised, otherwise you couldn’t be using addons.

If you are using supervised, why did you use mosquitto in an docker container, instead of the core-mosquitto addon?

mmcr · May 30, 2021, 6:37am

Sorry - what I have referred to as hass.io was a supervised install.

I am trying to migrate to core in docker

Because of the discovery issues I have rolled back to the supervised installation with the core-mosquito addon for now

nickrout · May 30, 2021, 6:50am

Why?

Also I haven’t seen anything about mosquitto 2 and home assistant. The addon uses 1.6.12.