Help with MQTT please

Can I please get a hand installing & using my Sonoff Zigbee 3 Plus to work with Zigbee2MQTT?

I’ve got the firmware updated. I’ve got the dongle recognised in HA OS from my VM running from a Synology 920+ on DSM7x.

ZHA recognises it - but if I use that it instantly kills any hope of using Zigbee2MQTT. Thusly, I have not clicked on ‘configure’. If I try to set up Z2Mqtt (configure tab) it looks nothing alike to the ones shown in videos, so it seems I have a very different thing going, at least judging by the configuration tab.

Below is my hardware tab pertaining to my Dongle+

ttyUSB0/dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_8882c180ba5fec118985365f25bfaa52-if00-port0

Subsystem:tty

Device path:/dev/ttyUSB0

ID:/dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_8882c180ba5fec118985365f25bfaa52-if00-port0

Under Attributes: it states…

DEVLINKS: >-
/dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_8882c180ba5fec118985365f25bfaa52-if00-port0
/dev/serial/by-path/pci-0000:00:05.0-usb-0:1:1.0-port0
DEVNAME: /dev/ttyUSB0
DEVPATH: /devices/pci0000:00/0000:00:05.0/usb2/2-1/2-1:1.0/ttyUSB0/tty/ttyUSB0
ID_BUS: usb
ID_MODEL: Sonoff_Zigbee_3.0_USB_Dongle_Plus
ID_MODEL_ENC: Sonoff\x20Zigbee\x203.0\x20USB\x20Dongle\x20Plus
ID_MODEL_ID: ea60
ID_PATH: pci-0000:00:05.0-usb-0:1:1.0
ID_PATH_TAG: pci-0000_00_05_0-usb-0_1_1_0
ID_REVISION: ‘0100’
ID_SERIAL: ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_8882c180ba5fec118985365f25bfaa52
ID_SERIAL_SHORT: 8882c180ba5fec118985365f25bfaa52
ID_TYPE: generic
ID_USB_DRIVER: cp210x
ID_USB_INTERFACES: ‘:ff0000:’
ID_USB_INTERFACE_NUM: ‘00’
ID_VENDOR: ITead
ID_VENDOR_ENC: ITead
ID_VENDOR_ID: 10c4
MAJOR: ‘188’
MINOR: ‘0’
SUBSYSTEM: tty
TAGS: ‘:systemd:’
USEC_INITIALIZED: ‘83284241532’

Now, as far as I can tell - the above is all good to go. But I will add, the dongle+ was not recognised for several days. It seems if I dare to unplug it, it loses everything and doesn’t come back for seemingly days… I have no idea why it doesn’t register for days and then suddenly, out of the blue, is recognised.

So moving on to ZHA (which I don’t want to use, as I want to start learning & using MQTT) it seems to have acknowledged the existence of the Dongle+ but I have not done anything with it as yet… (partly because I’m petrified of losing it for days on end again).

Moving on to MQTT…

I have set up a standard & local user for the broker.
The YAML for the Mosquitto broker reads as:

logins:
require_certificate: false
certfile: fullchain.pem
keyfile: privkey.pem
customize:
active: true
folder: mosquitto

The settings, under ‘Network’ (of mosquitto broker) are as follows:

  • 1883 / tcp — “normal MQTT”
  • 1884 / tcp — “MQTT over WebSocket”
  • 8883 / tcp — “Normal MQTT with SSL”
  • 8884 / tcp — “MQTT over WebSocket with SSL”

However, when I click on the ‘log’ tab, I am confronted with the following, which repeats over & over.

[13:13:42] INFO: Starting mosquitto MQTT broker…
1654172022: Error: Unable to open include_dir ‘/share/mosquitto’.
1654172022: Error found at /etc/mosquitto/mosquitto.conf:36.
[13:13:42] INFO: Successfully send discovery information to Home Assistant.
[13:13:42] INFO: Successfully send service information to the Supervisor.
[13:13:43] INFO: Starting mosquitto MQTT broker…
1654172023: Error: Unable to open include_dir ‘/share/mosquitto’.
1654172023: Error found at /etc/mosquitto/mosquitto.conf:36.
[13:13:43] INFO: Successfully send discovery information to Home Assistant.

I am aware there is an issue with it using the ‘include_dir/…’ but I am uncertain how to begin resolving this.

Moving on to the Zigbee2MQTT…

The initial Configuration tabs shows the config broken in to separate elements; socat, mqtt & serial.

SOCAT:

enabled: true
master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
options: ‘-d -d’
log: false
frontend:
port: 8099

MQTT

base_topic: zigbee2mqtt
user: xxx
password: xxx
server: http://my.ip.add.ress:1883

SERIAL

port: /dev/ttyUSB0

The NETWORK tab of Zigbee2MQTT shows the following:

  • 8485 / tcp — Socat tcp-listen port

The LOG tab for Zigbee2MQTT shows the following information, again, repeating over & over…

Zigbee2MQTT:info 2022-06-02 13:21:20: Stopped zigbee-herdsman
[13:21:22] INFO: Handing over control to Zigbee2mqtt Core …
[email protected] start
node index.js
Zigbee2MQTT:info 2022-06-02 13:21:27: Logging to console and directory: ‘/config/zigbee2mqtt/log/2022-06-02.13-21-27’ filename: log.txt
Zigbee2MQTT:info 2022-06-02 13:21:27: Starting Zigbee2MQTT version 1.25.2 (commit #unknown)
Zigbee2MQTT:info 2022-06-02 13:21:27: Starting zigbee-herdsman (0.14.34)
Zigbee2MQTT:info 2022-06-02 13:21:28: zigbee-herdsman started (resumed)
Zigbee2MQTT:info 2022-06-02 13:21:28: Coordinator firmware version: ‘{“meta”:{“maintrel”:1,“majorrel”:2,“minorrel”:7,“product”:1,“revision”:20220219,“transportrev”:2},“type”:“zStack3x0”}’
Zigbee2MQTT:info 2022-06-02 13:21:28: Currently 0 devices are joined:
Zigbee2MQTT:warn 2022-06-02 13:21:28: permit_join set to true in configuration.yaml.
Zigbee2MQTT:warn 2022-06-02 13:21:28: Allowing new devices to join.
Zigbee2MQTT:warn 2022-06-02 13:21:28: Set permit_join to false once you joined all devices.
Zigbee2MQTT:info 2022-06-02 13:21:28: Zigbee: allowing new devices to join.
Zigbee2MQTT:info 2022-06-02 13:21:28: Connecting to MQTT server at http://192.168.0.61:1883
Zigbee2MQTT:error 2022-06-02 13:21:32: MQTT failed to connect: read ECONNRESET
Zigbee2MQTT:error 2022-06-02 13:21:32: Exiting…
Zigbee2MQTT:info 2022-06-02 13:21:32: Stopping zigbee-herdsman…
Zigbee2MQTT:info 2022-06-02 13:21:42: Stopped zigbee-herdsman
[13:21:43] INFO: Handing over control to Zigbee2mqtt Core …
[email protected] start
node index.js
Zigbee2MQTT:info 2022-06-02 13:21:48: Logging to console and directory: ‘/config/zigbee2mqtt/log/2022-06-02.13-21-48’ filename: log.txt

So, if anyone could kindly assist me in understanding & getting this working, I would be forever indebted to you :slight_smile:

The MQTT, I just installed and never did anything. The configuration looks like this. I have one more use of MQTT, besides Z2M. I just use a administrator user for setting this up.

logins: []
require_certificate: false
certfile: fullchain.pem
keyfile: privkey.pem
customize:
  active: false
  folder: mosquitto

I beleive the Z2M looks very different from the videos due to a new setup in version 1.2??. However, you can still press the 3 dots and chose YAML mode. My setup looks like (I have deleted som of the network key lines, security. Get inspired. I using same coordinator as you.

data_path: /config/zigbee2mqtt
socat:
  enabled: false
  master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
  slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
  options: '-d -d'
  log: false
mqtt:
  base_topic: zigbee2mqtt
serial:
  port: >-
    /dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_d617560171d9eb11bf30158e6fe9f4d9-if00-port0
external_converters: []
devices: devices.yaml
groups: groups.yaml
homeassistant: true
permit_join: false
advanced:
  log_level: warn
  pan_id: 1
  channel: 13
  network_key:
  xxxxxx
    - 4
    - 6
    - 8
    - 10
    - 12
    - 13
  availability_blocklist: []
  availability_passlist: []
  last_seen: ISO_8601_local
device_options: {}
blocklist: []
passlist: []
queue: {}
frontend:
  port: 8099
experimental: {}
availability: false

EDIT: Have looked again. I beleive you should remove the user and server information on Z2M regarding MQTT. It will then revert to the default.

That should be:

server: mqtt://my.ip.add.ress:1883

(the :1833 is superfluous and can be left out)

1 Like

Alas, that seems to have made no difference… what do you mean, the default? Surely I need a username & p/w to have HAOS access the MQTT server?

That seems to have helped move me nearer to the goal, thanks :slight_smile:
I’m not sure what else is stopping me at this point…

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
cont-init: info: running /etc/cont-init.d/socat.sh
cont-init: info: /etc/cont-init.d/socat.sh exited 0
cont-init: info: running /etc/cont-init.d/zigbee2mqtt.sh
cont-init: info: /etc/cont-init.d/zigbee2mqtt.sh exited 0
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun socat (no readiness notification)
services-up: info: copying legacy longrun zigbee2mqtt (no readiness notification)
s6-rc: info: service legacy-services successfully started
Socat startup parameters:
Options: -d -d
Master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
Slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
[23:00:41] INFO: Starting socat process …
2022/06/02 23:00:41 socat[108] N PTY is /dev/pts/0
2022/06/02 23:00:41 socat[108] N listening on AF=2 0.0.0.0:8485
[23:00:42] INFO: Handing over control to Zigbee2mqtt Core …
[email protected] start
node index.js
Zigbee2MQTT:info 2022-06-02 23:00:46: Logging to console and directory: ‘/config/zigbee2mqtt/log/2022-06-02.23-00-46’ filename: log.txt
Zigbee2MQTT:info 2022-06-02 23:00:46: Starting Zigbee2MQTT version 1.25.2 (commit #unknown)
Zigbee2MQTT:info 2022-06-02 23:00:46: Starting zigbee-herdsman (0.14.34)
Zigbee2MQTT:info 2022-06-02 23:00:47: zigbee-herdsman started (resumed)
Zigbee2MQTT:info 2022-06-02 23:00:47: Coordinator firmware version: ‘{“meta”:{“maintrel”:1,“majorrel”:2,“minorrel”:7,“product”:1,“revision”:20220219,“transportrev”:2},“type”:“zStack3x0”}’
Zigbee2MQTT:info 2022-06-02 23:00:47: Currently 0 devices are joined:
Zigbee2MQTT:warn 2022-06-02 23:00:47: permit_join set to true in configuration.yaml.
Zigbee2MQTT:warn 2022-06-02 23:00:47: Allowing new devices to join.
Zigbee2MQTT:warn 2022-06-02 23:00:47: Set permit_join to false once you joined all devices.
Zigbee2MQTT:info 2022-06-02 23:00:47: Zigbee: allowing new devices to join.
Zigbee2MQTT:info 2022-06-02 23:00:47: Connecting to MQTT server at mqtt://core-mosquitto:1883
Zigbee2MQTT:error 2022-06-02 23:00:47: MQTT failed to connect: connect ECONNREFUSED 172.30.33.0:1883
Zigbee2MQTT:error 2022-06-02 23:00:47: Exiting…
Zigbee2MQTT:info 2022-06-02 23:00:47: Stopping zigbee-herdsman…
Zigbee2MQTT:info 2022-06-02 23:00:57: Stopped zigbee-herdsman
[23:00:58] INFO: Handing over control to Zigbee2mqtt Core …

It goes so far… then hits another brick wall. I’m not sure I recognise the IP (172.30.33.0) but I presume it’s an internal network built by HAOS for the purposes of MQTT and other related services…

I do not have a server, username or password in Z2M configuration for MQTT. As shown in my configration file for Z2M. It uses default (assuming you have both on the same device, hich you have as I read it).

If looked at it from the Z2M dashboard, it looks like this, the standard settings (I know you can not see this as Z2M will not start, just to show it is with no settings)

In your initial post, it tried to connect to a broker on 192.168.0.61.

Where does one change that, assuming that’s what you’re implying?

I’m not implying anything, I’m just mentioning an observation :smiley:

I don’t know where that IP-address came from, and if it makes sense to use. It was the MQTT broker IP-address that was initially used, I assumed it was configured as:

server: http://192.168.0.61:1883

(if so, it should have been mqtt://192.168.0.61)

I’ve tried removing the username/password & server information, so it uses defaults as suggested previously by another user - no go.

I’ve edited YAML/config to use localhost:1883, localhost, 192.168.0.61:1883,192.168.0.61, and no matter what details I put in there - it refuses to connect. It stumbles at the same point each time :frowning:

Zigbee2MQTT:info 2022-06-03 10:21:17: Zigbee: allowing new devices to join.
Zigbee2MQTT:info 2022-06-03 10:21:17: Connecting to MQTT server at mqtt://192.168.0.61:1883
Zigbee2MQTT:error 2022-06-03 10:21:23: MQTT failed to connect: read ECONNRESET
Zigbee2MQTT:error 2022-06-03 10:21:23: Exiting…
Zigbee2MQTT:info 2022-06-03 10:21:23: Stopping zigbee-herdsman…

No matter what I put in the config - it dies.

I’ve just gone back to my broker’s log (Mosquitto) and found this in the logs, repeating over & over…

[10:33:44] INFO: Starting mosquitto MQTT broker…
1654248824: Error: Unable to open include_dir ‘/share/mosquitto’.
1654248824: Error found at /etc/mosquitto/mosquitto.conf:36.

Is it likely that this error is causing the connection issues to from Z2M??? The log shows these three lines repeating over & over with different time-stamps.

Have to admit, I do not know enough about MQTT. Have you tried to remove MQTT add-on, reboot and then install again. Just to figure out if MQTT can start with no errors+

1 Like

Yeah, I usually pride myself on figuring out computer related problems with relative ease - but HOAS has been nothing but a pain in the rear-end with one or another issue constantly cropping up. I think half the issue is simply how quickly instruction videos & related pages become obsolete. Every video I’ve seen has a lengthy page of configuration, but since I’ve tried installing Z2M it’s always been a short (roughly 10 lines) YAML, not like the videos which look more like 30 lines.

It certainly doesn’t look good.

Any suggestions on how to fix that?

Google found this

So try active: false

Doesn’t make sense to me, but that’s what I found in my quick search.

In general, you have to get Mosquitto working first, nothing MQTT related will work until you fix it so it starts up ok.

I run mosquitto in a docker container, independent of HA. I know this is more work than running the mosquitto add on, but I like having my MQTT broker running independently, was worth it to me to spend the time to figure it out. I used IOTstack to install, which does most of the work.

Randy

1 Like

KarlM, how is you MQTT configuration looking.

Try using this, no changes.

logins: []
require_certificate: false
certfile: fullchain.pem
keyfile: privkey.pem
customize:
  active: false
  folder: mosquitto
1 Like

Awesome! That seems to have fixed the Mozzy Broker from logging repeated errors as previous - THANK YOU soooooo much, Randy! That’s one step nearer, I think…

Thanks, khvej… As above, the ‘false’ seems to have resolved the broker side of things :slight_smile:
Now… I need to resume work on the Z2M as that’s still hiccupping badly :frowning:

Zigbee2MQTT:info 2022-06-03 20:32:42: Connecting to MQTT server at mqtt://192.168.0.61:1883
Zigbee2MQTT:error 2022-06-03 20:32:42: MQTT failed to connect: Connection refused: Not authorized
Zigbee2MQTT:error 2022-06-03 20:32:42: Exiting…
Zigbee2MQTT:info 2022-06-03 20:32:42: Stopping zigbee-herdsman…

I stand corrected… I’ve just refreshed the Broker’s logs - and it’s chock full of this:

1654284972: Client disconnected, not authorised.
1654284973: New connection from 172.30.32.1:39290 on port 1883.

So I have restarted it and refreshed the logs as they’re being added to.

It seems to start without issue - then BAM - and all hope is gone!

[s6-init] making user provided files available at /var/run/s6/etc…exited 0.
[s6-init] ensuring user provided files have correct perms…exited 0.
[fix-attrs.d] applying ownership & permissions fixes…
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts…
[cont-init.d] mosquitto.sh: executing…
[20:40:53] INFO: Certificates found: SSL is available
[cont-init.d] mosquitto.sh: exited 0.
[cont-init.d] nginx.sh: executing…
[cont-init.d] nginx.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[20:40:53] INFO: Starting NGINX for authentication handling…
[20:40:54] INFO: Starting mosquitto MQTT broker…
1654285254: Warning: Mosquitto should not be run as root/administrator.
[20:40:55] INFO: Successfully send discovery information to Home Assistant.
[20:40:55] INFO: Successfully send service information to the Supervisor.
1654285254: mosquitto version 2.0.11 starting
1654285254: Config loaded from /etc/mosquitto/mosquitto.conf.
1654285254: Loading plugin: /usr/share/mosquitto/go-auth.so
1654285254: ├── Username/password checking enabled.
1654285254: ├── TLS-PSK checking enabled.
1654285254: └── Extended authentication not enabled.
1654285254: Opening ipv4 listen socket on port 1883.
1654285254: Opening ipv6 listen socket on port 1883.
1654285254: Opening websockets listen socket on port 1884.
1654285254: Opening ipv4 listen socket on port 8883.
1654285254: Opening ipv6 listen socket on port 8883.
1654285254: Opening websockets listen socket on port 8884.
1654285254: mosquitto version 2.0.11 running
1654285255: New connection from 127.0.0.1:37444 on port 1883.
1654285255: Client closed its connection.
1654285255: New connection from 172.30.32.1:51860 on port 1883.
1654285255: Client disconnected, not authorised.
1654285255: New connection from 172.30.32.1:48383 on port 1883.
1654285255: New client connected from 172.30.32.1:48383 as 3i2vLaFeouoc4jfyL2L4uq (p2, c1, k60, u’moz’).
1654285256: New connection from 172.30.32.1:51866 on port 1883.
1654285256: Client disconnected, not authorised.
1654285262: New connection from 172.30.32.1:44964 on port 1883.
1654285262: Client disconnected, not authorised.
1654285263: New connection from 172.30.32.1:44980 on port 1883.
1654285263: Client disconnected, not authorised.
1654285264: New connection from 172.30.32.1:44988 on port 1883.
1654285264: Client disconnected, not authorised.

Below is my configuration YAML for Mosquitto Broker…

logins:
require_certificate: false
certfile: fullchain.pem
keyfile: privkey.pem
customize:
active: false
folder: mosquitto

:roll_eyes:

OK so I uninstalled MQTT broker, uninstalled Z2M and reinstalled MQTT broker.

It had retained information from it’s previous install.

I manually deleted the User ‘moz’ and created a new local-only user called ‘mozzy’ with a password.

I went back to the ‘Integrations’ page of HA and found the MQTT broker integration. I clicked ‘Configure’ and then "re-configure MQTT’ at the top where I applied the newly created local-only user of ‘mozzy’ and it’s password.


ETA:
I’d forgotten to restart the broker…

Ignore everything I’d wrote before…

sigh :face_with_raised_eyebrow:


OMG OMG OMG OMG :open_mouth:

I think… I thiiiiiiiink… it’s working… :crossed_fingers:

I’ve just reinstalled Z2M and started the service. I’ve kept an eye on the logs and intermittently refreshed them.

Annnnnnd so far, it’s given the below results without further logs being added, as yet.

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
cont-init: info: running /etc/cont-init.d/socat.sh
[21:38:31] INFO: Socat not enabled, marking service as down
cont-init: info: /etc/cont-init.d/socat.sh exited 0
cont-init: info: running /etc/cont-init.d/zigbee2mqtt.sh
cont-init: info: /etc/cont-init.d/zigbee2mqtt.sh exited 0
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun socat (no readiness notification)
services-up: info: copying legacy longrun zigbee2mqtt (no readiness notification)
[21:38:32] INFO: Handing over control to Zigbee2mqtt Core …

[email protected] start
node index.js
Zigbee2MQTT:info 2022-06-03 21:38:36: Logging to console and directory: ‘/config/zigbee2mqtt/log/2022-06-03.21-38-36’ filename: log.txt
Zigbee2MQTT:info 2022-06-03 21:38:36: Starting Zigbee2MQTT version 1.25.2 (commit #unknown)
Zigbee2MQTT:info 2022-06-03 21:38:36: Starting zigbee-herdsman (0.14.34)
Zigbee2MQTT:info 2022-06-03 21:38:37: zigbee-herdsman started (resumed)
Zigbee2MQTT:info 2022-06-03 21:38:37: Coordinator firmware version: ‘{“meta”:{“maintrel”:1,“majorrel”:2,“minorrel”:7,“product”:1,“revision”:20220219,“transportrev”:2},“type”:“zStack3x0”}’
Zigbee2MQTT:info 2022-06-03 21:38:37: Currently 0 devices are joined:
Zigbee2MQTT:warn 2022-06-03 21:38:37: permit_join set to true in configuration.yaml.
Zigbee2MQTT:warn 2022-06-03 21:38:37: Allowing new devices to join.
Zigbee2MQTT:warn 2022-06-03 21:38:37: Set permit_join to false once you joined all devices.
Zigbee2MQTT:info 2022-06-03 21:38:37: Zigbee: allowing new devices to join.
Zigbee2MQTT:info 2022-06-03 21:38:37: Connecting to MQTT server at mqtt://core-mosquitto:1883
Zigbee2MQTT:info 2022-06-03 21:38:38: Connected to MQTT server
Zigbee2MQTT:info 2022-06-03 21:38:38: MQTT publish: topic ‘zigbee2mqtt/bridge/state’, payload ‘{“state”:“online”}’
Zigbee2MQTT:info 2022-06-03 21:38:38: Started frontend on port 0.0.0.0:8099
Zigbee2MQTT:info 2022-06-03 21:38:38: MQTT publish: topic ‘zigbee2mqtt/bridge/state’, payload ‘{“state”:“online”}’

:pray:

Does this seem about right to anyone who knows from previous how it should work?

I have not edited the configuration tab of Z2M as yet, so it seems the default config does work - even though everywhere tells me to edit it…

It’s working, after a full uninstallation & reinstallation.

What a hideous headache this has been…

1 Like