hmm, looks like I have had some success…
I have given up in trying to find the cert files on the local filesystem outside Home Assistant and have created the folder /ssl via Windows (Samba) and copied the certs into this folder. Then I configured the bridge file (/share/mosquitto/bridge.conf) accordingly. Now Mosquitto successfully establishes a connection as seen in the Mosquitto Home Assistant log - see below. If you have any idea why I can’t see this in the Linux file system I’d be interested to hear. However given the success below this is probably secondary now.
If “rx” is indeed the indication of the retain flag mine shows r0 as you can see in the log. However mine is a brand new default Home Assistant installation on a new “test system” I’m building (so that I don’t mess up the production server once in full swing) and I haven’t made any changes. But the good news then is, it must be a configuration somewhere that changes the what looks like to be default of retain off to on. Logically this can only be in configuration.yaml, mosquitto.conf or the MQTT bridge file.
Testing
-
Local to AWS-IoT On the AWS-IoT console I subscribed to the topic localgateway_to_awsiot and use Configuration | Integration | MQTT-configure menu to publish a message to topic localgateway_to_awsiot >> It’s successfully received on the AWS-IoT console
-
AWS-IoT to local I subscribed to the topic awsiot_to_localgateway on Mosquitto and sent a message from AWS_IoT to Mosquitto >> It was received successfully on the Mosquitto end
So from this initial test it looks like this is working.
I have produced the complete log from Mosquitto restart as comparison. So for a starter maybe you can have a glance through your log and verify whether you spot any differences in order to narrow this down. Also do you use Tasmota at all? There’s this guy called Stephan Hadinger who is a real guru on AWS/MQTT and he may have some further insight if nothing obvious jumps out. He is active on the Tasmota forums (Discord).
Lastly why is the retain flag on an issue in your case? From the little research I have done retain on may be beneficial for clients that connect in order to get the last message on the topic immediately without having to wait for the next message update.
[23:50:32] INFO: Setup mosquitto configuration
[23:50:32] WARNING: SSL not enabled - No valid certs found!
[23:50:32] INFO: No local user available
[23:50:32] INFO: Initialize Hass.io Add-on services
[23:50:32] INFO: Initialize Home Assistant discovery
[23:50:32] INFO: Start Mosquitto daemon
1603025432: Loading config file /share/mosquitto/bridge.conf
1603025432: mosquitto version 1.6.3 starting
1603025432: Config loaded from /etc/mosquitto.conf.
1603025432: Loading plugin: /usr/share/mosquitto/auth-plug.so
1603025432: |-- *** auth-plug: startup
1603025432: ├── Username/password checking enabled.
1603025432: ├── TLS-PSK checking enabled.
1603025432: └── Extended authentication not enabled.
1603025432: Opening ipv4 listen socket on port 1883.
1603025432: Opening ipv6 listen socket on port 1883.
1603025432: Opening websockets listen socket on port 1884.
1603025433: Warning: Mosquitto should not be run as root/administrator.
1603025433: Bridge local.bridgeawsiot doing local SUBSCRIBE on topic localgateway_to_awsiot
1603025433: Bridge local.bridgeawsiot doing local SUBSCRIBE on topic both_directions
1603025433: Connecting bridge awsiot (xxxxxxxxxxxxxxx.iot.us-east-1.amazonaws.com:8883)
1603025433: Bridge bridgeawsiot sending CONNECT
1603025433: Received CONNACK on connection local.bridgeawsiot.
1603025433: Bridge local.bridgeawsiot sending SUBSCRIBE (Mid: 1, Topic: awsiot_to_localgateway, QoS: 1, Options: 0x00)
1603025433: Bridge local.bridgeawsiot sending UNSUBSCRIBE (Mid: 2, Topic: localgateway_to_awsiot)
1603025433: Bridge local.bridgeawsiot sending SUBSCRIBE (Mid: 3, Topic: both_directions, QoS: 1, Options: 0x00)
1603025434: Received SUBACK from local.bridgeawsiot
1603025434: Received UNSUBACK from local.bridgeawsiot
1603025434: Received SUBACK from local.bridgeawsiot
1603025434: New connection from 172.30.32.1 on port 1883.
[INFO] found homeassistant on local database
1603025434: New client connected from 172.30.32.1 as auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1 (p2, c1, k60, u'homeassistant').
1603025434: Will message specified (7 bytes) (r0, q0).
1603025434: homeassistant/status
1603025434: Sending CONNACK to auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1 (0, 0)
1603025434: Received SUBSCRIBE from auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1
1603025434: homeassistant/# (QoS 0)
1603025434: auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1 0 homeassistant/#
1603025434: Sending SUBACK to auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1
1603025436: Received PUBLISH from auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1 (d0, q0, r0, m0, 'homeassistant/status', ... (6 bytes))
1603025436: Sending PUBLISH to auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1 (d0, q0, r0, m0, 'homeassistant/status', ... (6 bytes))
1603025490: Received SUBSCRIBE from auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1
1603025490: awsiot_to_localgateway (QoS 0)
1603025490: auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1 0 awsiot_to_localgateway
1603025490: Sending SUBACK to auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1
1603025493: Sending PINGREQ to local.bridgeawsiot
1603025493: Received PINGRESP from local.bridgeawsiot
1603025503: Received PUBLISH from auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1 (d0, q0, r0, m0, 'localgateway_to_awsiot', ... (21 bytes))
1603025503: Sending PUBLISH to local.bridgeawsiot (d0, q0, r0, m0, 'localgateway_to_awsiot', ... (21 bytes))
1603025521: Received PUBLISH from local.bridgeawsiot (d0, q0, r0, m0, 'awsiot_to_localgateway', ... (45 bytes))
1603025521: Sending PUBLISH to auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1 (d0, q0, r0, m0, 'awsiot_to_localgateway', ... (45 bytes))
1603025525: Received UNSUBSCRIBE from auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1
1603025525: awsiot_to_localgateway
1603025525: auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1 awsiot_to_localgateway
1603025525: Sending UNSUBACK to auto-F6C5B5B6-0630-007D-218C-CD97AC2979D1