Lumik
November 2, 2024, 8:55pm
1
Hello everyone, I would like to kindly ask for help with my struggle:
I am trying to make MQTT bridge to Victron Cerbo GX with HA Mosquitto Add-on using SSL connection.
The key requirements:
use SSL
use password authentication
Note:
Venus OS 3.50 with MQTT SSL/password auth support
it works on insecure connection using 1883
it works when not using password authentication
I am trying to make this work for several months - I’ve been waiting for Victron firmware updates every month to see if they already implemented password-protected MQTT communication, was trying to achieve this with previous firmwares, manual mqtt service mods inside Venus over SSH
I’ve read dozens of threads on the internet, followed many different guides, but none of them utilized the SSL+password protected connection
Connecting to Cerbo using MQTT explorer, with TLS:on, Verify:off and using username/password works perfectly
I’ve tried to solve this issue with Victron community (Where to get CA certificate for Venus SSL self-signed certificate - Modifications - Victron Community ), but it seems that since the MQTT communication works in another software (MQTT explorer), the problem will be in HA Mosquitto.
The issue:
When Mosquitto tries to connect to Victron, the connenction fails on SSL certificate verification. The Victron (Venus) SSL certificate is self-signed, has no CA in the chain.
To overcome the verification issues, I tried:
set bridge_insecure option
set bridge_cafile / bridge_capath options
reconfigure MQTT device / Mosquitto Add-on in HA, switch different “verify key” or similar options
I use the SSL certificate from Venus https/SSL, downloaded using browser.
Here is the bridge configuration file:
connection victron
address 192.168.1.15:8883
remote username victron
remote_password **************
bridge_insecure true
# Just for reference - the various options I tried to combine:
#bridge_cafile /share/mosquitto/certs/venus-ca.crt
#bridge_certfile /share/mosquitto/certs/venus-ca.crt
#bridge_capath /etc/ssl/certs
#bridge_tls_version tlsv1.3
#bridge_protocol_version mqttv311
topic N/# in 0 victron/
#topic R/# out 0 victron/
#topic W/# out 0 victron/
#topic # both 0
When I set:
bridge_insecure true
I get this in Mosquitto Add-on log:
2024-11-02 21:19:56: Connecting bridge victron (192.168.1.15:8883)
2024-11-02 21:19:56: Client local.core-mosquitto.victron closed its connection.
When I set:
bridge_insecure false
bridge_cafile /share/mosquitto/certs/venus-local-chain.crt
I get this in Mosquitto Add-on log:
2024-11-02 21:21:34: Connecting bridge victron (192.168.1.15:8883)
2024-11-02 21:21:34: OpenSSL Error[0]: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
2024-11-02 21:21:34: Bad socket read/write on client local.core-mosquitto.victron: A TLS error occurred.
Any ideas how to make this working? Did anyone make bridge connection with self-signed certificate?
2 Likes
Any news on this ? Cannot make it work it work either
MasterCATZ
(Christopher Staples)
May 17, 2025, 12:20pm
3
they Replaced Mosquitto with FlashMQ
opened 07:22AM - 05 Jun 23 UTC
closed 05:48AM - 23 Apr 24 UTC
We're going to replace Mosquitto + dbus-mqtt by [FlashMQ](https://www.flashmq.or… g/) + [dbus-flashmq](https://github.com/victronenergy/dbus-flashmq).
### The differences in performance:
- FlashMQ + dbus-plugin is much faster, publishing all topics (1500) happens in 10ms. In Python this was 3 seconds.
- FlashMQ + dbus-plugin uses far less TCP packets by maximizing MQTT data per TCP packet. In one sampe for a demo installation with 1500 topics, It was 2900 TCP packets with the old solution, and with FlashMQ it's 179 TCP packets.
- FlashMQ + dbus-plugin uses far less CPU on the GX device.
### Further details:
Architecture:
- Mosquitto in Venus OS will be replaced by FlashMQ
- dbus-mqtt (the python service) will be replaced by dbus-flashmq (plugin to FlashMQ)
- mqtt-rpc remains in place. This is the other python service, the one that takes care of all remote functionality like the dashboard on VRM; remote firmware updates, and so forth
Note that there will be a transitionary period, because FlashMQ currently does not support broker to broker connections. During that period we'll use FlashMQ for all local websocket communication (HTML5-App and Gui-v2). And at same time still use dbus-mqtt plus Mosquitto for connection to/via VRM.
Note that the cluster in the cloud is now also Mosquitto, and will be replaced by FlashMQ as well.
MQTT spec. details:
- Support of [Selective keep-alives](https://github.com/victronenergy/dbus-mqtt/blob/master/README.md#using-the-keepalive-method-for-selective-keep-alives) will be dropped. While now supported by dbus-mqtt, they will not be implemented in dbus-flashmq.
- New feature added: `"keepalive-options" : ["suppress-republish"]`. To allow keeping the flow of messages alive without triggering a full (re-)transmission of all topics. See [dbus-flashmq readme](https://github.com/victronenergy/dbus-flashmq) for further details; as well as below.
- Added a publish of `N/<portalid>/full_publish_completed` after full publish. This allows clients to know when they have received all topics, and can
shorten the wait period.
- Invalid values will be on mqtt as json `null` value, just like they were.
- MQTT messages will no longer be sent with the RETAIN flag. Details [here](https://github.com/victronenergy/venus/issues/1027) and [here](https://github.com/victronenergy/gui-v2/issues/313).
### Changes in apps with regards to keep alive and retain:
*This is how UIs (gui-v2, html5-app, VRM) should now work:*
1) subscribe to `N/<portalid>/#`
2) publish once to `R/<portalid>/keepalive` (lower case k). Empty payload.
2) thereafter, every 30 seconds, publish to `R/<portalid>/keepalive` again, with payload `{ "keepalive-options" : ["suppress-republish"] }`
There are more details, such has handling disconnects, what to show to the user if the app doesn't get any MQTT updates anymore, reconnecting, and so forth: which is explained in the Google doc called `Victron - designing a UI` chapter MQTT.
*How about retained messages?*
For backward compatibility reasons, especially VRM and VictronConnect need attention. We can do this client-side by interpreting 'retained' messages depending on Venus capability bit, or we can maybe let the server deal with. Pending discussion.
* HTML5 App: should have no code that does something special if received topic has retained flag set.
* guiv2: should have code that ignores retained messages; PR is ready for that [1]. This is to have it compatible with existing cloud brokers. Note that it could be that in one or two months we can delete that code; but by then its also harmless - for later.
* VRM Portal: more complicated story for later, also pending other discussions.
* VictronConnect: pending other discussions.
[1] https://github.com/victronenergy/veutil/commit/d9d618cd2d6bdafc97e56fb537ecdfaa3819a4d1
You're encouraged to read the [readme for the new FlashMQ plugin](https://github.com/victronenergy/dbus-flashmq#keep-alive), for more details.
One extra note: the HTML5 app now uses the selective keep-alives. Which is a bit of a mistake made a long time ago; and this is the moment to correct that. The reason to not use it is that it causes unnecessary CPU load at GX device.
### Todos for all local MQTT traffic:
- [x] ~Add FlashMQ + dbus-plugin into Venus OS~ - done per Venus OS v3.10~1
- [x] Test guiv2 - in progress
- [x] Change guiv2 wrt keep-alive and retain - [done](https://github.com/victronenergy/gui-v2/pull/369)
- [x] Test html5 app - in progress, see `#venus-html5` channel.
- [x] Change html5 app wrt keep-alive and retain
- [x] Change configs to make FlashMQ handle local MQTT + websocket traffic
- [x] Test that `system/0/Ac/ActiveIn/Source` and other paths mentioned in #936 behave as per spec (json `null`).
- [ ] Implement authentication against the bcrypt in `/data/conf/vncpassword.txt`; [This is done](https://github.com/victronenergy/dbus-flashmq/tree/feat-auth-with-vncpassword), but not yet offering for merge because it may surprise/confuse the testers if they have a VNC password.
- [x] Make sure the [venus-influx-data-loader](https://github.com/victronenergy/venus-influx-loader) project works with flashmq.
### Todos for MQTT traffic via VRM
- [x] Finish broker to broker connection implementation into FlashMQ (action @wiebeytec; mostly ready for testing, pending next point)
- [x] Make Venus Platform support the dbus call to the 'bridge registrator'. See [this branch](https://github.com/victronenergy/dbus-flashmq/tree/WIP-register-at-vrm-through-dbus-ipc).
- [x] Change VRM Portal implementation wrt keep-alive and retain
- [x] Test
- [x] Remove Mosquitto + dbus-mqtt completely.
- [ ] ~Remove all retained message DBs from the servers? What about older clients? Otherwise apps still need to include logic to ignore messages with the 'retain' flag, and that caused a lot of confusion so far, so a one-size fits all solution is preferred.~
- [x] Decide on a method for backwards compatibility wrt retained messages: based on a new capability bit that shows GX runs without retained messages: have everybody drop them, or run FlashMQ on the cloud servers and port the auth system so we can not relay retained messages.
### How to test:
1. Install Venus OS v3.10~3, or later
2. in /etc/mosquitto/mosquitto.conf, change the websockets listener port from 9001 to something else.
3. in /etc/flashmq/flashmq.conf change the websockets listener port from 9002 to 9001
4. restart both (in the right sequence 🙂 ); and wait a bit after stopping Mosquitto, since it waits a while before the port becomes available.
but as I think your [lumikcz] you already discovered that
The switch to FlashMQ doesn’t break MQTT.
I use the SSL certificate from Venus https/SSL, downloaded using browser.
That’s not correct. The certificate in use by the MQTT server is found on the GX as /data/keys/mosquitto.crt
. It’s still called mosquitto
, but that’s just legacy.
Be sure to enable the MQTT service for LAN use, because out of the box, MQTT is only available on websockets.