Mosquitto setup

Hi,

I am having some issues setting up Mosquitto. I searched the forums (as well as various pages on mosquitto and owntracks), but they all seemed to have more… advances issues than me…

Some info (I’m not sure which parts are relevant):
I have Homeassistant set up with a duckdns and let’s encrypt certificates, and working fine.
I’ve set up Mosquitto, and made certificates using this guide from owntracks, and the github repo linked to for TSL certificates.

I’ve gotten so far that I can run:

pi@hassbian:/etc/mosquitto $ sudo /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
1491740098: mosquitto version 1.3.4 (build date 2014-08-22 06:10:51+0000) starting
1491740098: Config loaded from /etc/mosquitto/mosquitto.conf.
1491740098: Opening ipv4 listen socket on port 1883.

But when I try to subscribe to a certain topic, as in that (and other) guides, I get

pi@hassbian:/etc/mosquitto $ mosquitto_sub --cafile /etc/mosquitto/conf.d/ca.crt -h hassbian -p 1883 -v -t 'owntracks/#'
Error: Connection refused

I also tried specifying adding arguements for the other *.crt and *.key file, as in this guide I found:

pi@hassbian:/etc/mosquitto $ mosquitto_sub -h hassbian -p 1883 -v -t 'owntracks/#' --cafile /etc/mosquitto/conf.d/ca.crt --cert /etc/mosquitto/conf.d/hassbian.crt --key /etc/mosquitto/conf.d/hassbian.key
Error: Problem setting TLS options.

I wasn’t completely sure about the hostname, so I also tried 127.0.0.1 and my REDACTED.duckdns.org, after port forwarding 1883 and 8883.

My /etc/mosquitto/conf.d/mosquitto.conf file has:

# allow_anonymous false
autosave_interval 1800

connection_messages true
log_dest stderr
log_dest topic
log_type error
log_type warning
log_type notice
log_type information
log_type all
log_type debug
log_timestamp true

#message_size_limit 10240

#password_file jp.pw
#acl_file jp.acl

persistence true
persistence_file mosquitto.db
persistent_client_expiration 1m

#pid_file xxxx

retained_persistence true

#listener 1883
listener 1883 127.0.0.1

#listener 8883
tls_version tlsv1
cafile /etc/mosquitto/certs/ca.crt
certfile /etc/mosquitto/conf.d/hassbian.crt
keyfile /etc/mosquitto/conf.d/hassbian.key
require_certificate false

and my /etc/mosquitto/mosquitto.conf is left as is (i.e.):


# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid
persistence_location /var/lib/mosquitto/

persistence true

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d
```

An `ls -la /etc/mosquitto/conf.d` shows 

drwxr-xr-x 2 mosquitto nogroup 4096 Apr 9 02:06 .
drwxr-xr-x 5 mosquitto nogroup 4096 Apr 9 01:34 …
-r–r--r-- 1 mosquitto nogroup 1326 Apr 9 01:43 ca.crt
-r-------- 1 mosquitto nogroup 1704 Apr 9 01:43 ca.key
-rw-r–r-- 1 mosquitto nogroup 17 Apr 9 01:43 ca.srl
-r–r--r-- 1 mosquitto nogroup 1899 Apr 9 01:43 hassbian.crt
-rw-r–r-- 1 mosquitto nogroup 1001 Apr 9 01:43 hassbian.csr
-r-------- 1 mosquitto nogroup 1679 Apr 9 01:43 hassbian.key
-rw-r----- 1 mosquitto nogroup 634 Apr 9 02:23 mosquitto.conf
-rw-r----- 1 mosquitto nogroup 630 Apr 9 01:43 mosquitto.conf-20170409-015312
-rw-r----- 1 mosquitto nogroup 630 Apr 9 01:53 mosquitto.conf-20170409-020615
-rw-r–r-- 1 mosquitto nogroup 142 Aug 22 2014 README

And here, I am stuck. I've tried a few times, using slightly different guides, but this is as far as I get. With [another guide](http://rockingdlabs.dunmire.org/exercises-experiments/ssl-client-certs-to-secure-mqtt), it was also the part where I tried to subscribe that failed (Step 3 in that). Anyone know what I'm doing wrong?

I also had another (maybe stupid) question though, that might make things easier? Since I already have let's encrypt certificates in

`/etc/letsencrypt/live/REDACTED.duckdns.org/`

Can I use the same for mosquitto? Just point to those *.pem files in the sudo nano 

`/etc/mosquitto/conf.d/mosquitto.conf`?

I tried just pointing to them, but that didn't work (maybe due to ownership?)
1 Like

You’re using tlsv1. I believe it needs to be tlsv1.2

remember 1883 is unencrypted, 8883 is encrypted (TLS)

You’re trying to sub to a topic but you aren’t specifying a user or password, let alone a user key/cert.
Have you set up users/passwords?

Are you running mosquitto on the same system as HA? If so use -h localhost

Try setting up user names and passwords and go from there. You should be able to subscribe from the command line of the server with a simple

mosquitto_sub -t "#" -h localhost

on the 1883 port

@forsquirel
I’m running on the same system as HA, so localhost is the way to go for that. And this, I can get up and running. Thanks, at least something is working now. For testing this, the /etc/mosquitto/mosquitto.conf is:

# allow_anonymous false
autosave_interval 1800

connection_messages true
log_dest stderr
log_dest topic
log_type error
log_type warning
log_type notice
log_type information
log_type all
log_type debug
log_timestamp true

#message_size_limit 10240

password_file /etc/mosquitto/passwd
#password_file jp.pw
#acl_file jp.acl

persistence true
persistence_file mosquitto.db
persistent_client_expiration 1m

#pid_file xxxx

retained_persistence true

listener 1883
#listener 1883 127.0.0.1

#listener 8883
#tls_version tlsv1.2
#cafile /etc/mosquitto/certs/ca.crt
#certfile /etc/mosquitto/conf.d/hassbian.crt
#keyfile /etc/mosquitto/conf.d/hassbian.key
#require_certificate false

I though that either by using

/etc/mosquitto $ mosquitto_sub -h hassbian -p 1883 -v -t 'owntracks/#' --cafile /etc/mosquitto/conf.d/ca.crt --cert /etc/mosquitto/conf.d/hassbian.crt --key /etc/mosquitto/conf.d/hassbian.key

(though I was using the wrong port as you pointed out) or just by adding the paths in the config would specify the key/cert? In any case, your approach (after a reboot) works!

But when I try going back to the TLS (after changing the *.conf file again to add paths to *.crt and *.key fils, and changing back to listener 8883), I get into trouble.

mosquitto_sub -t 'owntracks/#' -h localhost -p 8883 -d
Client mosqsub/860-hassbian sending CONNECT
Client mosqsub/860-hassbian sending CONNECT

where it just repeats the last line. Same happens if I specify the keys and cert as arguments

mosquitto_sub -t 'owntracks/#' -h localhost -p 8883 -d --cert /etc/mosquitto/conf.d/hassbian.crt --key /etc/mosquitto/conf.d/hassbian.key
Client mosqsub/884-hassbian sending CONNECT
Client mosqsub/884-hassbian sending CONNECT

though if I also specify the ca.crt, I just get:

mosquitto_sub -t 'owntracks/#' -h localhost -p 8883 -d --cafile /etc/mosquitto/conf.d/ca.crt --cert /etc/mosquitto/conf.d/hassbian.crt --key /etc/mosquitto/conf.d/hassbian.key
Error: Problem setting TLS options.

So it’s not connecting, but the “-d” option isn’t really giving me anything that I know what to do with.

Ok you’re specifying port 8883 but you’ve commented that section out. gimme a sec, i’m gonna throw mine up on hastebin. Your only problem at this time may be that you’ve removed the TLS sections, but continue reading just in case.

this is my working mosquitto.conf file

https://hastebin.com/ciruhowibe.nginx

for port 1883 I’m using a simple user/pass authentication which is just my home_assistant server. Doesn’t make sense to use TLS on a localhost connection. Now if my user/pass combo were to be figured out? well then just nix the port 1883 section and go only TLS or remove the user/pass combo from the passwd file.

I’m using a key/cert combo that I found that works after reading up on a few sites…

helped out a lot. As for setting up keys and certs I used the Generate-CA.sh script from

I’ll be honest, the hardest thing I set up was my mosquitto broker and an nginx reverse proxy. I had never been so stumped. Following the guide I mentioned and my CONF as a guidline should get you working.

Hope this helps!

I’m sorry if I didn’t specify that clearly, but when I tried that I edited the *.conf file back again, so it looks basically like in the first post (changed to tls1.2, but I think otherwise it’s like the first).

# allow_anonymous false
autosave_interval 1800

connection_messages true
log_dest stderr
log_dest topic
log_type error
log_type warning
log_type notice
log_type information
log_type all
log_type debug
log_timestamp true

#message_size_limit 10240

password_file /etc/mosquitto/passwd
#password_file jp.pw
#acl_file jp.acl

persistence true
persistence_file mosquitto.db
persistent_client_expiration 1m

#pid_file xxxx

retained_persistence true

#listener 1883
#listener 1883 127.0.0.1

listener 8883
tls_version tlsv1.2
cafile /etc/mosquitto/certs/ca.crt
certfile /etc/mosquitto/conf.d/hassbian.crt
keyfile /etc/mosquitto/conf.d/hassbian.key
require_certificate false

I will take a look and try with your cert script when I’m home. I’ll keep you posted. Thanks!

Sooo… not quite working. I tried following the guide you provided, and tried step by step to find where it’s failing (because it still is).

I tried completely uninstalling and re-installing mosquitto (though I didn’t uninstall libmosquitto or mosquitto-client, is there anything with them, that could be messed up?)

I can get the most basic to work; just a

mosquitto_sub -v -t "test" -d

in one terminal and a

mosquitto_pub -t "test" -m "Hello"

in the other (it apparently defaults to localhost and port 18883) will send and receives just fine. Same if I do the above, but specify port 8883, that works. But as soon as I try to specify the ca.crt file, it goes awry:

mosquitto_sub --cafile /etc/mosquitto/certs/ca.crt -v -t "test" -p 8883 -d
Unable to connect (8).

So as best I can tell, I’m messing something up with the certs/keys/something like that. I am running all this as user:pi. If I try sudo’ing (not sure if that even makes sense in this case, but):

Client mosqsub/1264-hassbian sending CONNECT
Error: Protocol error

Do I need to run the *.sh as a specific user? I have just set chown mosquitto:nogroup to all certs/keys after running, and putting them into folders with the same owner.

so:

ls -la /etc/mosquitto/certs/
total 28
drwxr-xr-x 2 root      root    4096 Apr 11 12:37 .
drwxr-xr-x 5 root      root    4096 Apr 11 12:21 ..
-r--r--r-- 1 mosquitto nogroup 1326 Apr 11 12:15 ca.crt
-r--r--r-- 1 mosquitto nogroup 1899 Apr 11 12:37 hassbian.crt
-rw-r--r-- 1 mosquitto nogroup 1001 Apr 11 12:37 hassbian.csr
-r-------- 1 mosquitto nogroup 1679 Apr 11 12:37 hassbian.key
-rw-r--r-- 1 root      root     130 Feb 21 00:48 README

I also tried adding a bit to the conf file under TSL connections:

# MQTT over TLS/SSL
listener 8883
cafile /etc/mosquitto/certs/ca.crt
certfile /etc/mosquitto/certs/hassbian.crt
keyfile /etc/mosquitto/certs/hassbian.key
log_type error
log_type warning
log_type notice
log_type information

and now when I try connecting, and specifying the cafile, I get an Error: Protocol error.

After googling this for a while, I found that (maybe?) it is an issue with hostname verification. I run:

openssl x509 -in /etc/mosquitto/certs/ca.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            b5:c8:71:0c:65:64:c6:56
    Signature Algorithm: sha512WithRSAEncryption
        Issuer: CN=An MQTT broker, O=OwnTracks.org, OU=generate-CA/[email protected]
        Validity
            Not Before: Apr 11 10:13:50 2017 GMT
            Not After : Apr  7 10:13:50 2032 GMT
        Subject: CN=An MQTT broker, O=OwnTracks.org, OU=generate-CA/[email protected]
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)

As I can read, the “CN” should match the host I am connecting to (localhost or hassbian?), and it doesn’t. But if this is the issue, I can’t really figure out what to do about it?

mosquitto_sub --cafile /etc/mosquitto/certs/ca.crt -v -t "test" -p 8883 -d

you’re not specifying the user key/cert in that line, that’s 1 thing. Have you generated the user files?

referenced here

After restarting the broker will should not be able to logon anonymously:

# mosquitto_pub --cafile /etc/mosquitto/certs/ca.crt -h localhost -t "test" -m "message" -p 8883 -d
Client mosqpub/25688-pcortex sending CONNECT
Client mosqpub/25688-pcortex received CONNACK
Connection Refused: not authorised.
Error: The connection was refused.
**But providing the user name and password:**

# mosquitto_pub --cafile /etc/mosquitto/certs/ca.crt -h localhost -t "test" -m "message" -p 8883 -d -u 
user1 -P secret
Client mosqpub/25709-pcortex sending CONNECT
Client mosqpub/25709-pcortex received CONNACK
Client mosqpub/25709-pcortex sending PUBLISH (d0, q0, r0, m1, 'test', ... (7 bytes))
Client mosqpub/25709-pcortex sending DISCONNECT 

and from, for generating user key/cert, http://owntracks.org/booklet/features/tlscert/

./generate-CA.sh client username

as for the last part, that’s not an issue. The only time you need something to match is with your server files and you modify that in the generate-ca.sh file. Look for

IPLIST="192.168.1.XXX 127.0.0.1"
HOSTLIST="localhost hostname your.ddns.hostname"

make sure those lines are populated correctly and not commented out.

As for permissions all my files are root:root

Sorry you’re having so much trouble.

Man, this Mosquitto is just insisting on not being cool about this.

But I think now, after all your help, that I probably just don’t know what I’m doing with making the certs.

I just followed the guides (first the two I found, then the one you provided) pretty much to the letter. But that didn’t include info about altering the generate-CA.sh. And let me straight up admit, I don’t get most of that script. So it’s probably just that I didn’t do/change something that’s obvious to others?

I did change it a bit, uncommenting the IPLIST and HOSTLIST, as you also mentioned, and added a few IPs and hosts (where 192.168.0.160 is the local IP of my homeassistant/mosquitto pi, hassbian is the hostname, the duckdns is what I use to access from outside of home network.).

#!/usr/bin/env bash
#(@)generate-CA.sh - Create CA key-pair and server key-pair signed by CA

# Copyright (c) 2013-2016 Jan-Piet Mens <jpmens()gmail.com>
# All rights reserved.
# 
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
# 
# 1. Redistributions of source code must retain the above copyright notice,
#    this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
# 3. Neither the name of mosquitto nor the names of its
#    contributors may be used to endorse or promote products derived from
#    this software without specific prior written permission.
# 
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

#
# Usage:
#       ./generate-CA.sh                creates ca.crt and server.{key,crt}
#       ./generate-CA.sh hostname       creates hostname.{key,crt}
#       ./generate-CA.sh client email   creates email.{key,crt}
#
# Set the following optional environment variables before invocation
# to add the specified IP addresses and/or hostnames to the subjAltName list
# These contain white-space-separated values
#
IPLIST="127.0.0.1 192.168.0.160"
HOSTLIST="localhost hassbian REDACTED.duckdns.org"

set -e

export LANG=C

kind=server

if [ $# -ne 2 ]; then
        kind=server
        host=$(hostname -f)
        if [ -n "$1" ]; then
                host="$1"
        fi
else
        kind=client
        CLIENT="$2"
fi

[ -z "$USER" ] && USER=root

DIR=${TARGET:='.'}
# A space-separated list of alternate hostnames (subjAltName)
# may be empty ""
ALTHOSTNAMES=${HOSTLIST}
ALTADDRESSES=${IPLIST}
CA_ORG='/O=OwnTracks.org/OU=generate-CA/[email protected]'
CA_DN="/CN=An MQTT broker${CA_ORG}"
CACERT=${DIR}/ca
SERVER="${DIR}/${host}"
SERVER_DN="/CN=${host}$CA_ORG"
keybits=2048
openssl=$(which openssl)
MOSQUITTOUSER=${MOSQUITTOUSER:=$USER}

# Signature Algorithm. To find out which are supported by your
# version of OpenSSL, run `openssl dgst -help` and set your
# signature algorithm here. For example:
#
#       defaultmd="-sha256"
#
defaultmd="-sha512"

function maxdays() {
        nowyear=$(date +%Y)
        years=$(expr 2032 - $nowyear)
        days=$(expr $years '*' 365)

        echo $days
}

function getipaddresses() {
        /sbin/ifconfig |
                grep -v tunnel |
                sed -En '/inet6? /p' |
                sed -Ee 's/inet6? (addr:)?//' |
                awk '{print $1;}' |
                sed -e 's/[%/].*//' |
                egrep -v '(::1|127\.0\.0\.1)'   # omit loopback to add it later
}


function addresslist() {

        ALIST=""
        for a in $(getipaddresses); do
                ALIST="${ALIST}IP:$a,"
        done
        ALIST="${ALIST}IP:127.0.0.1,IP:::1,"

        for ip in $(echo ${ALTADDRESSES}); do
                ALIST="${ALIST}IP:${ip},"
        done
        for h in $(echo ${ALTHOSTNAMES}); do
                ALIST="${ALIST}DNS:$h,"
        done
        ALIST="${ALIST}DNS:localhost"
        echo $ALIST

}

days=$(maxdays)

if [ -n "$CAKILLFILES" ]; then
        rm -f $CACERT.??? $SERVER.??? $CACERT.srl
fi

if [ ! -f $CACERT.crt ]; then

        #    ____    _    
        #   / ___|  / \   
        #  | |     / _ \  
        #  | |___ / ___ \ 
        #   \____/_/   \_\
        #                 

        # Create un-encrypted (!) key
        $openssl req -newkey rsa:${keybits} -x509 -nodes $defaultmd -days $days -extensions v3_ca -keyout $CA$
        echo "Created CA certificate in $CACERT.crt"
        $openssl x509 -in $CACERT.crt -nameopt multiline -subject -noout

        chmod 400 $CACERT.key
        chmod 444 $CACERT.crt
        chown $MOSQUITTOUSER $CACERT.*
        echo "Warning: the CA key is not encrypted; store it safely!"
fi


if [ $kind == 'server' ]; then

        #   ____                           
        #  / ___|  ___ _ ____   _____ _ __ 
        #  \___ \ / _ \ '__\ \ / / _ \ '__|
        #   ___) |  __/ |   \ V /  __/ |   
        #  |____/ \___|_|    \_/ \___|_|   
        #                                  

        if [ ! -f $SERVER.key ]; then
                echo "--- Creating server key and signing request"
                $openssl genrsa -out $SERVER.key $keybits
                $openssl req -new $defaultmd \
                        -out $SERVER.csr \
                        -key $SERVER.key \
                        -subj "${SERVER_DN}"
                chmod 400 $SERVER.key
                chown $MOSQUITTOUSER $SERVER.key
        fi

        if [ -f $SERVER.csr -a ! -f $SERVER.crt ]; then

                # There's no way to pass subjAltName on the CLI so
                # create a cnf file and use that.

                CNF=`mktemp /tmp/cacnf.XXXXXXXX` || { echo "$0: can't create temp file" >&2; exit 1; }
                sed -e 's/^.*%%% //' > $CNF <<\!ENDconfig
                %%% [ JPMextensions ]
                %%% basicConstraints        = critical,CA:false
                %%% nsCertType              = server
                %%% keyUsage                = nonRepudiation, digitalSignature, keyEncipherment
                %%% nsComment               = "Broker Certificate"
                %%% subjectKeyIdentifier    = hash
                %%% authorityKeyIdentifier  = keyid,issuer:always
                %%% subjectAltName          = $ENV::SUBJALTNAME
                %%% # issuerAltName           = issuer:copy
                %%% ## nsCaRevocationUrl       = http://mqttitude.org/carev/
                %%% ## nsRevocationUrl         = http://mqttitude.org/carev/
                %%% certificatePolicies     = ia5org,@polsection
                %%% 
                %%% [polsection]
                %%% policyIdentifier        = 1.3.5.8
                %%% CPS.1                   = "http://localhost"
                %%% userNotice.1            = @notice
                %%% 
                %%% [notice]
                %%% explicitText            = "This CA is for a local MQTT broker installation only"
                %%% organization            = "OwnTracks"
                %%% noticeNumbers           = 1

!ENDconfig

                SUBJALTNAME="$(addresslist)"
                export SUBJALTNAME              # Use environment. Because I can. ;-)

                echo "--- Creating and signing server certificate"
                $openssl x509 -req $defaultmd \
                        -in $SERVER.csr \
                        -CA $CACERT.crt \
                        -CAkey $CACERT.key \
                        -CAcreateserial \
                        -CAserial "${DIR}/ca.srl" \
                        -out $SERVER.crt \
                        -days $days \
                        -extfile ${CNF} \
                        -extensions JPMextensions

                rm -f $CNF
                chmod 444 $SERVER.crt
                chown $MOSQUITTOUSER $SERVER.crt
        fi
else
        #    ____ _ _            _   
        #   / ___| (_) ___ _ __ | |_ 
        #  | |   | | |/ _ \ '_ \| __|
        #  | |___| | |  __/ | | | |_ 
        #   \____|_|_|\___|_| |_|\__|
        #                            

        if [ ! -f $CLIENT.key ]; then
                echo "--- Creating client key and signing request"
                $openssl genrsa -out $CLIENT.key $keybits

                CNF=`mktemp /tmp/cacnf-req.XXXXXXXX` || { echo "$0: can't create temp file" >&2; exit 1; }
                # Mosquitto's use_identity_as_username takes the CN attribute
                # so we're populating that with the client's name
                sed -e 's/^.*%%% //' > $CNF <<!ENDClientconfigREQ
                %%% [ req ]
                %%% distinguished_name  = req_distinguished_name
                %%% prompt                      = no
                %%% output_password             = secret
                %%% 
                %%% [ req_distinguished_name ]
                %%% # O                       = OwnTracks
                %%% # OU                      = MQTT
                %%% # CN                      = Suzie Smith
                %%% CN                        = $CLIENT
                %%% # emailAddress            = $CLIENT
!ENDClientconfigREQ

                $openssl req -new $defaultmd \
                        -out $CLIENT.csr \
                        -key $CLIENT.key \
                        -config $CNF
                chmod 400 $CLIENT.key
        fi

        if [ -f $CLIENT.csr -a ! -f $CLIENT.crt ]; then

                CNF=`mktemp /tmp/cacnf-cli.XXXXXXXX` || { echo "$0: can't create temp file" >&2; exit 1; }
                sed -e 's/^.*%%% //' > $CNF <<\!ENDClientconfig
                %%% [ JPMclientextensions ]
                %%% basicConstraints        = critical,CA:false
                %%% subjectAltName          = email:copy
                %%% nsCertType              = client,email
                %%% extendedKeyUsage        = clientAuth,emailProtection
                %%% keyUsage                = digitalSignature, keyEncipherment, keyAgreement
                %%% nsComment               = "Client Broker Certificate"
                %%% subjectKeyIdentifier    = hash
                %%% authorityKeyIdentifier  = keyid,issuer:always

!ENDClientconfig

                SUBJALTNAME="$(addresslist)"
                export SUBJALTNAME              # Use environment. Because I can. ;-)

                echo "--- Creating and signing client certificate"
                $openssl x509 -req $defaultmd \
                        -in $CLIENT.csr \
                        -CA $CACERT.crt \
                        -CAkey $CACERT.key \
                        -CAcreateserial \
                        -CAserial "${DIR}/ca.srl" \
                        -out $CLIENT.crt \
                        -days $days \
                        -extfile ${CNF} \
                        -extensions JPMclientextensions

                rm -f $CNF
                chmod 444 $CLIENT.crt
        fi
fi

Is there anything else I should have altered?
I can get

mosquitto_sub -t "owntracks/#" -u aephir -P MosquittoPie -d --cafile /etc/mosquitto/ca_certificates/ca.crt -p 8883

to work, with

mosquitto_pub -t "owntracks/test" -u aephir -P MosquittoPie -d --cafile /etc/mosquitto/ca_certificates/ca.crt -p 8883 -m "Hello"

With this, when I have my /etc/mosquitto/mosquitto.conf as:

# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

log_type error
log_type warning
log_type notice
log_type information
#topic owntracks/
connection_messages true
log_timestamp true
allow_anonymous false
password_file /etc/mosquitto/passwd
persistence_file mosquitto.db

and my /etc/mosquitto/conf.d/mosquitto.conf as

#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

# Plain MQTT protocol
listener 1883
log_dest syslog
log_dest stdout
log_dest topic
log_type error
log_type warning
log_type notice
log_type information
connection_messages true
log_timestamp true

# End of plain MQTT configuration

# MQTT over TLS/SSL
listener 8883
cafile /etc/mosquitto/ca_certificates/ca.crt
certfile /etc/mosquitto/certs/hassbian.crt
keyfile /etc/mosquitto/certs/hassbian.key
log_type error
log_type warning
log_type notice
log_type information
#tls_Version tls1.2

# End of MQTT over TLS/SLL configuration

But this, of course, doesn’t have the “require cert” that you have in your conf, and when I add that, it fails.

Here’s keeping my fingers crossed that it’s just some stupid thing in the generate-CA.sh script I didn’t know to change.

EDIT: If this matters, as soon as I un-comment tls_Version tls1.2 to the config, I get an Error: Connection refused when trying use the mosquitto_sub I listed above

Ok, sorry it’s taken so long to get back. I worked Wednesday and 24 hour shifts don’t give much time to do home stuff and I was booked all day yesterday. I’ve kinda made a quick video of the setup. It should be super simple. Just adjust the directories to the stuff you want to use. I just used TEMP/ for vid purposes. Should also add this uses the my config file from a previous post up top.

I did leave out one thing, when making a second user/pass combow with mosquitto_passwd you don’t need the -c tag or else it will over write your current password file

mosquitto_passwd -c passwordfile username

for first user and passwordfile creation

mosquitto_password passwordfile  username2

for any additional users.

This should get you up and working.

Wow, thanks a lot! I can hardly complain over anything taking “long” when you’re spending you free time trying to help.

I will give it another go when I get home. And be really embarrassed if I still can’t make it work…

Well, the more I mess around with what you tell me, the closer I get to understanding what, exactly, it is I don’t understand :slight_smile:

And the reason I’m slow to reply is that I’m really trying to figure out the rest by myself (believe it or not).

Briefly, I can communicate from HASS to terminal with mosquitto_sub, but not from external.

As far as I can tell, mosquitto runs at boot.

sudo service mosquitto status
● mosquitto.service - LSB: mosquitto MQTT v3.1 message broker
   Loaded: loaded (/etc/init.d/mosquitto)
   Active: active (exited) since Sat 2017-04-15 17:50:55 CEST; 1min 52s ago
  Process: 474 ExecStart=/etc/init.d/mosquitto start (code=exited, status=0/SUCCESS)

Apr 15 17:50:55 hassbian systemd[1]: Starting LSB: mosquitto MQTT v3.1 message broker...
Apr 15 17:50:55 hassbian mosquitto[474]: Starting network daemon:: mosquitto.
Apr 15 17:50:55 hassbian systemd[1]: Started LSB: mosquitto MQTT v3.1 message broker.
pi@hassbian:~ $ 

But nothing works just yet, and homeassistant gives an error about setting up MQTT. Until I manually start as daemon with sudo mosquitto -d (just running mosquitto -d starts as user:pi), whereafter:

ps -ef | grep '[m]osquitto'
mosquit+   890     1  0 17:55 ?        00:00:00 mosquitto -d

Restarting home assistant now enables sub/pub:

mosquitto_sub -h 127.0.0.1 -v -t "home-assistant/#" -d
Client mosqsub/926-hassbian sending CONNECT
Client mosqsub/926-hassbian received CONNACK
Client mosqsub/926-hassbian sending SUBSCRIBE (Mid: 1, Topic: home-assistant/#, QoS: 0)
Client mosqsub/926-hassbian received SUBACK
Subscribed (mid: 1): 0
Client mosqsub/926-hassbian sending PINGREQ
Client mosqsub/926-hassbian received PINGRESP
Client mosqsub/926-hassbian received PUBLISH (d0, q0, r0, m0, 'home-assistant/test', ... (17 bytes))
home-assistant/test I hope this works

I can send this text from both terminal (as user:pi) and from the services tab in home assistant. Communication on the loopback, and user/password in home assistant configuration is fine (so that’s one step better than before). Also running a daemon on port 8883, now:

ps -ef | grep '[m]osquitto'
mosquit+   890     1  0 17:55 ?        00:00:00 mosquitto -d
mosquit+  1005     1  0 18:01 ?        00:00:00 mosquitto -d -p 8883 -v

But the client (certs, I suppose) are still an issue. Sorry. The last part of my /var/log/mosquitto/mosquitto.log is:

1492197517: New connection from MY_PHONE'S_IP_ADDRESS on port 8883.
1492197517: OpenSSL Error: error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown
1492197517: OpenSSL Error: error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure
1492197517: Socket error on client <unknown>, disconnecting.
1492198209: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
1492198514: Error in poll: Interrupted system call.
1492198514: mosquitto version 1.4.11 terminating

There is still no connection from outside (in this case, owntracks). I get an error in the owntrack app

Error
Endpoint state
MqttException (0) - javax.net.ssl.SSL.HandshakeException: 
SSK handshake.aborted: ssl=0xbc1d7840: 
I/O error during system call, Connection reset by peer

Just for info (since we’ve established I don’t completely know what I’m doing), my owntracks is set up as:


Mode: Private MQTT
Host: https://MY.duckdns.org
port: 8883
Use Websockets: Off
Identification:
Username: CLIENTNAME (the name used the using the generate-CA.sh client clientname, which is the same as the username I used to create one set of username/password)
Password: PASSOWRD (the password used with the username above when using mosquitto_passwd command)
Device ID: phone (this should’t matter for the connection, right?)
Tracker ID: mine (shouldn’t matter either?)
Security:
TLS: enabled
CA certfile: ca.crt (the one created with the first execution of the generate-CA.sh)
Client certificate: CLIENTNAME-1.p12 (the p12 file created as instructed, and imported into my phone)
Client certificate password: P12_PASS (the password I made when I created the *.p12 file)
Parameters: Off, nothing done here.


Sooo… sorry for all the questions and issues I have. And thanks so far!

1 Thing I noticed is that you’re running 2 instances of mosquitto?

You should only have to run 1 instance, and with proper config it will differentiate between the two. The other possibility is that your config file isn’t pointing directly to your certs/keys.

Your android setup looks right so It’s looking like a broker error.

Long story short; it works. First off, thanks a lot, @forsquirel, I really didn’t want to take as much of your time as I did!

Unfortunately, I can’t give a full troubleshooting, since the thing that finally made it work was a fresh install of everything. Raspbian, homeassistant, duckdns, letsencrypt, mariadb, mosquitto, etc.

What I can say that I did differently (without knowing if it is related), is:

  • I did a “manual” install of homeassistant, in a virtual environment (last time I think I used the “all-in-one” install)
  • I ran the mosquitto-setup.sh included in the tools for owntracks mosquitto certs (links in previous posts), then modified the settings after to suit my needs instead of manually modifying all *.conf.

But I still followed the guides, exactly the same way as the last ~5 times (not like the 5-10 times before that; I’m slightly smarter than I was then). Other than that, I really don’t think I did anything differently.

If anyone finds this thread, and needs this (probably not), my /etc/mosquitto/mosquitto.conf has:

# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

And my /etc/mosquitto/conf.d/mosquitto.conf has:

# allow_anonymous false
autosave_interval 1800

connection_messages true
log_dest stderr
log_dest topic
log_type error
log_type warning
log_type notice
log_type information
log_type all
log_type debug
log_timestamp true

#message_size_limit 10240

password_file /etc/mosquitto/tools/TLS/password
#acl_file jp.acl

persistence true
persistence_file mosquitto.db
persistent_client_expiration 1m

#pid_file xxxx

retained_persistence true

listener 1883
#listener 1883 127.0.0.1
log_type error
log_type warning
log_type notice
log_type information

listener 8883
tls_version tlsv1.2
cafile /etc/mosquitto/tools/TLS/ca.crt
certfile /etc/mosquitto/tools/TLS/MYPRIVATE.duckdns.org.crt
keyfile /etc/mosquitto/tools/TLS/MYPRIVATE.duckdns.org.key
require_certificate false
log_type error
log_type warning
log_type notice
log_type information
1 Like

Glad to hear it’s working!

1 Like

For the error “Error: Problem setting TLS options”, one specific cause was fixed like this:

– in the mosquitto config file, the lines of config parameters with cert and CA filenames contained a ‘space’ character after the filename, and before the end-of-line character.

– removing the space just before the end-of-line character caused the error to no longer appear.

– the mosquitto broker started up with no errors.

hi folks,
so trying to understand what is happening as i have NGINX set up with MQTT. I have another install in the lab at work without NGINX and MQTT works like a charm.

Using NGINX as a reverse proxy (on the HA box) will allow 443 / 80 traffic only from a site (duckdns.domain.com) that has a matching certificate.

What i dont understand is what NGINX does locally on the MQTT network.

I dont need my MQTT traffic encrypted as its only in my network (people can add extra security issues here) - but for the most part i’m trying to get the Wink Relays running on MQTT.

If i’m ok using just username & password then 1883 should do it right? If i decide to use 8883 (and a cert) there are 2 parts i dont understand.

  1. The cert is for duckdns.domain.com and so wont apply for internal traffic (so i need to use a local CA)
  2. how do i get the cert to a wink relay to use it for authentication (that sounds like a major hassle)

Can someone help me understand so i can get MQTT running behind NGINX?

Many thanks

ok, so after testing - incase others run into the same query. NGINX can just handle external commands.

Internally i can use 1883 with just a username and password

:slight_smile:

I have my mosquitto also behind a nginx and it works without any problems. All my internal mqtt devices are in an isolated vlan and connecting on public FQDN with TLS.

For the nginx it doesn´t matter, if the client connection is internal or external.

Maybe your router is missing loopback support?