Setting up Mosquitto Broker with TLS not working

andarotajo · March 19, 2021, 9:02pm

Hi ,

I recently got a miflora plant sensor and currently in the process of setting it up. Since the plant is quite far away from my server, I searched for a kind of extender and found https://github.com/ThomDietrich/miflora-mqtt-daemon

On my Home Assistant I installed the Mosquitto add-on and on a Raspberry Pi I set up the miflora-mqtt-daemon. It works perfectly fine using the port 1883 and no TLS. But I just can’t get the TLS working.

I’ve used several tutorials to create certificates and so on and copied them over to the Raspberry Pi etc. but it just doesn’t work. Everytime I get errors like “bad certificate” or “wrong version number”.

Can anyone please tell me what I actually have to do or maybe some tutorial that works?

CentralCommand · March 19, 2021, 9:36pm

So there’s a number of clarifications needed here:

Does mosquitto start or do you get an error about a bad or missing certificate in the add-on logs?
Assuming it starts, how did you create the certificate?
What URL are you using to connect to mosquitto? Does it match what is on the certiicate?
Assuming you created the certificate yourself, did you provide the miflora app with the certificate of the CA used to sign it so it can verify your self-signed certificate?
Is require_certificates true or false in the mosquitto add-on’s configuration?

andarotajo · March 19, 2021, 9:45pm

I’ll do my best to answer all the questions as my knowledge about the certificate stuff is pretty limited.

The add-on starts without problems and listens on the standard ports. The error occurs when I try to run the miflora-mqtt-daemon. It then fails to connect (only when I activate the TLS stuff)
I tried various guides but last one I used was https://hackaday.io/project/163666-secure-esp8266-mqtt-poc/log/158992-secure-mqtt-broker-setup
In the miflora-mqtt-daemon I’ve set the URL to the IP of my Home Assistant server. In the certificates I’ve created I’ve always used this IP for the CN
I did copy the CA over, yeah. Since both systems are accessed by SSH, I did this by creating a file and copying the content over. I’ve also read something about a .pem extension, but the guides always had .crt ones. Don’t know, what difference there is tho
I’ve tried both but I can’t remember, if the add-on always gave the same error.

CentralCommand · March 19, 2021, 10:10pm

Ok so first problem is #3 - you can’t do that. In that hackaday guide you sent me notice in the step labeled “Generate the MQTT Server self-signed certificate” it has this in the command CN=your-hostname. Your supposed to fill that in and whatever you put in there has to exactly match the host you access MQTT at. It also can’t be an IP address, it has to be a hostname. Can the miflora app get to HA using homeassistant.local? If so that’ll work. If not you’ll need a way to get to HA via a hostname and not by IP so you can put that in the certificate.

The reason is because that’s how TLS certificate verification works. When it gets a certificate first thing it does is see if it was signed by someone it trusts. It sounds like you took care of that based on your response in #4 so that should be ok. Next it checks to see if the certificate is actually for the service it was trying to contact, it does this by checking if its common name (CN) (or one of its alternate names if you added those) matches the hostname it was trying to reach it at. That’s where its failing because you aren’t even giving it a hostname to match on, you’re just using an IP address.

If you really truly cannot use a hostname and can only access HA by IP then you may be able to get that to work. Some googling turned up this guide for making a certificate that can be verified even when accessing the service by IP address. YMMV though, as you can see in that guide even if you follow all the steps Firefox still won’t trust it. So it depends on whether the miflora app will trust that since it has no way to skip verification, it must be able to verify in order to connect.

Also set require_certificates to false, it’s a whole separate thing. Enabling that turns on mTLS (or mutual TLS). Essentially it requires the client to also present a certificate and then Mosquitto checks to see if it trusts it before allowing that client to connect. It’s an alternate form of authentication, probably not what you’re looking for right now.

EDIT: Oh sorry, PEM vs. CRT shouldn’t matter. PEM is the format you generated the certificate in based on the guide you linked. But extensions are just sugar for GUIs and remembering what things are, it doesn’t actually matter what extension you pick or if the file has one at all. As long as the path you provide the add-on points to a certificate file that exists its good with it.

CentralCommand · March 19, 2021, 10:18pm

Sorry I kind of went on auto-pilot once I saw you used the IP address but I’m now realizing you said you always use the IP address as the CN. I guess I don’t know what apps you’re using, things I’ve tried don’t accept that. Although I have to admit I don’t try all that much since I try to avoid using IP addresses and use hostnames instead. Do you use apps which allow and verify certificates like that or do you usually have to find the “turn off SSL verification” feature that a lot of apps have?

Regardless it sounds like MiFlora won’t accept that. And from scanning MiFlora’s config I don’t see a “turn off ssl verification” option so you have to use something MiFlora accepts. Would recommend trying again using a hostname or trying to make a certificate with the SAN extension like the guide I linked.

andarotajo · March 19, 2021, 10:24pm

Thank you very much for the detailed explanation! I will try it with your suggestions.
As for the hostname in the CN. I can’t use homeassistant.local but the hostname of the system is server-hassio. Now my question is, do I just set the CN as server-hassio or do I have to put in the port as well, which I have to use to connect on my browser, so server-hassio:8123?

Sorry, I phrased it wrong I think. What I meant was, that I used the IP in the 3 certificates that the guide from my initial post creates. So the CA, server certificate and client certificate all had the IP as CN

CentralCommand · March 19, 2021, 10:40pm

Perfect! No port required, just server-hassio. Although if you haven’t accessed HA via that hostname before from miflora I’d recommend turning SSL off for a quick test before going further. Just make sure the miflora app is able to connect to HA using that hostname on your network before making certificates and everything. Assuming it works then make a certificate with that name and flip SSL back on.

EDIT: Actually if you can SSH into the miflora container then there’s an even easier way to test. Just run nslookup server-hassio from inside and make sure it comes back with the correct IP.

Oh ok. So in a self-signed certificate situation nothing actually checks the CN of the CA certificate. Unless you’re trying to make a public CA you an basically put whatever you want there. I know I personally just put MY-CA in mine haha.

If you have a client certificate that means you made one for MiFlora and you’re filling in the tls_certfile and tls_keyfile values in its config.ini? So that is mTLS. In that case, that might’ve actually been what is screwing this whole thing up haha.

I would take that out for now and put the client certificate aside. Get everything working without that just with normal SSL. If you do want to do that to ensure no one can talk to your broker without a client certificate then we have a few extra steps.

Mosquitto wants the CN of client certificates to match the client name they use to register with it. I don’t see an option to set client name option in that config.ini file though so that might be a problem. When Miflora does successfully connect, who does it say it connected as in the Mosquitto logs? That’s what you should use for CN. If its random you won’t be able to do that so I guess we’ll have to find out if Mosquitto requires that or just recommends it, not sure.
Set require_certificates to true in the mosquitto configuration. Then it will request a certificate from clients and block them if they don’t have one
Set the cafile option in the mosquitto configuration to the path to your CA certificate. It also won’t accept certificates it doesn’t trust so you need to tell it who signed it.

andarotajo · March 19, 2021, 10:57pm

Just tried it and it didn’t work at first. I added the hostname to the hosts file and after that I managed to connect fortunately Also nslookup didn’t return the IP even after I’ve edited the hosts file

So just the CN of the CA certificate has to be my hostname and the CN of the other certificates is irrelevant?

I actually did yeah haha. I will try without those.

Do you mean the auto… part? Unfortunately that is random, the ones above this entry are different. Maybe I can edit the python file that I execute to use something fixed. But that will be after it works with normal SSL

How do I do this? Is that the Generate the CA signed certificate to use in the MQTT Client part in the guide?

joelp · March 19, 2021, 11:02pm

I used Let’s Encrypt as the CA since I have full control of my domain’s DNS. This solves the trust issue with clients and self-signed. It was pretty straight forward following the Mosquitto docs.

I run Mosquitto in Docker and I needed to add permissions using a group on the host server to allow the container instance to read the certs directory (i.e. /etc/letsencrypt/). The container runs as user mosquitto and needs to be added to that group (note it’s the group id that is important). Something like this:
add group cert-reader chown -R :cert-reader /etc/letsencrypt/
I had to create a Dockerfile with something like this:

FROM eclipse-mosquitto:latest RUN addgroup -S -g 2006 cert-reader && addgroup mosquitto cert-reader

CentralCommand · March 19, 2021, 11:04pm

Yea that’s what I meant, the auto… part. Ok if regular SSL works then yea you’ll have to revisit that. It might be just good practice to have the CN match that and not actually required. There is a mode mosquitto can run in where it uses the CN as the client name but the Mosquitto add-on doesn’t have that option.

You don’t have to generate anything else for this, you already have the CA file since you already made the certificates. You just have to put it in /ssl and think put its path in the cafile option

CentralCommand · March 19, 2021, 11:06pm

Yes this works too. If you have full control over DNS on your network (like say via the Adguard add-on, Pihole or just if your router has advanced DNS options) then you can use DNS rewrite to make it so you can use a public URL without actually leaving your network.

Just to be clear, the author did say they are using the add-on though so was trying to find something that works within those bounds. But if you do have DNS rewrite capability seems like you could set this up with the add-on pretty easily I think? Use the Let’s Encrypt add-on to get a certificate and serve that up from Mosquitto. Then use a DNS rewrite to direct that URL to the LAN address of the Pi within your network.

andarotajo · March 19, 2021, 11:28pm

I think I’ve set everything up as you suggested but I’m getting this error. My add-on config looks like this:

logins:
  - username: '!secret mqtt_username'
    password: '!secret mqtt_password'
anonymous: false
customize:
  active: false
  folder: mosquitto
certfile: mosquitto/serv.crt
keyfile: mosquitto/serv.key
require_certificate: false
cafile: mosquitto/ca.crt

The CN of the CA ist the hostname (server-hassio) and the CN of the server and client certificates are something else. In my miflora app I’ve set the hostname to server-hassio and the port to 8883. Then I’ve set TLS to true and only supplied the path to the CA certificate. Did I miss something that causes this error?

CentralCommand · March 20, 2021, 12:02am

You can remove cafile for now as it is only applies to client certificates. That shouldn’t cause that error though, hm.

What does your config.ini file have on the otherside? I’m not familiar with this app but looking up its config here the MQTT part should look something like this:

[MQTT]
hostname = server-hassio
port = 8883
username = <value from mqtt_username secret>
password = <value from mqtt_password secret>
tls = true
tls_ca_cert = <path to ca.crt in miflora container>

I dropped all the comments and the base topic parts to keep it small. You have the tls parts filled in like this though right?

andarotajo · March 20, 2021, 8:01am

My config.ini looks like this. Maybe the certificate has to be in the PEM format, if that’s any different?
Edit 1: I also tried to connect using MQTT Explorer on my Windows pc and there it works. It had the options for TLS and “Validate Certificate” and I just checked TLS and provided the CA certificate. Maybe it’s something in the source code of the miflora app that’s missing
Edit 2: I tried logging the Exception in the python script and the connection returns the error
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for ‘server-hassio’. (_ssl.c:1056) which doesn’t make sense to me. I think that’s the error I got when I used the MQTT Explorer with “Validate Certificate” set to true.

[General]
reporting_method = homeassistant-mqtt
adapter = hci0

[Daemon]
enabled = true
period = 300

[MQTT]
hostname = server-hassio
port = 8883
keepalive = 60
base_topic = plants
username = mqtt_man
password = redacted

tls = true
tls_ca_cert = /home/pi/certs/ca.crt
#tls_keyfile = /home/pi/certs/client.key
#tls_certfile = /home/pi/certs/client.crt

[Sensors]
plant_1 = C4:7C:8D:6B:A4:10

CentralCommand · March 20, 2021, 2:53pm

Huh, that is really confusing. I guess let’s just confirm the certificate real fast. So in the command line you can do this to see the certificate:

openssl x509 -in my-cert_file.crt -text -noout

That’ll give you output like this. Note that you can chop out the big blocks of hex when pasting it in, that’s all fine, just looking at the header info. Also if you put your email or home info in there obviously scrub that before posting with placeholders. Should look something like this:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            <hex>
        Signature Algorithm: sha512WithRSAEncryption
        Issuer: C = <Country>, CN = MY-CA
        Validity
            Not Before: Mar 19 11:42:32 2021 GMT
            Not After : Mar 19 11:42:32 2026 GMT
        Subject: C = <Country>, ST = <State>, L = <City>, O = Home, CN = a0d7b954-nodered
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (4096 bit)
                Modulus:
                   <hex>
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Authority Key Identifier:
                keyid:<hex>

            X509v3 Basic Constraints:
                CA:FALSE
            X509v3 Key Usage:
                Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment
            X509v3 Subject Alternative Name:
                DNS:localhost, DNS:127.0.0.1, DNS:::1, DNS:a0d7b954-nodered, DNS:red.home, DNS:homeassistant.local
    Signature Algorithm: sha512WithRSAEncryption
         <hex>

The x509v3 bit is an additional part I added because I have multiple ways to get to the same place depending on whether the caller is inside the docker network, outside or if its calling itself so I list them all on the certificate. You shouldn’t need that though for this, just the CN value.

andarotajo · March 20, 2021, 3:03pm

I’m pretty confused too This is the output of the ca.crt on the client machine, so where the miflora app runs on:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            hex
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = country, ST = state, L = city, O = Home Assistant, OU = CA, CN = server-hassio, emailAddress = email
        Validity
            Not Before: Mar 19 23:09:58 2021 GMT
            Not After : Mar 17 23:09:58 2031 GMT
        Subject: C = country, ST = state, L = city, O = Home Assistant, OU = CA, CN = server-hassio, emailAddress = email
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus: hex
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                hex
            X509v3 Authority Key Identifier:
                hex

            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
         hex

CentralCommand · March 20, 2021, 3:27pm

Oh wait sorry I actually meant the other side. So that looks like the ca.crt you made in the Create an X509 CA key and certificate for self-signing step in the hackaday post. What I wanted to see was the certificate you added to mosquitto. The one you made in the step Generate the MQTT Server self-signed certificate and then put in the certfile field of the add-on.

Also looking at the hackady post a bit more closely I’m realizing it asked you to create two CA certificates. It wanted you to make one in the step Create an X509 CA key and certificate for self-signing and another in the step Generate the CA signed certificate to use in the MQTT Mosquitto Server. The first one you made is the one you need to add to miflora, not the second one. The second one is for client certificates so we’re leaving that one aside for now. The first one needs to be copied to miflora and referenced from tls_ca_cert in the config.ini

andarotajo · March 20, 2021, 3:37pm

Ah you meant that one. About the second part. I think I got that right. The one in the step Create an X509 CA key and certificate for self-signing is my ca.crt that I copied over to miflora and the one from Generate the CA signed certificate to use in the MQTT Mosquitto Server is my serv.crt that I use in the certfile parameter in Mosquitto. After that the guide creates an additional certificate for the client but I ignore that one for now as that was for the mTLS, if I understand correctly.

Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number:
            <hex>
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = <country>, ST = <state>, L = <city>, O = Home Assistant, OU = CA, CN = server-hassio, emailAddress = <email>
        Validity
            Not Before: Mar 19 23:14:31 2021 GMT
            Not After : Mar 17 23:14:31 2031 GMT
        Subject: C = <country>, ST = <state>, L = <city>, O = Home Assistant, OU = MQTT, CN = homeassistant, emailAddress = <email>
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    <hex>
                Exponent: 65537 (0x10001)
    Signature Algorithm: sha256WithRSAEncryption
         <hex>

CentralCommand · March 20, 2021, 4:02pm

Ah ok, I think we’ve found the issue. So first of all, the value that goes in certfile in mosquitto is the certificate created by the step Generate the MQTT Server self-signed certificate. The value for keyfile is the file created right before that in the step Generate the MQTT Server private key. The certificate created by the step Generate the CA signed certificate to use in the MQTT Mosquitto Server is not used right now, that’s used to sign client certificates for mTLS.

Also looking at that certificate I see the subject is this:

Subject: C = <country>, ST = <state>, L = <city>, O = Home Assistant, OU = MQTT, CN = homeassistant, emailAddress = <email>

That CN value in the subject has to be server-hassio, that’s what it is trying to match with the hostname in the URL to verify. So you’ll have to redo that step to make a new certificate for mosquitto. Make sure to adjust the command in the Generate the MQTT Server self-signed certificate to this

openssl req -new -key mosq_serv.key -out mosq_serv.csr -subj "/C=your-country/ST=your-state/L=your-city/O=Home Assistant/OU=MQTT/CN=server-hassio/[email protected]"

(after filling in your blanks with your personal info)

You can re-use the same key as long as you still have the csr file. If you deleted that since its not used once you have the certificate then you’ll have to redo Generate the MQTT Server private key as well and copy both the key and crt files to mosquitto.

andarotajo · March 20, 2021, 4:18pm

Okay I tried it out and Mosquitto says that it can’t load the certificate. The step Generate the MQTT Server private key generates the serv.csr file and that’s what I put into the certfile property.

Edit: I just reverted certfile to use the serv.crt with the changed CN and it actually works now Now that I changed the CN in the server certificate to server-hassio, do I need to change it in the client certificate too, if I want to use the mTLS stuff?

Edit 2: Just tried the mTLS stuff out without generating a new client certificate and it works yaay. So now I’ve changed the Mosquitto config to have

require_certificate: true
cafile: mosquitto/ca.crt

The key and certificate for the client have been added in the miflora app config, so that I have

tls = true
tls_ca_cert = /home/pi/certs/ca.crt
tls_keyfile = /home/pi/certs/client.key
tls_certfile = /home/pi/certs/client.crt

and the port is 8883. I also modified the python file of the miflora app to use a fixed user_id homeassistant which is also the CN in the client certificate. Just a quick write up for the people using this app

@CentralCommand
I think all my problems are resolved now. Thank you so so much for all the help and explanations you’ve given. I understand this stuff a little bit better now which should help me in the future. And it finally works It might be unnecessary since I only use MQTT for this 1 plant sensor at the moment but this definitely opens up the possibility of using MQTT for future devices now that I’ve worked with it.