TLS with Owntracks and Mosquitto not working

Dear users, (cc. @pvizeli),

This post is about a problem I have with setting up Hassio with Mosquitto MQTT and Owntracks. I’m running Hassio 49.1 on a RPi3. My Pi is connected to a router and that router is connected to a modem. I’m using Let’s Encrypt together with duckdns.

Setting up Mosquitto looks quite simple from the documentation available for Hassio, but I think, compared to the documentation in at the “components” page, there are many versions about doing things (I’m offering my help to make tutorials/explanations for the community).

I’ve tried to setup Mosquitto with Owntracks without TLS and that worked. I forwarded the port 1883 from my modem to the router, and from the router to my Pi. I could see in the logs receiving the location from my Owntracks instance on my Android phone. HA tracked my device :slight_smile:.

The configuration I used for this is as follows:
Configuration of the Mosquitto MQTT broker Add-On.

 {
  "plain": true,
  "ssl": false,
  "anonymous": false,
  "logins": [
    {
      "username": "xxx",
      "password": "yyy"
    }
  ],
  "customize": {
    "active": false,
    "folder": "mosquitto"
  },
  "certfile": "fullchain.pem",
  "keyfile": "privkey.pem"
}

The settings of my Owntracks android instance:

Mode: Private MQTT

Host:
host: myduckdns.duckdns.org
port: 1883
Use Websockets: NO

Identification:
Username: xxx
Password: yyy

Security:
TLS: No

Parameters:
No adjustments made.

In my config.yaml file, I did add the following things (Be aware of the differend broker IP than is used in the Hassio Mosquitto add-on website, somebody from Discord helped me figuring out that IP):

mqtt:
    broker: 127.0.0.1 
    username: xxx
    password: yyy

device_tracker:
  - platform: owntracks

For debugging purposes I also have enabled this in my yaml file:

logger:
  default: info
  logs:
    homeassistant.components.mqtt: debug

I also read here that I had to adjust the mqttProtocolLevel to 4, so I did.

When I look in the logs of HA, everythings looks fine and I receive messages, my Hassio sees my phone and makes a known_devices.yaml file. The logs of the Mosquitto MQTT Add-on looks as follows:

starting version 3.2.2
1501250464: mosquitto version 1.4.12 (build date 2017-06-01 13:03:48+0000) starting
1501250464: Config loaded from /etc/mosquitto.conf.
1501250464: Opening ipv4 listen socket on port 1883.
1501250464: Opening ipv6 listen socket on port 1883.
1501250464: Warning: Mosquitto should not be run as root/administrator.
1501250466: New connection from 172.17.0.1 on port 1883.
1501250466: New client connected from 172.17.0.1 as **asdfasdfasdf** (c1, k60, u'xxx').
1501250489: New connection from **mypublicip** on port 1883.
1501250489: Client xxxherolte disconnected.
1501250489: New client connected from mypublicip as xxxherolte (c0, k3600, u'xxx').

Now, when I switch to TLS some strange things happen. First I make sure I delete the known_devices.yaml file, since it should be created automatically again. I also adjust the port on my router and modem to 8883, as this port is used for TLS.

The configuration of the Mosquitto Add-On looks like this:

{
  "plain": false,
  "ssl": true,
  "anonymous": false,
  "logins": [
    {
      "username": "xxx",
      "password": "yyy"
    }
  ],
  "customize": {
    "active": false,
    "folder": "mosquitto"
  },
  "certfile": "fullchain.pem",
  "keyfile": "privkey.pem"
}

The settings in my Owntacks app are the following:

Mode: Private MQTT

Host:
host: myduckdns.duckdns.org
port: 8883
Use Websockets: NO

Identification:
Username: xxx
Password: yyy

Security:
TLS: Yes

Parameters:
No adjustments made.

The configuration in my config.yaml file looks like this:

mqtt:
    broker: 127.0.0.1
    username: xxx
    password: yyy

device_tracker:
  - platform: owntracks

logger:
  default: info
  logs:
    homeassistant.components.mqtt: debug

My Owntracks app, in the status menu gives the following information:

Connected
Endpoint state

Not available
Endpoint state message

0
Endpoint queue

When I look in the logs of the Mosquitto Add-On I get the following:

starting version 3.2.2
1501251171: mosquitto version 1.4.12 (build date 2017-06-01 13:03:48+0000) starting
1501251171: Config loaded from /etc/mosquitto.conf.
1501251171: Opening ipv4 listen socket on port 8883.
1501251171: Opening ipv6 listen socket on port 8883.
1501251171: Warning: Mosquitto should not be run as root/administrator.
1501251197: New connection from **mypublicip** on port 8883.
1501251198: Client xxxherolte disconnected.
1501251198: New client connected from **mypublicip** as xxxherolte (c0, k3600, u'xxx').

In the logs I don’t see any location update from my Owntracks app. But as you can read, the app says it’s connected. If I push the queue message button in the upper right of the App, so it published my data to the MQTT broker, it does so, but I don’t receive anything.Since HA doesn’t get the messages from Owntracks, obviously, it didn’t make a new known_devices file.

Since the app says it has a connection and also the logfiles from the Add-on says there is a connection, I can safely assume that my portforwarding is working correctly.

Then there are 3 options left:

  1. I have to adjust the Mosquitto MQTT Add-on config file.
  2. I have to adjust the config.yaml file
  3. I have to import certificates to the Owntracks app.

My best guess it has something to do with all 3 of it, but because of the certificates I need to specify. In the Add-on config, there are already specified, certificates, as you can see.

I hope your expertise can help me further!

Yours sincerely,
Maharball

Let’s Encrypt needs to confirm that you own and control each hostname for which you’re seeking a certificate. It supports three ways of doing that:

  1. The HTTP-01 challenge requires that the public Internet be able to connect to http://:80$HOSTNAME, for each $HOSTNAME you request a cert for. The --http-01-port flag will let certbot listen on a different port, in case you have port forwarding, a reverse proxy, or some other such thing going on, but the Let’s Encrypt servers must still be able to connect via port 80.

  2. The TLS-SNI challenge requires that the public Internet be able to connect to port 443 on $HOSTNAME. As above, you can specify that certbot listen on a different port, but the Let’s Encrypt servers will connect to port 443.

  3. The DNS-01 challenge requires that you be able to add TXT DNS records for each requested hostname, ideally through an API or some automated mechanism.

1 Like

Dear @Ciquattro,

I already have a working certificate on my Hassio instance. I can connect via https://myduckdns.duckdns.org, so that is not a problem.

With kind regards,
Maharball

what about the ACL on MQTT… Have you configured it correctly?

What is the ACL? For you information, I’m using Hassio, there is no command line work involved over here, so if you are referring to that, that’s not possible and is probably not the aswer for my question, unfortunately.

I had another look at my Hassio log. This error message came up. It’s not strange that it gives this error, since it didn’t create that file.

2017-07-28 22:16:53 ERROR (MainThread) [homeassistant.components.device_tracker] Unable to load /config/known_devices.yaml: Config file not found: /config/known_devices.yaml

I was just looking in the logs, this is what appeared in the Addon logs of the Mosquitto broker when I just tried again:

Nevermind, I forgot to activate TLS in the Owntracks App.

Never tried getting TLS to work with Owntracks. Now that I am on hass.io and found this thread I thought I would like to give this a go. Things I figured out/did:

  • SSL is working correctly with Duckdns and Let’s Encrypt
  • Owntracks --> MosquittoMQTT is working (without TLS)
  • The Owntracks app (Android) expects a CA certificate and Client certificate to be uploaded/selected.
  • On the Samba “SSL” I see the fullchain.pem and privkey.pem files
  • I read somewhere that the CA cert is probably not necessary as Let’s Encrypt is already trusted (true?)

Please provide me with instructions to set up a secure Owntracks/MosquittoMQTT connection.
Thank you.

Yes, I still have the problem where I receive my location without TLS, and with TLS it seems connected (it says in the app and the addon), but I don’t receive the messages. So if there are experts out here, help please! :slight_smile:

Hi! i’m having a bit of trouble getting MQTT/Owntracks to work with Let’s Encrypt. The way I have it setup is:

In my config-file:

mqtt:
  broker: 127.0.0.1 
  username: !secret mqtt_user
  password: !secret mqtt_pass

device_tracker:
  - platform: asuswrt
    host: 192.168.1.1
    username: !secret asus_router_user
    password: !secret asus_router_pass
    
  - platform: owntracks
    max_gps_accuracy: 200

On my router I have port forward on 443 to 8123 and also 8883 to 8123.

In HassIO MQTT-Options I have

{
  "plain": true,
  "ssl": true,
  "anonymous": true,
  "logins": [
    {
      "username": "XXX",
      "password": "YYY"
    }
  ],
  "customize": {
    "active": false,
    "folder": "mosquitto"
  },
  "certfile": "fullchain.pem",
  "keyfile": "privkey.pem"
}

In the Owntracks I keep getting this error message: “TLS Client Certificate - Incorrect file or passphrase”. The app settings can be seen in the attached pictures:

And finally the log from HassIO-MQTT-Logs:

starting version 3.2.2
1502303887: mosquitto version 1.4.12 (build date 2017-06-01 13:03:48+0000) starting
1502303887: Config loaded from /etc/mosquitto.conf.
1502303887: Opening ipv4 listen socket on port 1883.
1502303887: Opening ipv6 listen socket on port 1883.
1502303887: Opening ipv4 listen socket on port 8883.
1502303887: Opening ipv6 listen socket on port 8883.
1502303887: Warning: Mosquitto should not be run as root/administrator.
1502303900: New connection from 172.17.0.1 on port 1883.
1502303900: New client connected from 172.17.0.1 as 17c26b12-b118-40ba-8efe-0b9c7001e4c7 (c1, k60, u'XXX').

I’m guessing I have not managed to set it up properly somewhere (most likely with the TLS-settings…) but not sure where :slight_smile:, can someone have a look?

Is that correct?

hmm, perhaps not? Suggestions on what it should be? :slight_smile: (I thought MQTT with TLS used 8883…)

Is there somebody out there who can help me?

Would just like to bump this as I am sure there are some experts out there that can quickly point us in the right direction!
Anyone, please?

2 Likes

You need to forward 8883 from your router to your-local-hass-ip:8883

Currently you are forwarding MQTT connections from your phone to the HTTP server listening on port 8123, whereas you actually want to forward MQTT connections to the MQTT broker.

I have MQTT-over-SSL working with the following configuration:

Mosquitto Options

Note that I still allow unencrypted connections on port 1883, which I use for devices on my LAN. SSL is only used for the connection from my phone.

{
  "plain": true,
  "ssl": true,
  "anonymous": false,
  "logins": [
    {
      "username": "HASSIO",
      "password": "..."
    },
    {
      "username": "Home-Devices",
      "password": "..."
    },
    {
      "username": "Sam",
      "password": "..."
    }
  ],
  "customize": {
    "active": false,
    "folder": "mosquitto"
  },
  "certfile": "fullchain.pem",
  "keyfile": "privkey.pem"
}

Let’s Encrypt Options

{
  "challenge": "https",
  "email": "...",
  "domains": [
    "..."
  ],
  "certfile": "fullchain.pem",
  "keyfile": "privkey.pem"
}

Home Assistant Config

mqtt:
  broker: localhost
  port: 1883
  client_id: home-assistant-1
  keepalive: 60
  username: HASSIO
  password: ...

device_tracker:
  - platform: owntracks
    max_gps_accuracy: 200

OwnTracks Android Connection Settings

Mode: Private MQTT
Host:
    Host: my-address.com
    Port: 8883
    Use Websockets: no
Identification:
    Authentication: yes
    ...
Security:
    TLS: yes
    CA certificate: none
    Client certificate: none

Router Port Forwarding

HTTPS:  443  -> hass-local-ip:443  (required for Let's Encrypt cert renewal)
HassIO: 8123 -> hass-local-ip:8123 (provides remote access to HASS instance)
MQTT:   8883 -> hass-local-ip:8883 (forwards secure MQTT connections to mosquitto broker)

This has been working for me so far. When I first set it up I got a lot of the following errors, but after a few minutes OwnTracks was able to connect and has been working correctly since:

Client connection from 172.23.1.1 failed: error:140260FC:SSL routines:ACCEPT_SR_CLNT_HELLO:unknown protocol.

Edit: the SSL errors are actually generated when I try to use mosquitto_sub from my local machine without providing a CA file, and have nothing to do with the OwnTracks connection

4 Likes

I’m very curious, will have a look tomorrow and I hope I did something terribly wrong :wink:

I have hassio 0.53.1 on a RPi3 with Duck DNS, Mosquitto broker and Let’s Encrypt and Zanzito on my android device (Samsung S6 Edge) working nicely via SSL on port 8883. My setup is:

configuration.yaml

mqtt:
  broker: localhost
  username: ....
  password: ....

device_tracker:
  - platform: mqtt_json
    devices:
      my_device: zanzito/samsung_s6/location

binary_sensor:
 - platform: mqtt
   name: "Zanzito status"
   state_topic: "zanzito/samsung_s6/status"
   payload_on: "1"
   payload_off: "0"
   sensor_class: connectivity

sensor:
  - platform: mqtt
    name: "Zanzito Battery"
    state_topic: "zanzito/samsung_s6/battery_level"
    qos: 0
    unit_of_measurement: "%"

  - platform: mqtt
    name: "Zanzito Barometer"
    state_topic: "zanzito/samsung_s6/lps25h_barometer_sensor"
    qos: 0
    unit_of_measurement: "hPa"

Mosquitto Options

{
  "plain": true,
  "ssl": true,
  "anonymous": false,
  "logins": [
    {
      "username": "....",
      "password": "...."
    }
  ],
  "customize": {
    "active": false,
    "folder": "mosquitto"
  },
  "certfile": "fullchain.pem",
  "keyfile": "privkey.pem"
}

Let’s Encrypt Options

{
  "challenge": "https",
  "email": "[email protected]",
  "domains": [
    ".....duckdns.org"
  ],
  "certfile": "fullchain.pem",
  "keyfile": "privkey.pem"
}

Router Port Forwarding

ha_ssl: internal 192.168.1.110:8123 -> external 443
ha_mqtt: internal 192.168.1.110:1883 -> external 1883
ha_secure_mqtt: internal 192.168.1.110:8883 -> external 8443

Zanzito Android Connection Settings

MQTT Connection
Host: ....duckdns.org
Port: 8443
Security:
Username: .... (MQTT username)
Password: .... (MQTT password)
SSL/TLS connection: enabled (no CA certificate imported, I expect Let's Encrypt is trusted)

Connected Sensors
Batter level: checked
LPS25H Barometer Sensor: checked

In the Mosquitto logs I see no errors or what so ever :+1:

New connection from xx.xx.xxx.xx on port 8883.
New client connected from xx.xx.xxx.xx as pahoxxxxxxxxxxxxxxxxx

With some customization it looks like:

2 Likes

I took the approach to use cloudmqtt for owntracks and bridge to it from my local mosquito server.
It works great, with no need for https/tls for incoming mqtt connections.
Owntracks connects to cloudmqtt with HTTPS/TLS and my local sensors connect to my insecure mosquitto, which is not open to internet.

Mosquito add-on config:

{
  "plain": true,
  "ssl": false,
  "anonymous": true,
  "logins": [
    {
      "username": "someuser",
      "password": "somepw"
    }
  ],
  "customize": {
    "active": true,
    "folder": "mosquitto"
  },
  "certfile": "fullchain.pem",
  "keyfile": "privkey.pem"
}

Mosquito bridge .conf file (put in /share/mosquitto):

connection couldmqtt
  address mxx.cloudmqtt.com:port
  topic # in 1
  try_private true
  notifications false
  start_type automatic
  remote_clientid someclientid
  remote_username someottheruser
  remote_password someotherpassword
  keepalive_interval 300
  cleansession true
  bridge_protocol_version mqttv311
  local_clientid someclientid
  bridge_cafile /etc/ssl/certs/ca-certificates.crt
  bridge_insecure false

The above is discussed in another topic.

1 Like