MQTT Broker with certificates (AWS IoT)

Tags: #<Tag:0x00007f7396fbb9e8>

Hi folks, I’m looking for some help with mqtt integration. I have to use certificates to auth against an external mqtt broker I’ve setup in configuration.yaml. But I’m getting errors when homeassistant tries to connect.

I have my own MQTT broker (an AWS IoT endpoint). The goal is to eventually use the Statesteam configuration to push all entity event changes up to AWS where I can persist and graph for longer. So, I’ve started work, adding the lines below to my configuration.yaml. See below:

mqtt:  
  broker: !secret awsiot-url
  port: 8883
  certificate: /config/certs/AmazonRootCA1.pem
  client_key: /config/certs/b5dd525b0f-certificate.pem.crt
  client_cert: /config/certs/b5dd525b0f-private.pem.key
  tls_version: '1.2'
  tls_insecure: false
  protocol: 3.1.1

I’ve jumped onto the console of my homeassistant container and verified the cert paths are ok. I’ve also checked these certificates and broker address from a local MQTT client. The local client connects and can publish and subscribe to topics. But with the config above in place, HomeAssistant Config check reported ok, I reboot HomeAssistant, and in my logs I see the below (and I cannot call the mqtt/publish service):

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 216, in async_setup hass, self
File "/usr/src/homeassistant/homeassistant/components/mqtt/__init__.py", line 649, in async_setup_entry tls_version=tls_version,
File "/usr/src/homeassistant/homeassistant/components/mqtt/__init__.py", line 799, in __init__ tls_version=tls_version,
File "/usr/local/lib/python3.7/site-packages/paho/mqtt/client.py", line 819, in tls_set context.load_cert_chain(certfile, keyfile) 
ssl.SSLError: [SSL] PEM lib (_ssl.c:3880)

I’ve been working away at this for the last couple of days, and not been able to get this to work. I have a suspicion that maybe my problem relates to the chain of the root cert, rather than anything else. Any ideas?

Have you been able to get this working? I’m trying to connect to my AWS IoT instance from Home Assistant.

However unlike you I’m not able to verify the file attributes as I can’t find the /config directory on the console. I have installed Home Assistant via the official .vmdk file onto VirtualBox running on a Fedora machine.

I’m gaining access to configuration.yaml via the VSC add-on in the home assistant UI and have created a directory within config to hold the 3 key/cert files.

Questions/Points:

  • I still don’t really understand the install via .vmdk (other than following the guide). Is this a docker installation? If so how do I find the config directory from the command line (I know how to get access to the host system)?

  • I get an error ERROR (MainThread) [homeassistant.components.mqtt] Failed to connect to MQTT server due to exception: [Errno -3] Try again but can’t find any info on this. First I’d like to ensure that file access is ok

  • I can successfully connect via MQTTBox to AWS IoT so the issue is on Home Assistant

  • I have removed tls_version: '1.2' as it’s being removed

mqtt:
  certificate: AWS_IoT/xxxxxxx.pem
  client_key: AWS_IoT/xxxxxxx-private.pem.key
  client_cert: AWS_IoT/xxxxx-certificate.pem.crt
  broker: xxxxxxxxx-1.amazonaws.com
  port: 8883
  tls_insecure: false

So I’m stuck on next steps to establish the connection to AWS IoT

I’m afraid I’m still struggling with this. I’m the same in that I can connect to AWS directly from my MQTT client, but not from HA. I’m no further and now not clear on where I need to configure this MQTT bridge information. Should I configure it in:

  1. “configuration.yaml” - as a new mqtt key (as attempted above).

  2. Use the MQTT add-in from the supervisor, and add an additional configuration for AWS-IOT into …/hassio/share/mosquitto which will be loaded alongside the core MQTT configuration.

Can anyone help us work past this?

I get the feeling most home automation setups are local only for security and availability reasons and hence there may be limited focus/experience. Having said that the official documentation supports public brokers (https://www.home-assistant.io/docs/mqtt/broker) so getting AWS IoT to work should be feasible.

There’s also this post on a Home Assistant user utilising AWS IoT to monitor plants (https://medium.com/@crhuber/monitoring-plants-with-aws-iot-and-home-assistant-e5aaae42a582) and I also found an article on the AWS IoT forum (https://aws.amazon.com/blogs/iot/how-to-bridge-mosquitto-mqtt-broker-to-aws-iot/).

Upon comparing these two it looks like the plant setup is using the concept of an “MQTT bridge” whereas I have tried to go down the path of only having the AWS IoT’s broker without a local mosquitto setup.

I might try this bridge setup and see how far I get. Reason I’m doing is to get the best of both worlds: Local monitoring and control via Home Assistant while having the ability to securely monitor and manage the home remotely via the AWS IoT Core via MQTT.

On your specific setup you mention “I’ve jumped onto the console of my homeassistant container and verified the cert paths are ok”. How did you install Home Assistant? I struggle to find my config folder via the console so I can’t check the certs can actually accessed as I used Windows (via Samba) and VSC to edit configuration.yaml

When I access the host system from the VM running on VirtualBox the default directory is/root and all there is is a .docker and .ssh directory

I have to take a couple of steps back as my head is spinning.

What I would like to do is to setup an MQTT bridge to AWS IoT to mimic the setup https://medium.com/@crhuber/monitoring-plants-with-aws-iot-and-home-assistant-e5aaae42a582. However I have a different starting point hence the confusion.

I have setup Home Assistant as a VM on VirtualBox (installed the official .vmdk file) running on a Fedora 25 system. HA comes up fine and I have a GUI for configuration. Firstly is this known as a docker install or does docker not come into play at all for this installation method?

Why the question? To configure this bridge, I have enabled the official Mosquitto add-on and when looking at the log there’s a mention that

1600258818: Config loaded from /etc/mosquitto.conf.

but when I go to the console (from within the VM; i.e. ‘root’ followed by ‘login’) I can’t find /etc/mosquitto.conf. When going to the host system outside the VM again I can’t find /etc/mosquitto.conf. So where is mosquitto on my system?

As part of the MQTT bridge setup I need to install AWS CLI and I need to first understand the overall setup before I can attempt to make any installations.

Anyone able to shed some light into this?

Ok, received some info that I should do a Linux (or Docker) install for AWS-CLI. Have done the Linux install - I believe successfully.

As I couldn’t find /etc/mosquitto.conf (as indicated in my earlier post) I have uninstalled Mosquitto from the Home Assistant UI and done a Linux install on the command line sudo dnf mosquitto. This has created /etc/mosquitto including the mosquitto.conf. So I still have no idea why mosquitto installed via Home Assistant doesn’t create this - still very interested.

However another challenge now is when I edit configuration.yaml to add the mqtt section; i.e.

mqtt:
  certificate: /etc/mosquitto/certs/rootCA.pem
  client_key: /etc/mosquitto/certs/private.key
  client_cert: /etc/mosquitto/certs/cert.crt
  broker: xxxxxxxxxxxxxxxx.iot.us-east-1.amazonaws.com
  port: 8883
  tls_insecure: false

and check the validity of configuration.yaml in the HA Server Controls I get the following error

Invalid config for [mqtt]: not a file for dictionary value @ data['mqtt']['client_cert']. Got '/etc/mosquitto/certs/cert.crt'
not a file for dictionary value @ data['mqtt']['client_key']. Got '/etc/mosquitto/certs/private.key'
not a valid value for dictionary value @ data['mqtt']['certificate']. Got '/etc/mosquitto/certs/rootCA.pem'. (See /config/configuration.yaml, line 14). 

I should have access to these files as per the following

[[email protected] mosquitto]# ls /etc/mosquitto/certs -al
total 24
drwxr-xr-x 2 root root 4096 Sep 18 09:29 .
drwxr-xr-x 4 root root 4096 Sep 18 09:53 ..
-rw-r--r-- 1 root root 1220 Sep 18 09:24 cert.crt
-rw-r--r-- 1 root root 1675 Sep 18 09:24 private.key
-rw------- 1 root root  451 Sep 18 09:24 public.key
-rw-r--r-- 1 root root 1188 Apr 15 16:24 rootCA.pem
[[email protected] mosquitto]#

To further complicate things I have installed the official Terminal & SSH add-on. When I start that and check for the certs via ls /etc/mosquitto/certs I get a No such file or directory error. In fact the whole /etc folder looks very different to when I login to the system from outside Home Assistant.

So why is this not working? I do apologize if I’m missing something totally obvious but I just figure it out :slight_smile: