Howto: Create a reverse SSH tunnel, with easy HTTPS at the other end

I use a 4G connection at home that does not give me a public IP address. In order to allow me to access the Hassio UI/API from outside my home I’m using an SSH reverse tunnel.

NOTE: This is WIP. Sometimes it seems to stop working. I’m putting it here in the hopes that people use/fix it. Maybe we can turn it into an addon. Or maybe all this has already been done and better, and I just missed it.

In order for this to work, you need SSH access to a server that is accessible from the internet. The smallest, cheapest linux server you can find will work.

On the server, we run software called Caddy, that will reverse proxy the connection home, and automatically get an SSL/TLS certificate from LetsEncrypt.

First, find/generate an SSH key for Hassio. This is the key that it will use to log into your server.
Instructions are here: https://www.digitalocean.com/community/tutorials/how-to-configure-ssh-key-based-authentication-on-a-linux-server

Save the private key into your config directory, as id_rsa

Next, install the SSH - Secure Shell add-on.

In the SSH server options, add this

"packages": [
    "autossh"
],
"init_commands": [
    "autossh -N -M 18123 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o \"PubkeyAuthentication=yes\" -o \"PasswordAuthentication=no\" -i ~/config/id_rsa -R 8888:localhost:8123 [email protected] &"
]

This will automatically install autossh, and then connect to your server, exposing the port 8123 as 8888 on the server (only accessible from the server itself).

Now, on the server we need to proxy this, and expose it via HTTPS.

Log into your server, and Install Caddy - https://caddyserver.com/

Save a file called Caddyfile somewhere, and put this in it

home.myserver.com

proxy / localhost:8888 {
    websocket
}

And start caddy.

After a minute, you should be able to get to your dashboard at https://home.myserver.com

Have fun!

3 Likes

I’m in the same situation and am trying to use AWS EC2 as my gateway, but I’m a bit out of my technical depth. I’m 95% sure I’ve got Caddy running correctly on AWS (and using a fixed IP and a DNS entry for my personal domain name), but I’m getting this error in my local SSH - Secure Shell log:

FATAL: Configuring a password or authorized keys is mandatory!

In my case, I’ve put my AWS private key (.pem version) in my HASSIO config folder and named it “id_rsa”, but not sure if that’s correct.

Ah, looks like you need to configure the ssh server on hassio too (i.e. the actual point of the addon we’re using)

The instructions are on the addon page (where it says “In order to start this add-on for the first time”).

Yup. Figured that out and got it to start, but I’m now having a challenge with my AWS private key. SSH - Secure Shell add-on starts all the way through, but the log has this part in it:

Host key verification failed.

I can SSH manually into my AWS instance using Putty and authenticating with my private key (in .ppk format), so I know the key is working. I suspect autossh needs a different format private key to use to log on but I haven’t figured out which one. I’m new to this part! Any ideas?

I’m not sure. I generated and transferred the key using the instructions above (ssh-keygen then ssh-copy-id).

I generated a new keypair, following the instructions. I put the private key named id_rsa into my Home Assistant config directory, and amended the public key to my AWS server “authorized_keys” file. I restarted both Caddy on my AWS server and the SSH-Secure Shell plugin, but still get the same Host key verification failed error.

I feel like I’m tantalizingly close. Hopefully someone sees this and has some advice.

Ooooh. host key. Its having trouble because it thinks that the server has changed its key, its not a problem with your key.

Try adding -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null to the autossh command. It is a bit of a security issue (someone could mitm you, pretending to be your server i think) but not really a problem.

Great idea Elliot.

I was brushing up on ssh tunnels and found a recommended setting in my searches that might help stabilize the connection.

-o “ExitOnForwardFailure=yes”

ExitOnForwardFailure
Specifies whether ssh should terminate the connection if it cannot set up all requested dynamic, tunnel, local, and remote port forwardings.

It seems that without this if autossh detects the dropped connection before the remote end notices the next reconnect will be unable to bind to the remote port because it’s still bound. This forces it to drop when it can’t bind and will try again instead of staying in a stuck state.

This is not my area of expertise but just something I ran across and thought it might help.

Rich

Aha! Nice find… thanks. That could be what’s happening. I’ll try it out when i get home, and update the instructions.

OK, I tried taking another swing at this following the directions again. Two items I could use help with. When I go to https://ha.myserver.com it returns:

502 Bad Gateway

ha.myserver.com (not the real address) is an AWS instance running Ubuntu and Caddy, with ports 22, 80, 443 open.

Also, during startup of SSH-Secure Shell I get a log error:

command-line: line 0: Bad configuration option: \342\200\234exitonforwardfailure

My startup options in HA is

{
“log_level”: “info”,
“port”: 22,
“username”: “NOT_SHOWING_THIS”,
“password”: “NOT_SHOWING_THIS”,
“authorized_keys”: [],
“sftp”: false,
“compatibility_mode”: false,
“allow_agent_forwarding”: false,
“allow_tcp_forwarding”: false,
“packages”: [
“autossh”
],
“init_commands”: [
“autossh -N -M 18123 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o “PubkeyAuthentication=yes” -o “PasswordAuthentication=no” -o “ExitOnForwardFailure=yes” -i ~/config/id_rsa -R 8888:localhost:8123 [email protected] &”
]
}

Any ideas?

Cleaned up the init_commands. The log error is gone, but it didn’t make a difference in my ability to log on.

“init_commands”: [
“autossh -N -M 18123 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o PubkeyAuthentication=yes -o PasswordAuthentication=no -o ExitOnForwardFailure=yes -i ~/config/id_rsa -R 8888:localhost:8123 [email protected] &”
]

Progress. If I SSH into my AWS server and use wget localhost:8888, it does appear to pull the homepage from my HA instance. So the HA plug-in and commands appear to be working. The issue seems to be that my “public internet” request to https://ha.myserver.com is getting blocked.

I guess that could be from AWS, but I’ve opened my server instance to port 443. Hmmmm.

Hi @egangreenstein, did you succeed in solving this issue in the meantime. I experience the same ‘502 Bad Gateway’ error.

When I SSH directly into the server with the ‘wget’ command like you did, I do get a html response from hassio although it’s displaying the message ‘Home assistant had trouble connecting to the server’.

I was never able to get this working. One challenge to troubleshooting is that each time you restart Caddy (say, to try a different Caddyfile), it requests a new SSL certificate from Lets Encrypt, and you can only do that 5 times in a week before it throws an error that causes Caddy to exit (https://letsencrypt.org/docs/rate-limits/). You have to wait another week before trying something else.

Any updates on this? I am also using LTE for my home internet and still havent found a way to have external access.

@elliot Just wanted to say thanks!

Couple of months back I came across your post. I tried this approach to expose HA frontend over internet without public IP (My ISP do not provide public IP). I used aws t2 micro EC2 and everything was stable and working fine. But I gave up due to cost, then I realized I can go even lower on aws so now using aws t3 nano (cheapest EC2 at this time) and everything is working great. It’s almost a month now and its super stable. Thanks again for sharing this.

Mind sharing how you set this all up?

Sure, I will try to describe my setup in high-level steps. I don’t want to hijack this thread being focused on Hassio plugin.

Background:

I use hassbian and wanted to access HA frontend over internet using HTTPS. Main issue for me is, my ISP do not provide public IP. Upon some googling, I came across this thread and learned about reverse SSH tunnel. Quickly tried it out and found to be promising. I already own a domain, so all I needed was a small server with public IP where I can connect from my RPi and run proxy. I chose aws and got it working.

High level steps:

A. Setup aws instance

  1. Followed EC2 guide to have an instance up and running (t3 nano) Amazon Linux Image GP2 - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html
  2. Setup an Elastic IP (public IP so that I can reach to above instance) - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html
  3. Point my domain to above Elastic IP using aws Route 53 - https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/getting-started.html

B. Setup Raspberry Pi

  1. Created a service at /etc/systemd/system/autossh-ha-tunnel.service with content:
[Service]
Environment="AUTOSSH_GATETIME=0"
ExecStart=/usr/bin/autossh -N -M 18123 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o "PubkeyAuthentication=yes" -o "PasswordAuthentication=no" -i <path to PEM key> -R 8888:localhost:8123 [email protected]<my domain>
  1. Start and enable service on boot:
sudo systemctl start autossh-ha-tunnel.service
sudo systemctl enable autossh-ha-tunnel.service

C. Run proxy on EC2 instance (Same as mentioned by OP in first post):

  1. Install Caddy curl https://getcaddy.com | bash -s personal
  2. Create Caddyfile:
<my domain>

proxy / localhost:8888 {
    websocket
}
  1. Start proxy: caddy

After a while, I was able to access HA frontend using https://<my domain>. Additionally, I created haaska skill and I can now control HA using alexa.

Hope this helps!

2 Likes

Just wondering. How much does this setup cost you to run with aws?

I paid $27 upfront for the T3 Nano EC2 Reserved Instance for one year. Additionally, I get less than $1.5 monthly billed for domain hosting and S3 space from aws. So around $45 yearly. That’s about it. I am running this since couple months now, zero down time. I regularly use following services using this setup:

  • Access HA instance remotely over internet
  • Access security camera recordings from file storage via CloudCommander remotely
  • ssh to my raspberry pi from aws instance
  • Alexa for HA
  • Occasionally I use this EC2 instance for experimenting other stuffs - learning purposes