Howto: Create a reverse SSH tunnel, with easy HTTPS at the other end

elliot · January 13, 2018, 2:29am

I use a 4G connection at home that does not give me a public IP address. In order to allow me to access the Hassio UI/API from outside my home I’m using an SSH reverse tunnel.

NOTE: This is WIP. Sometimes it seems to stop working. I’m putting it here in the hopes that people use/fix it. Maybe we can turn it into an addon. Or maybe all this has already been done and better, and I just missed it.

In order for this to work, you need SSH access to a server that is accessible from the internet. The smallest, cheapest linux server you can find will work.

On the server, we run software called Caddy, that will reverse proxy the connection home, and automatically get an SSL/TLS certificate from LetsEncrypt.

First, find/generate an SSH key for Hassio. This is the key that it will use to log into your server.
Instructions are here: How To Configure SSH Key-Based Authentication on a Linux Server | DigitalOcean

Save the private key into your config directory, as id_rsa

Next, install the SSH - Secure Shell add-on.

In the SSH server options, add this

"packages": [
    "autossh"
],
"init_commands": [
    "autossh -N -M 18123 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o \"PubkeyAuthentication=yes\" -o \"PasswordAuthentication=no\" -i ~/config/id_rsa -R 8888:localhost:8123 [email protected] &"
]

This will automatically install autossh, and then connect to your server, exposing the port 8123 as 8888 on the server (only accessible from the server itself).

Now, on the server we need to proxy this, and expose it via HTTPS.

Log into your server, and Install Caddy - https://caddyserver.com/

Save a file called Caddyfile somewhere, and put this in it

home.myserver.com

proxy / localhost:8888 {
    websocket
}

And start caddy.

After a minute, you should be able to get to your dashboard at https://home.myserver.com

Have fun!

egangreenstein · January 14, 2018, 5:19pm

I’m in the same situation and am trying to use AWS EC2 as my gateway, but I’m a bit out of my technical depth. I’m 95% sure I’ve got Caddy running correctly on AWS (and using a fixed IP and a DNS entry for my personal domain name), but I’m getting this error in my local SSH - Secure Shell log:

FATAL: Configuring a password or authorized keys is mandatory!

In my case, I’ve put my AWS private key (.pem version) in my HASSIO config folder and named it “id_rsa”, but not sure if that’s correct.

elliot · January 14, 2018, 10:12pm

Ah, looks like you need to configure the ssh server on hassio too (i.e. the actual point of the addon we’re using)

The instructions are on the addon page (where it says “In order to start this add-on for the first time”).

egangreenstein · January 14, 2018, 10:23pm

Yup. Figured that out and got it to start, but I’m now having a challenge with my AWS private key. SSH - Secure Shell add-on starts all the way through, but the log has this part in it:

Host key verification failed.

I can SSH manually into my AWS instance using Putty and authenticating with my private key (in .ppk format), so I know the key is working. I suspect autossh needs a different format private key to use to log on but I haven’t figured out which one. I’m new to this part! Any ideas?

elliot · January 14, 2018, 10:26pm

I’m not sure. I generated and transferred the key using the instructions above (ssh-keygen then ssh-copy-id).

egangreenstein · January 14, 2018, 10:59pm

I generated a new keypair, following the instructions. I put the private key named id_rsa into my Home Assistant config directory, and amended the public key to my AWS server “authorized_keys” file. I restarted both Caddy on my AWS server and the SSH-Secure Shell plugin, but still get the same Host key verification failed error.

I feel like I’m tantalizingly close. Hopefully someone sees this and has some advice.

elliot · January 14, 2018, 11:05pm

Ooooh. host key. Its having trouble because it thinks that the server has changed its key, its not a problem with your key.

Try adding -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null to the autossh command. It is a bit of a security issue (someone could mitm you, pretending to be your server i think) but not really a problem.

rich-d-thompson · January 15, 2018, 9:23pm

Great idea Elliot.

I was brushing up on ssh tunnels and found a recommended setting in my searches that might help stabilize the connection.

-o “ExitOnForwardFailure=yes”

ExitOnForwardFailure
Specifies whether ssh should terminate the connection if it cannot set up all requested dynamic, tunnel, local, and remote port forwardings.

It seems that without this if autossh detects the dropped connection before the remote end notices the next reconnect will be unable to bind to the remote port because it’s still bound. This forces it to drop when it can’t bind and will try again instead of staying in a stuck state.

This is not my area of expertise but just something I ran across and thought it might help.

Rich

elliot · January 16, 2018, 4:09am

Aha! Nice find… thanks. That could be what’s happening. I’ll try it out when i get home, and update the instructions.

egangreenstein · January 27, 2018, 7:27pm

OK, I tried taking another swing at this following the directions again. Two items I could use help with. When I go to https://ha.myserver.com it returns:

502 Bad Gateway

ha.myserver.com (not the real address) is an AWS instance running Ubuntu and Caddy, with ports 22, 80, 443 open.

Also, during startup of SSH-Secure Shell I get a log error:

command-line: line 0: Bad configuration option: \342\200\234exitonforwardfailure

My startup options in HA is

{
“log_level”: “info”,
“port”: 22,
“username”: “NOT_SHOWING_THIS”,
“password”: “NOT_SHOWING_THIS”,
“authorized_keys”: ,
“sftp”: false,
“compatibility_mode”: false,
“allow_agent_forwarding”: false,
“allow_tcp_forwarding”: false,
“packages”: [
“autossh”
],
“init_commands”: [
“autossh -N -M 18123 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o "PubkeyAuthentication=yes" -o "PasswordAuthentication=no" -o “ExitOnForwardFailure=yes” -i ~/config/id_rsa -R 8888:localhost:8123 [email protected] &”
]
}

Any ideas?

egangreenstein · January 27, 2018, 8:16pm

Cleaned up the init_commands. The log error is gone, but it didn’t make a difference in my ability to log on.

“init_commands”: [
“autossh -N -M 18123 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o PubkeyAuthentication=yes -o PasswordAuthentication=no -o ExitOnForwardFailure=yes -i ~/config/id_rsa -R 8888:localhost:8123 [email protected] &”
]

egangreenstein · January 27, 2018, 8:34pm

Progress. If I SSH into my AWS server and use wget localhost:8888, it does appear to pull the homepage from my HA instance. So the HA plug-in and commands appear to be working. The issue seems to be that my “public internet” request to https://ha.myserver.com is getting blocked.

I guess that could be from AWS, but I’ve opened my server instance to port 443. Hmmmm.

victorvdb · February 23, 2018, 2:19pm

Hi @egangreenstein, did you succeed in solving this issue in the meantime. I experience the same ‘502 Bad Gateway’ error.

When I SSH directly into the server with the ‘wget’ command like you did, I do get a html response from hassio although it’s displaying the message ‘Home assistant had trouble connecting to the server’.

egangreenstein · March 17, 2018, 8:34pm

I was never able to get this working. One challenge to troubleshooting is that each time you restart Caddy (say, to try a different Caddyfile), it requests a new SSL certificate from Lets Encrypt, and you can only do that 5 times in a week before it throws an error that causes Caddy to exit (https://letsencrypt.org/docs/rate-limits/). You have to wait another week before trying something else.

dmoses1969 · August 27, 2018, 2:28am

Any updates on this? I am also using LTE for my home internet and still havent found a way to have external access.

krupaldesai · November 3, 2018, 8:10am

@elliot Just wanted to say thanks!

Couple of months back I came across your post. I tried this approach to expose HA frontend over internet without public IP (My ISP do not provide public IP). I used aws t2 micro EC2 and everything was stable and working fine. But I gave up due to cost, then I realized I can go even lower on aws so now using aws t3 nano (cheapest EC2 at this time) and everything is working great. It’s almost a month now and its super stable. Thanks again for sharing this.

dmoses1969 · November 5, 2018, 5:40am

Mind sharing how you set this all up?

krupaldesai · November 12, 2018, 10:18am

Sure, I will try to describe my setup in high-level steps. I don’t want to hijack this thread being focused on Hassio plugin.

Background:

I use hassbian and wanted to access HA frontend over internet using HTTPS. Main issue for me is, my ISP do not provide public IP. Upon some googling, I came across this thread and learned about reverse SSH tunnel. Quickly tried it out and found to be promising. I already own a domain, so all I needed was a small server with public IP where I can connect from my RPi and run proxy. I chose aws and got it working.

High level steps:

A. Setup aws instance

Followed EC2 guide to have an instance up and running (t3 nano) Amazon Linux Image GP2 - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html
Setup an Elastic IP (public IP so that I can reach to above instance) - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html
Point my domain to above Elastic IP using aws Route 53 - https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/getting-started.html

B. Setup Raspberry Pi

Created a service at /etc/systemd/system/autossh-ha-tunnel.service with content:

[Service]
Environment="AUTOSSH_GATETIME=0"
ExecStart=/usr/bin/autossh -N -M 18123 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o "PubkeyAuthentication=yes" -o "PasswordAuthentication=no" -i <path to PEM key> -R 8888:localhost:8123 ec2-user@<my domain>

Start and enable service on boot:

sudo systemctl start autossh-ha-tunnel.service
sudo systemctl enable autossh-ha-tunnel.service

C. Run proxy on EC2 instance (Same as mentioned by OP in first post):

Install Caddy curl https://getcaddy.com | bash -s personal
Create Caddyfile:

<my domain>

proxy / localhost:8888 {
    websocket
}

Start proxy: caddy

After a while, I was able to access HA frontend using https://<my domain>. Additionally, I created haaska skill and I can now control HA using alexa.

Hope this helps!

aagaeta · February 3, 2019, 3:19pm

Just wondering. How much does this setup cost you to run with aws?

krupaldesai · February 4, 2019, 12:03pm

I paid $27 upfront for the T3 Nano EC2 Reserved Instance for one year. Additionally, I get less than $1.5 monthly billed for domain hosting and S3 space from aws. So around $45 yearly. That’s about it. I am running this since couple months now, zero down time. I regularly use following services using this setup:

Access HA instance remotely over internet
Access security camera recordings from file storage via CloudCommander remotely
ssh to my raspberry pi from aws instance
Alexa for HA
Occasionally I use this EC2 instance for experimenting other stuffs - learning purposes