Warning: Allowing remote access is dangerous. Here I am documenting what I have done but I am not saying this is totally ideal or secure and it is complicated. If you aren’t prepared to check logs and review settings every so often then go look at Nabu Casa for remote access. It really does sort it all out for you and funds HA development too. Also if you don’t want Alexa integration don’t do this either, just use a VPN like Tailscale or Wireguard.
TL;DR
I have implemented a Cloudflare tunnel mandating client certificates with a workaround for iOS devices
My Goal
I want to have secure access to my HA instance when I am not on my home network and I want to be able to control HA via Alexa too.
The Problem
This all sounds quite simple but in reality web security is hard. Nothing is totally secure and it all a question of risk and mitigating risk. Over the years I have used a few different methods to achieve some or all of this. I started out using Wireguard , this was great, local, secure and everything worked well. I was able to access my HA instance remotely but when I started to integrate Alexa I found out that it needs to access a HTTPS device that is accessible on the internet on port 443.
Solution #1 - Local Reverse Proxy
Summary
My first working solution to this was to use an Nginx reverse proxy (using Nginx Proxy Manager in Docker) in front of HA and expose that to the internet with port forwarding. I already had a domain name and Dynamic DNS setup with Cloudflare for this to work. This is OK and using a Lets Encrypt certificate I could setup remote access and Alexa (Amazon Alexa - Home Assistant)
This works and in all honesty is probably OK for most people but you only need to trawl through the server logs to see just how many attempts there are to probe and attempt to exploit the connection.
Solution #2 - Cloudflare Proxy
Summary
This wasn’t a great leap forward but I implemented a Cloudflare proxy which meant that most access via the domain name now went through a Cloudflare server and I was easily able to block all those connections looking for Wordpress plugins, PHP info or from places like Russia and China. This cuts out some of the noise but you can still see lots of dubious requests and the port it still open on your router so you still get things directly probing your IP (albeit much less than #1)
The main issue here is that Cloudflare treats Alexa as a bot and goes out of it’s way to block it accessing your server. It also means that now you have two sets of logs to trawl through and you haven’t actually increased the security just filtered out some of the noise.
Attempt #3 - Alexa via VPN
Summary
I came across Haaska and Tailscale: Alexa Smart Home without exposing to the internet which looked quite interesting and got me thinking that I could go back to using a VPN if I could get Alexa to a VPN too.
I ended up tweaking this to work more like the documented setup as above and configured a Lambda to run in a container that used TailScale to connect and then ran the Python code as before.
I came up with GitHub - jamesonuk/haalts: Home Assistant, Alexa and Tailscale which I was testing out but hadn’t got round to figuring out the authentication requirements for Alexa when I gave up. The issue being that the Lambda container gets killed after a period of inactivity and then when you ask Alexa to do anything the container has to be fired up and you end up waiting around 10 seconds for it to respond After that it was responsive until it got aged off again and this made this solution unusable.
Solution #4 - Cloudflare Tunnel and Client Certificates
My goal here was to remove the port forwarding on my router so nothing was being exposed directly. Cloudflare tunnels do just that but without any additional config you are still essentially exposing your end of the tunnel to the internet. So I looked at the authentication options and whist these are all great for remote access they aren’t great for Alexa (and I am always dubious about say using Google as I never know what Google will do with that information).
Someone than posted on the HA Facebook group about mTLS (Client Certificates) which I have used before (and already use for some internal services too). This is pretty secure and you can setup a WAF rule on the tunnel to only allow requests that submit a valid certificate through. I won’t go through setting up a tunnel as there are lots of resources out there which explain it better than I can (and as I have found the Cloudflare dashboard changes so often it will be out of date by next week…) Just register your hostname with Cloudflare (using them as the registrar I believe is actually cheaper than anywhere else as they don’t add any fees and just pass on the cost) and then go into Zero Trust → Networks → Tunnels and follow the instructions to setup a CloudflareD tunnel. You then need to install CloudflareD on your host, I did this as a Docker container as I run HA via Docker but I believe there is a HA addon to do the same). If this is all configured you should be able to access your HA instance remotely via the tunnel (but so can anyone else depending on your authentication options).
So now on to the additional security. In you Cloudflare dashboard go into your domain and go into SSL/TLS → Client Certificates and create a certificate, store the key and certificate somewhere safe as these will be the keys to your tunnel.
Now if you go to the WAF (there is a link to create a rule from a template when you create the client certificate or navigate to Security → WAF. I entered a rule of (not cf.tls_client_auth.cert_verified) with an action of block which means anything without a certificate gets blocked.
Now when you try to visit your HA instance over the tunnel you should get a Cloudflare error saying your access has been blocked. To sort out access on windows and Android devices we need to create a keystore form the key and certificate we were provided with, for this we need a machine with openssl installed (most *nix machines and I am not sure if HAOS can do this) but we run this referencing the two files cloudflare provided.
openssl pkcs12 -inkey key.pem -in cert.pem -export -out cf.pfx
you shoudl give this a decent password and then copy it across to your android device.
if you then search in the settings to install a certificate ytou should see a VPN & app user certificate option, if you click that and point it at the pfx file you copied to the phone it will prompt you for the password and then install it.
For Windows you can search for manage user certificates from where you should be able to expand Personal → Certificates and right click and select All Tasks → Import
At this point you should then be able to connect to your HA instance over the tunnel in a browser or via the companion app. It will prompt your to confirm to use the certificate but then it should just work 
Just two small issues. First off Alexa doesn’t work and secondly iOS doesn’t properly support client certificates.
Alexa
This is a bit of a hack and realistically the key and certificate should be stored in AWS secrets but that has a cost associated with it. So for now I have simply added the key and certificate pem files in the same location as the slightly amended Lambda code (I moved the code around a bit following Amazon’s lambda best practice but largely just included the key / certificate in the request
This sorts out the actual Alexa connectivity but still leaves account linking. To get around this I followed the HAASKA instructions to use Login with Amazon Setting up haaska · mike-grant/haaska Wiki · GitHub
(you can ignore the bits about setting up the Lambda, it is just the skill and account linking bits you need to follow).
Now this just leaves one issue in that previously we were using HA tokens from the account linking in the Lambda, This isn’t ideal but given the Lambda is private and should only be triggered by a a private skill I figured this is probably lower risk than having things exposed. I therefore created a long lived token in HA and used this in the Lambda.
iOS
Now iOS is a pain. iOS itself doesn’t support client certificates with websockets and HA developers aren’t gong to add client certificate support to the iOS companion app. 
I therefore added a WAF rule of
(http.user_agent contains "(io.robbie.HomeAssistant; build:" and http.user_agent contains "iOS" and ip.geoip.asnum in {123,456})
and set it to the first rule, got it skip all remaining rules and set the rule to Skip.
I got the ASN numbers (IP ramges for providers) by looking through the WAF event logs when things were blocked and identified one for home and one for mobile network (means iOS devices can not connect from different wifi etc.). This is far from ideal but the tunnel provides TLS and combined with good passwords and 2FA I am happy to accept this risk (it is still better than #1 where I was just forwaring the port direct to HA).
Just dumping my notes to (a) remind me what I did and (b) in case anyone else is looking to do something similar. Sorry for the lack of details but I reallty should have taken better notes as I went though 