Cloudflare ZeroTrust + Cloudflared Add-On + Companion Apps === Better Security

Lewis · February 27, 2024, 6:49am

START TLDR

For many years I had cable internet, and even moving house and city a few times, the cable ISPs all provided unofficial static public IP address for the service.

It was straight forward to assign my FQDN to my IP through DNS. Maybe once every two years it would need an update, which I would do manually.

But that’s not really the best choice, and I switched to Fiber Internet (better prices, much better speeds). And my Fiber ISP seems to change my IP almost daily. The above previous “plan” to keep HA available fell apart quickly.

A second use case arose when I found that in addition to the HA front end, I also wanted to have separate points of entry for NodeRed and VaultWarden.

The best solution for this is the Cloudflared Add-On.

I won’t go into how to set this up in this post; that’s already been written many times, and the official guide is easy to follow.

The major benefit of Cloudflared, aside from hiding the actual IP address of my home internet, is being able to route multiple subdomains to various ports over a single tunnel. This is not possible with simple/traditional DNS.

Great! I’ve got Cloudflared setup with my Cloudflare ZeroTrust and then… it’s all broken! (Unless using a standard browser!)

So I disabled the Cloudflare ZeroTrust Application, and simply let all traffic pass through, with the exception of setting up a country block in the WAF. Along with a single webhook endpoint for Samsung Smartthings to send data to HA.

And I didn’t look much at any of that for many months. But I finally started looking into the monthly update reports Cloudflare had been sending. And it’s just absolutely SHOCKING just how much traffic is trying to scrape and troll nearly any endpoint that may be exposed.

There have been many posts in this forum, and elsewhere, that speak to this exact problem:

1. a desire to have an extra layer of some type of security in addition to the built-in

2. the need to have remote access to the services (in my case: HA and VaultWarden)

3. not to have the Companion Apps break completely, or function poorly, or randomly get locked out by Cloudflare trying to protect me from myself

After many hours and days of reading through far too many threads with really no one coming up with a decent solution, I have merged much of this disjointed information together and have come to a very compatible compromise.

END TLDR

Let’s assume the following pre-requisites:

Home Assistant is already up and running
you already have an FQDN
Cloudflared Add-On is installed and configured and connected to Cloudflare
Cloudflare ZeroTrust Access has a Self-Hosted Web Application configured with at least one IdP, and your HA instance is available.
- easiest IdP is to use the CF TOTP, and restrict this to just your own email address, or to [email protected]
- when loading your naked domain, the Cloudflare authentication page does not show the required email or email domain; someone attempting to gain entry would need to know your whitelist, and have access to that email
Congratulations! You’ve added additional security and authentication to your HA instance.
- But ONLY when using a browser (Safari, Chrome, Firefox)
- The HA Companion App will be totally broken, or partially functional. Or may work for a period of time until it unexpectedly fails partially or completely.
- Same goes for VaultWarden / Bitwarden App

**

Let’s first focus on mTLS.

If you’re using Android for your Companion App, this is fairly straight forward.

Head into your Cloudflare Dashboard and get to SSL/TLS.

Create your Certificate.

Detailed instructions for installing the CF Root CA and your Client Certificate can be found here.

Next → Create mTLS Rule

This should be the first WAF Custom Rule. Straight forward: IF the hostname == sub.yourdomain.tld AND the Client Certificate is verified THEN Skip all remaining rules.

This allows for a bypass of the CF Access IdP login.

Remember to take your Client Certifcate pem and key and convert them to a p12, then copy this to your Android and add it to your trusted certificates.

When launching HA Companion App, you will be prompted for the Client Certificate. Select the one you installed, and that’s it! Your WAF rule will currently only pass traffic for your HA instance which is providing the Client Certificate (which no one but you has).

But this is only good for Android… What about iOS??

iOS Companion App doesn’t allow for Client Certificates.

And that’s where most get stuck, and turn off CF Access.

But there is a better way!

WAF Rules can Skip, Challenge, or Block when a rule matches True/False.

Let’s focus on contructing a complex rule which will SKIP when TRUE.

My template developed to become this:

What three key/value pairs in the WAF conditions, when combined, are likely only going to select me?

Through many trials, with both Skips and Blocks, and combing through the WAF Events, I decided on the following:

This is the first of FOUR conditions.

AS Num is a unique identifier Cloudflare uses to determine the ISP from which the incoming request originated.

Hostname is your sub.domain.tld

User Agent CONTAINS “Home Assistant”. (Although Android HA CompApp appears to only present a consistent user agent, iOS does not do this consistently. At first I had this set to equals, but it started to fail. Hence contains is strong enough.

And this rule is SKIP. So, when the following condition is TRUE: the ISP == myISP AND the hostname == mydomain.tld AND the UA == Home Assistant, the request SKIPS all remaining rules.

Primarily I use two ISPs, my home internet, and my mobile carrier. Go ahead and add an OR rule with another set of AND statements to capture your mobile ISP. If for some reason I would be in a condition that I cannot SKIP the WAF on my iPhone, I also have my Android with the private Client Certificate.

If required, also add a full set of ASNum + Hostname + UA (contains) Bitwarden Mobile.

Finally, add another WAF Rule to SKIP on Webhooks.

Some posts had suggested wildcarding the webhooks to SKIP all webhooks. But that’s just a bit too open door for me!

It’s easy to just provide the URI Full webhook endpoint.

My webhooks are for Smartthings and something else I’ve now forgotten… It’s also an option to add the webhook endpoints of your Companion Apps. I believe that once set, these remain the same (?).

Something fancier would be an automation to monitor the device webhook and update the WAF vai CF API when necessary.

Finally, add another rule to BLOCK.

The simplest key would be Country. And here’s the catch:

Country EQUALS YourCountry

OR

Country DOES NOT EQUAL YourCountry

BLOCK.

This rule would effectively BLOCK ALL TRAFFIC that hasn’t previously passed on the above rules.

I think that this is a very good workaround / compromise for the Companion Apps not dealing with third party Identity Providers.

ribera96 · February 27, 2024, 9:36am

Thanks a lot for the guide, it works fine for me though I haven’t tested the iOS connection.
My question is if Cloudfare checks if the client certificate is mine or does it work with any certificate.

Lewis · February 29, 2024, 6:03am

The Client Certificate checked for is the one you made.

Test it out by setting the rule to Skip or Block, and then refresh your Companion App, and view the CF Events in WAF.

ripsta · March 11, 2024, 9:13am

Thanks for your tips, really appreciate that!

However: Am i assuming right, that you have disabled your applications within Zero Trust? Because those skip rules do not skip the zero trust rules on my site.

Thanks for your quick feedback!

Lewis · March 12, 2024, 5:26pm

Yes, I have not added an Access (formerly Zero Trust) application for the sub.domain.tld I use for connecting my mobile devices.

I use the internal / external URL scheme. When internal, Companion app uses https://domain.tld and my DNS rewrites the IP to the LAN IP of the HA instance.

When external, Companion app uses https://sub.domain.tld which is on Cloudflare.

I have tried different configurations with setting a self-hosted application with Access and ingress rules. But the reality is that I don’t know I would ever use it; the only case would be if I needed to use a computer when not at home to view my HA. But this seldom happens.

Lewis · March 18, 2024, 3:30am

I should provide a follow up:

My WAF rule 5 (free tier of Cloudflare allows 5 WAF rules) is a BLOCK rule.

The first 4 are all skip rules. With the and /or operators, the rules can get pretty complex.

Anyway, it’s my Home Assistant and no one but me needs access to it (and family members on Companion App).

As I have been monitoring the events logs, I could still see a lot of traffic getting through.

Probing for wordpress login endpoints, attempting to load favicons, and from all over the world.

Anyway, the best way I found to truly achieve LOCK DOWN mode is to set rule 5 as a Source IP with a BLOCK and set the IP to 0.0.0.0/0

Yes. That means block the entire internet.

Some more stuff that I wanted to work got blocked. But i just went through one by one examining the event logs and then devising a WAF rule to SKIP the blocked service.

evilpig · May 19, 2024, 9:47pm

I can’t install the certificate on my android phone. I have copied the two keys to files and get errors. I have converted it using commands like this but it won’t install. Can you elaborate on how you have done this?

openssl pkcs12 -export -in test.crt -inkey test.key -out test-combined.p12

Lewis · May 22, 2024, 4:19am

I am using a Mac, and Homebrew in the Terminal.

I reviewed my post, and followed my links; then realised I left out this important detail! And honestly, I can’t remember what I did!

However, I’ve reviewed my shell history, and found the few lines of commands that can shed some light…

brew install cfssl
brew update
cd ~/.certs/cert-auth/root
ls -al
cfssl genkey -initca ca-csr.json | cfssljson -bare ca
openssl pkcs12 -export -out key.p12 -inkey halkey.pem -in halcert.pem
cd ~/.cert
ls
openssl pkcs12 -export -out key.p12 -inkey halkey.pem -in halcert.pem
cd ..
cd..
cd ..
openssl pkcs12 -export -out key.p12 -inkey halkey.pem -in halcert.pem

Apparently there was a need for a package from CF for SSL. The repeated commands indicate that I was in the wrong directory, or in a restricted directory.

Your command looks like it’s correct. But have you also got the Cloudflare SSL ?

Gailahan · May 27, 2024, 4:13pm

Don’t know if I did something wrong, but I couldn’t get this to work. I never get prompted for a certificate. I noticed on the Cloudflare site under Mutual TLS it says, “Only available on Enterprise plans.”

Lewis · May 27, 2024, 6:23pm

Though the name is similar, that article is referencing a different Cloudflare service or product.

To generate the mTLS certificate for use in a WAF, login to your dash.cloudflare.com

Choose your domain name.

Then select Client Certificates under the SSL/TLS menu.

From there the client certs can be generated.

tispokes · May 28, 2024, 11:47am

EDIT: see my next post.

There is one step missing. I wasn’t able to recreate your work. Then I found this on the Certificate site:

You have to add your host here!

Edit: nah changes nothing for me. Or maybe I have to recreate the cert?
But maybe it’s the “origin configuration”?

tispokes · May 29, 2024, 7:09am

I did it all new and it works fine, now. The transition from the initializiation via cloudfared plugin inside HA to remote controled failed. Have seen this in the logs.

More details:
According to How tos · brenner-tobias/addon-cloudflared Wiki · GitHub at point 14 “If everything went well, you should be connected to your tunnel.” you see this log:

INF Updated to new configuration config=“{"ingress":[{"hostname":"xxx.yyy.zz", "service":"http://homeassistant:8123"}, {"service":"http_status:404"}], "warp-routing":{"enabled":false}}” version=1

And after I migrated the inside Cloudflared from local to remote controlled I have seen a different “new configuration config”, that made me wonder and I investigated. In my update there was no “warp-routing”, but indetad something like “noTLSverify:true”.

Deleted the CNAME, the connection, the cloudflared HA addon, the rules, the certificate and started from zero. Thats it. Just took some time to settle down, got some connection issues, but no “blocking screen” as I experienced before.

Hope this may help someone

woodhorse · May 30, 2024, 1:37am

Just came to say thank you soooo much! This works like a charm.

kzaoaai · July 2, 2024, 5:22pm

Great tutorial. One additional tip to use less WAF rules (you only get 5 for free);
Use the OR to keep different apps like HA and Bitwarden under the same rule.

GeorgeCastanza · July 14, 2024, 10:06pm

@kzaoaai I take it you’re not using Zero Trust?

kzaoaai · July 14, 2024, 10:33pm

I do use Zero Trust in combination with the WAF rules. WAF rules is first line of defense for me.

For homeassistant, I use the WAF rules to bypass as shown in this tutorial, and then use “Bypass” rules in Zero Trust Access rules to bypass for certain access groups. For example, I have an access group which I can simply add a new IP to using CF API from iOS shortcuts. So if I’m on the road, and my random cellular IP is blocked by ZT (since this new random IP is not in the “Bypass” list), I run the shortcut and that IP gets added to the access group list. I instantaneously obtain access to my HA app without worrying about ZT login screens.

And the iOS shortcut is basically SSH’ing to my home server, and running a script while passing my current IP as a variable.

Not very simple to implement, but once implemented it doesn’t require much user input. I have tried many ways including wireguard, tailscale, ZT WARP, but I like this one since I don’t need to have a separate VPN on for it to work.

AxeBro · July 29, 2024, 1:15pm

This is a bit over my head - just wondering, if the certificate is essentially like certificate authentication? meaning if the client doesn’t have a certificate, they don’t even see the login page or beyond CF’s infrastructure??

jurbanek · August 5, 2024, 1:09am

Really liking this idea for my own iOS Companion App + Cloudflare Tunnel situation when the iOS companion app doesn’t work correctly with Cloudflare Access in place.

How are you “SSH’ing” into your desktop from the iOS shortcut? Are you using Cloudflare Tunnel SSH support? I haven’t been able to get Cloudflare Tunnel SSH working yet but it’s on my list. Are you leaving that that SSH’ability “wide open” or do you have some kind of Access rules protecting it?

kzaoaai · August 5, 2024, 4:10pm

Using tailscale only for that part. You might ask, why not just use tailscale to access HASS and get it over with? I find keeping VPNs on my phone and even on-demand settings to mess with internet connectivity on the phone especially if cell reception drops. Here is the simplified sequence of the iOS shortcut:
1- Get current IP
2- Connect tailscale
3- Run script over SSH. I have a script on the debian VM and I just call it with the IP as a variable. e.g. “./cf_access.sh 172.125.25.25”
4- Disconnect tailscale

The script in step 4 uses CF api to modify an access group and replace the used IP with the one I provide.

jurbanek · August 11, 2024, 10:19pm

Thank you for the update and details.

I am implementing something similar but instead of using Tailscale I am using Cloudflare’s WARP client for “VPN-like” connectivity (don’t call it a VPN in front of Cloudflare SE’s however).

Figured if I am already using Cloudflare Tunnels + Cloudflare Access, why not layer on the WARP client as well. Its use is still free as part of the free ZeroTrust account.

Took about 45 minutes of reading and playing around to get it working with pre-existing Cloudflare Tunnels and an IdP already in place.

Followed this guide here
Concepts · Replace your VPN · Learning paths (cloudflare.com)

Frankly, with how easy the WARP client is to use (no username/password required after a device token is issued) I might end up not even using the “scripting against the Cloudflare Access API to add my public IP when remote” paradigm and just enable the WARP client on mobile when needed accessing Home Assistant via WARP client.