Disclosure: Supervisor security vulnerability

We’ll actually I am not sure how deep is the impact? So are only credentials have to be seen as compromised. Or should I go for a fresh install of home assistant? I mean, if someone is actually having the possibility to manage addons. How deep could someone get into our system? As @PeeDee mention could someone install a trojan via addon into homeassistant and execute it. Then remove the addon so nobody suspects?

Credential rotating… okay kinda logic and should be straight forward. But is this actually enough? A compromised system could leak that data immeadiatly again.

I just want to point out that using nabu casa. For me at least in germany in the part of the remote access Nabu casa explicitly is saying “Home Assistant Cloud bietet eine sichere Remote-Verbindung zu deiner Instanz” (freely translated to “Home Assistant Cloud provides a secure remote connection to your instance”). In my opinion this is definetly misleading. There should be no unauthorized routing to the local instance at all, maybe authorizing at least with the nabu casa credentials on the nabu-casa servers.

But at all I still love home assistant :slight_smile: Just a good product, those things happen, everytime, everywhere. We just should mitigate the impact.

FYI: If not said by anyone: To logout all users, changing password is not enough. By deleting all refresh tokens in “/config/.storage/auth” (Can be access with File editor addon, but you have to remove it from file exclusion in configuration) all users will be logged out and forced to login again. But be aware messing this file up could maybe brick the system.

Yeah I reinstalled the update and the error returns.

Did you set up authenticated origin pulls? Blocking non-cloudflare IPs is fine I guess but since cloudflare is itself a PaaS it’s not really a solution. Anyone can host a custom service on cloudflare so traffic coming from cloudflare isn’t necessarily trusted. Also I mean ip address spoofing is a thing.

Hence why they recommend the authenticated origin pulls option. That’s what guarantees traffic coming to your reverse proxy came through your WAF.

2 Likes

IT’s the latest “Core” update the caused it, so this is not the place-holder for this topic

The deepness? You should consider that there is a high chance your HA instance and the device running it compromised and potentially infected. IF any bad actor found out about this before it was fixed it would have been sooooo easy to fully own all public and Nabu Casa HA instances.

If also you didn’t separate your Home Assistant device to for example a guest of IOT network potentially more devices could be owned.

1 Like

sorry to be a simple soul, but, even that terminology seems shop talk. If we are advised to change our password (or even full credentials meaning login user name and password) for HA, or HA and all used add-ons, I feel that should be made explicit in the most simple of terms.

Rotate credentials could imply other measures than changing ones pw, and we need to be 100% sure what is meant by the terminology unambiguously

Added to that, should we do that for all registered users and devices too?

A yes would be a true pain here, as I can not believe a single person here does so on a regular base… that would be over 30 combinations each periodic change.

Not adding to the trouble we would have for allowing registered users to login, remotely or locally, and what that would do to our registered device_trackers (the iOS app is notoriously troublesome at that)

Lastly, if subscribing to Nabu Casa does not make life safer than logging in remotely through our own services (like the mentioned DuckDNS or likewise services), I must confess to have been misunderstanding my long standing subscription to that service completely…

6 Likes

Well but those addresses must have been spidered somehow, maybe disclosed by their owners or at Nabu Casa they’re managing it very badly…
I also manage subdomains for my customers (nothing to do with home assistant) and third level domains are not available anywhere… They’re evaluated at web requests and if they matches the correct tunnel is served.
Do you know about other ways?

Frenck I respect you and I love your work but you’re a little rude sometimes.
Port scanning is what an Hacker will do knowing that a certain service has a vulnerability to find targets manually or automating using a script to find targets with a not updated software.
Or maybe using a dedicated search engine like this: https://www.shodan.io/search?query=Home+Assistant
This will list you all instances that has “Home Assistant” in the http reply. And there you can find all people that are using port forwarding as they expose that port to port scanning (that is what shodan is doing to index those data)
That’s different for Nabu Casa users that has no open ports to target in a port scan.
You have to know that exact subdomain to be coupled to the private tunnel of that home assistant instance…
And to prevent brute force trying it’s enough for Nabu Casa to add an increasing lockout time for each ip source that is trying a wrong subdomain…
I’m I so wrong in your opinion? Why?

5 Likes

Yes, I enabled authenticated origin pulls in cloudflare. I set the origin certificates generated by Cloudflare in nginx. However, I did not use WAF, but Cloudflare Zero Trust Access.

Of course it’s not perfect but still better than just nginx with lets encrypt which wouldn’t have helped at all against this vulnerability. Even though i was still vulnerable i made it less intuitive and easy to abuse. No nabu casa subdomain, security check of visitor by Cloudflare and confinement to IPs from certain countries only. People outside those countries wouldn’t even know a HA-Instant exist behind my Domain. Both my HA-Instant and nginx are running in separate VMs (with nginx as well as a firewall for the VM set up to only accept Cloudflare IPs). Both are backed up regularly, which would allow me to roll back if I notice a compromise.

Do you think there is still room for further improvement?

1 Like

Am I right in reading this: This gives an attacker access to install Home Assistant updates and manage add-ons and backups.

As “This gives an attacker complete system access”?

What’s the difference?

1 Like

I also have this error. You are not alone.

What I know is that if you google “How to find subdomains” there are numerous results.

My only reason to comment was because you said

I believe the answer it’s not ‘far less likely’.

Time for a reality check guys! This vulnerability has existed for 6 years. If it had been exploited don’t you think you would have had numerous of your linked services hacked already. Hackers aren’t going to sit around on stolen credentials for six years.

13 Likes

That’s interesting and I thank you for the input. I’ll look about how those subdomains are get as this will change the concept of “far less likely”. If subdomains are spidered as disclosed by the users somehow that will not change my mind (I’ll never disclose mi subdomain!!!)
If there is a real list that contains even my own instance then Nabu Casa is doing it wrong and that should be addressed.
My experience however tells me that my subdomains is not available anywhere on the net as otherwise I would have stated at least one access trial with wrong credential. But that never happened so I’m inclined to consider the first hypothesis the most trustable.

@123
@henrik_sozzi
Certificate Authoritys (CA) like Let’s Encrypt have to send their logs to public servers for auditing. (keyword: “Certificate Transparency”)
So just see crt.sh | nabu.casa

Hostnames leak the moment you get SSL for them.
(A solution would be to use wildcart certs)

Those random hostnames are just security theater.

4 Likes

This is a great callout, @gubiq . Thank you.

This is effectively proof that there is a public directory of all Nabu Casa Cloud-proxied HA instances. (I confirmed this by locating one of my own HA instances in the list. An instance I am 100% confident the URL hasn’t been crawled or leaked elsewhere.)

There are many APT groups (Nation-states and NGOs alike) who have been known to exploit zero days, establish backdoors, and then do nothing with them until they decide they “need” them. It’s the “hack everything” approach.

I’d wager that many HA users are in tech fields, and these APT groups may consider us as valuable targets, to serve for potential pivots into more sensitive (e.g. Corporate) networks.
(Of recent note: The recent lastpass breach was via a superadmin’s home Plex server. His personal devices were targeted by the attackers to facilitate an eventual pivot into LastPass.)

I say this next bit as someone who’s also been in cybersecurity for a decade+, with the majority of that time in a Detection & Response specialty. Diving through logs and performing post-compromise forensics is my day-to-day bread-and-butter. I’m also very familiar with responding to “vulncidents”, where no active exploitation has been observed, but the ease of RCE and the potential impact/risk associated is significant enough to warrant treating it like there may be abuse of the vuln in the wild:

Given the public directory @gubiq pointed out and this context that compromising personal servers is an emerging TTP, I think that it is critical we get some more technical details on the vulnerability, soon, from a “where to start doing forensics” perspective.
If it was exploitable via the Nabu Casa “tunnel”, we need to know how it may have been exploited, and how that may have appeared in supervisor logs.
It doesn’t matter if Nabu Casa Cloud tunnels are E2E encrypted if the adversary can be one of the endpoints and exploit the vuln. AIUI, this isn’t a MitM situation, so E2EE does nothing. (As mentioned above, client certificates could have been a layer of defense here, since they would effectively prevent the adversary from “being” one of the endpoints allowed to talk to HA.)

This is the kind of thing where if anyone using HA can find evidence of exploitation on their instance, then we need to start changing how everyone is responding to this. (Not trying to stir up fear or panic here. Just trying to indicate that if we find evidence of even ONE case where this has been exploited in-the-wild, then this shouldn’t just be considered a vulnerability anymore; there’s potentially an attack campaign that’s gone undetected, and that campaign could have potentially just gone through the public directory of HA instances and compromised them all.)

We can’t even start digging into this and gaining confidence that there’s not exploitation ITW until we know where to start looking.

I very much appreciate the Nabu Casa team’s approach on patching this vuln and disclosing it like this. That’s a mark of a team that takes their product seriously and respects their users.
Given the no-auth RCE and open directory of vulnerable targets though, I think the very respectable thing to do is to provide the guidance users need to determine if their instances may have been exploited.

I’m very interested to see how this vuln impacts Nabu Casa strategy and security going forward.

14 Likes

Wow. That is an eye opener. I also had the false impression that this URL and its long random subdomain was secret. I never used it beyond trying for fun. I even remember turning it off but when I looked now it was active again

I normally had a public domain and in the past year I changed to not exposing but instead using my own Wireguard. Both the IOS and Android clients can be set up to use the tunnel or go direct based on target IP so you just setup the 192.168 urls in the HA app and you have only one hole in the firewall, the Wireguard which I trust more than this Python soup of multiple Docker containers that makes up HA.

I am really shocked that the xxx.ui.nabu.casa URLs are not secret. I would have wanted to know that

3 Likes

Oh wow, thank you for pointing out that! In fact, for the similar service I’m managing for my customers, I’m using a wildcard certificate valid for all third level domains. That way you can use the third level domain as a “string” to identify the correct insance and apply some fail2ban strategy when that identification is wrong to depotentiate brute force trials.
That said I’ve tried the link at crt.sh you provided and my specific instance was not there. Then I tried even subdomains.whoisxmlapi.com and my instance wasn’t there too but I’m sure the reason is just that the resultset is not the complete one (10.000 records…), just luck.
I was believing that my third level was secret but it wasn’t so! That’s not good.
So, maybe someone from Nabu Casa can explain why not to use wildcard cert there? I’m sure they evaluated that as that’s even easier to set-up.

Note that when you disable Remote Control, an attacker that manage to get access to your account at Nabucasa can remotely enable remote control again. And you cannot disable that it seems. That security risk needs to get plugged.

See Nabu Casa - disable possibility to turn on "remote UI" remotly - #19 by KennethLavrsen

3 Likes

it does after all state:

In a compromised situation like we currently experience, this should probably also be notified in this panel, and auto-turnoff?

the statement btw is a strong one, and non tech security savvy users like most of us are, should be able to rely on this. nothing less. If this is not the case: ‘HA Cloud provides a secure external connection’ , and this being the foremost reason to subscribe, then I fear the livelihood of NC…

1 Like