Unhealthy state

Configuration
61

That general idea has been discussed and knocked on the head by the Devs, I can’t remember the exact words used, but something along the lines of unfounded and ridiculous. I’m personally not convinced that something like this won’t happen though.

The forewarning was given, 6 months ago, that is the point. Notice was given that at some point you will face an issue as the project grows and matures. You’ve had ample time to evaluate the best method to move forward to ensure you don’t end up in the exact position your are in now, but you chose not to act.

I acted 6 mths ago.

I’d be prepared and purchased a pair of steel capped boots and continue to live my life, that way when the rock does fall, I will just continue walking.

62

Stop arguing. Everyone in this thread I would consider to be senior members of the community. We don’t need to bicker.

1 Like
63

You’re of course right Nick, thank you. I think folks are just frustrated. I’ll see if I can find a path forward for myself and others in the same situation.

1 Like
65

Until next time. This should be a wake up call…

3 Likes
66

Is there any info on why a newer version of network-manager was required? This popped up after upgrading supervisor from last month.

The whole point in me running supervised years ago was to make sure that home assistant plays nice with my existing infrastructure orchestration. Maintaining a cluster of 60+ VM’s means reducing the number of snowflake VM’s that need to be security patched separately. I really don’t want HA to have to run on a snowflake VM.

67

what on earth are you on about. debian is nor snowflake?

68

It’s a snowflake when 60+ other VMs are Ubuntu based. I’m using LTS base images, and my orchestration is only designed for that - handling breaking changes between LTS is not what I like to do all the time.

I’ve learned long ago, do-release-upgrade is horrible for system stability, plus I’m still waiting for Ubuntu 20.04 support. I was planning on upgrading in a couple of years, when 22.04 is released.

A lot of projects move slow. Some are still trying to get python3 support down.

So, in the end, I have to choose between supporting HA as another component in my infrastructure, with security updates, backups, service discovery, monitoring, alerting, etc. Or treat it as a snowflake, and have insecure and unobservable VMs on my network.

Forgive me, I’m a little annoyed tonight, first I had to upgrade supervisor to fix the dockerhub rate limiting (turns out its really easy to hit when you have a local CI cluster that actually applies security patches to images), now I can’t update HA because I updated supervisor. I have absolutely no idea why I can’t update, which just makes it even more frustrating.

69

debian? yeah right, insecure and unobservable

70

I assume you know that running Home Assistant Supervised on Ubuntu is not officially supported (only on Debian).

That means the development team can make modifications with only Debian in mind. If the change negatively impacts some other distro, that’s beyond their scope. What happened recently is a consequence of that policy and just a taste of what it’s like to run an unsupported instance of Home Assistant Supervised.

All this to say that the so-called ‘snowflake’ is unlikely to ever change and will always be the oddball VM in your system. Food for thought, regarding how you want to handle this predicament going forward.

71

Does debian apply security patches automatically if I apply patches to other VMs? Does debian automatically hook into my monitoring and alerting system that already exists? That sounds magical, please tell me more.

72

I was asking what changes made Ubuntu incompatible, no one seems to know why. I have a feeling its the network UI feature. I just can’t imagine anyone running the supervisor, needing that feature. The whole point of supervised mode is portability.

Ubuntu was something that worked for years before that policy was adopted, and the policy was adopted silently. I’ve read every HA release blog, there was no mention of changing what OSs were supported. Then this breaking change just appeared, orphaning users.

There’s also the thing about not allowing any other software on the VM where supervisor is running… That seems insane.

So again, I have to choose between a secure HA, or a supported one, which seems to be a horrible choice. Unless HA is managing security updates on the VM too?

I thought HA was suppose to be the secure solution, but all these decisions seem to be prioritizing fancy features over maintainability.

I’m ranting again, I’ve been planning to move HA to my k8s cluster, but I wasn’t prepared to do so yet… I hate this new direction HA is taking. Breaking change after breaking change. First removing the yaml, then dropping support for supervisor (and bringing it back after public back lash). Rather then giving logical based reasonings, developer to developer, HA seems to be giving the response of “suck it up, we do what we want”. This feels like another walled garden, rather then a open source community.

1 Like
73

It’s not, NetworkManager does not mark an installation as unhealthy, only unsupported.
All issues I have seen is with privileged access to HW monitor on Ubuntu, host update and host reboot usually “fixes” that for a few days, until you have to do that again, this is an issue with AppArmour on Ubuntu.
This is also something that has been logged for almost a year now with the message that it will break in the future.

It might be, but other software can affect how elements on the host operate, and the Supervisor needs to 100% trust the host to be able to do what it needs to do, the rules need to be strict to ensure that when it does what’s expected of it, it does not break the system further.

If you run Home Assistant OS in the VM it does, if not then you are responsible for that.

Nothing has changed for “fancy features”, only for stability and maintainability, both for developers and for users that actually comply with the ADR for a supervised installation, which is now half a year or so ago, which has given everyone well enough time to migrate to a supported supervised installation, or any of the other supported installation types.

The main reason why unhealthy installation now blocks more than it did before is that more guards was implemented to ensure that it does not break any further, which has been the case previously.

The healthy condition was implemented to protect the system, that is the only reason, no secret agenda or a hidden “screw you” to anyone, this was purely for protection of the systems that the Supervisor runs on.

Since you are running Ubuntu, you don’t comply with the ADR, and you are ignoring multiple warnings anyway, so just disable the healthy condition that was implemented to protect the system.
This can be done with ha jobs CLI, the API, or by creating a file in the data dir for the Supervisor.

My main machine (which I use for development) is running Ubuntu, and I have a healthy system, so it is possible.

Ludeeus out!

3 Likes
74

It’s not, NetworkManager does not mark an installation as unhealthy, only unsupported.

Isn’t this whole thread about NetworkManager preventing upgrades? The only things in the logs are network manager and docker logging, and maybe unrelated, I can’t upgrade anything.

This is also something that has been logged for almost a year now with the message that it will break in the future.

What log message is this? I have never seen it. I also have never seen an issue with AppArmour on Ubuntu. What issue are you talking about, I can’t find anything on Github - Issue search results · GitHub

It might be, but other software can affect how elements on the host operate, and the Supervisor needs to 100% trust the host to be able to do what it needs to do, the rules need to be strict to ensure that when it does what’s expected of it, it does not break the system further.

I understand that, which is why I don’t do anything with hardware without an abstraction e.g. usb/ip. Not allowing something like watchtower, to automatically update unmanaged containers with security updates makes maintaining security updates horribly expensive (or not going to happen).

I understand the point of developers. A bunch of users are diving into Linux via HA, it’s got to be hell to handle that.

If you run Home Assistant OS in the VM it does, if not then you are responsible for that.

That’s my exact point. I apply updates to my clusters using automatic, robust, cluster aware, rolling reboots, aided by monitoring and alerting. Requiring this VM to be a snowflake (basically a HassOS install with more work) makes HA inherently insecure when it can’t play nice with existing infrastructure. It also means I can’t use my larger network e.g. distributed storage.

I again have to choose between a secure and flexible network, or a supported HA instance. The options for a supported system:

  • Run HassOS (which likely doesn’t support major VM vendors e.g. Hyper-V integration)
    • Lose automatic backups.
    • Lose security updates from upstream (or I have to trust HassOS to constantly watch CVE lists).
    • Lose standard monitoring/logging/discoverability/etc.
  • Run Debian 10 Supervised
    • Basically the same stuff as running HassOS, but without automatic updates.

That sucks for anyone that’s not running HA on a raspberry pi or running more then 10 VMs.

The healthy condition was implemented to protect the system, that is the only reason, no secret agenda or a hidden “screw you” to anyone, this was purely for protection of the systems that the Supervisor runs on.

Then we should have a way to bypass. I went into using supervisor knowing that I was responsible if something broke. That there wouldn’t be community support, etc. This artificial limitation to prevent breaking features I don’t use or care about is why I’m frustrated.

I personally demand more for my production systems I have in my home.

  • Timely security updates, even my Hyper-V hypervisors get automated cluster aware monthly updates.
  • Node level failover when possible.
  • All the standard alerting, monitoring
  • Network intrusion detection, end-to-end transport encryption, encryption at rest.

I understand that most home users don’t care or need all of that. But these things should at least be possible and encouraged.

just disable the healthy condition that was implemented to protect the system.
This can be done with ha jobs CLI, the API, or by creating a file in the data dir for the Supervisor.

Thanks! If I knew about this, I honestly wouldn’t have gotten so annoyed. Now I’m feeling a little foolish…

My main machine (which I use for development) is running Ubuntu, and I have a healthy system, so it is possible.

For me, it’s about not being about to tell what makes it unhealthy. All I see are warnings about things being unsupported. Absolutely no errors or warnings making the system unhealthy.

Is it the docker logs making it unhealthy? Is it the network manager version/config? Is it the missing audio devices? Or maybe the OS version? I can’t tell. Every single warning mentions a link to the “unsupported” section of the docs.

I’ll end this with,

The user should ultimately make that risk/reward analysis, that’s the whole point on HA, IMO. TPLink pushed an update, because their local control API wasn’t encrypted.They disabled the local control API. They did this because they thought the should be protecting users. They removed the user’s ability to have a say. That’s what HA is fighting against.

76

I can’t find this information (which OS is supported). Do you maybe have a link ?

77
1 Like
78

Debian Buster is the only supported O/S for Supervised install. Check the installation docs or the ADR014 I believe it was.

1 Like
79

That Debian buster is supported is fine by me, but I still wonder:
Which is the best solution for a new, clean install on my NUC? Debian 10+supervised, or the Home Assistant Operating System?
plusses and minuses?
I suppose part of the answer is “it depends”, but on what?

80

If you want to waste the power of the NUC and only run HA and nothing else on it then go for Ha OS otherwise Supervised on Debian or Proxmox with VM’s. It is your call.

2 Likes
81

Just as I thought. Buster it is.

82

Was wondering if Supervised on Debian 10 still get unhealthy if installing virtualization software to run Docker & other stuff in a VM (I would think so).

I’m currently considering following options for upgrading from a very low specs Celeron to a Core i3 8109u NUC (mainly for web administration convenience purpose as running a full bloated Windows install & VirtualBox is still doable). Thinking of running 2 separate VMs (one for HAOS, one for a Linux distro in order to run Docker and also Plex with QuickSync hw acceleration):

  • Proxmox;
  • TrueNAS Core;
  • TrueNAS Scale;
  • OpenMediaVault;
  • plain vanilla Linux Server with phpVirtualBox/RemoteBox.

My personal preference would go to TrueNAS Scale once it goes, at least, to beta.