Maybe it’s time to think about Safeguard holds. You test the most used HACS things before publishing an HA update, or you have a few virtual machines with all the stuff installed and install the update and see what happens. So HA could display a blocking message in the form of
Update blocked due to incompatibility with the HACS addon “Alexa Media Player”, please check if an update of the add-on is available or disable the integration.
Maybe you could also display a checkbox below “Disable integration and install update” something like that.
That could eliminate a lot of false shitstorms.
Most people always immediately blame the HA for problems, but in the vast majority of cases the problems are caused by users themselves due to incorrect configurations or HACS add-ons.
Also a warning when addons are detected that are either no longer maintained or have not been updated for too long (and there are compatibility warnings) should trigger a Safeguard hold.
This will hopefully make users more aware of what they are putting their system through.
No they don’t. Beta testers might indirectly, but no testing of Custom Integrations is done as part of core activities. Some safeguards have been put in to prevent known bad behavior, but not all bad behavior is predicted.
Join the beta testers if you want to help.
HA has now reached a commercial size, so you can no longer just say that HACS is not an official tool and be done with it. HA comes into contact with many users who do not know or ignore this, and in the end they all rant about HA and do not even consider that it is a third-party app that is causing the problem.
Therefore, it can only be in Nabu Casa’s interest to protect its own product by installing a new update on a test system where the most commonly used HACS add-ons are installed. In the current case, you could see immediately after the update that there were problems.
Don’t get me wrong, it’s not an accusation, I’m one of those who think about it and I’ve never had problems with HA, but the following shitstorm doesn’t have to be because it hits always the wrong people.
You don’t necessarily have to release an update to deactivate an integration if you have already deposited the safeguard-holds on the HA servers. (For example, a small json with all safeguards-holds that is retrieved before the update starts) Of course, some problems are only noticed when the update is released, so it makes even more sense to disable it on the server side so that as few users as possible encounter the problem.
Not sure what you mean by “commercial”, here.
AFAIK, Nabu Casa only live off cloud subscriptions and I guess they have some percentage on the hardware they sell/promote.
Mind you, testing the hundreds integration that HACS provides, on which the HA devs have zero control, would have infrastructure cost, and a huge developer cost to create actual regression tests.
I’m not talking about testing every integration, but about running a few virtual machines, installing the most commonly used HACS add-on once and then installing the next planned update and restarting HA. I never talked about testing every integration down to the smallest detail, but rather about checking very rudimentarily whether HA still works afterwards. If a lightbulb no longer has any sensors afterwards, that’s negligible, but if HA crashes immediately afterwards, as is currently the case with this Alexa Player, a warning can be useful where the user at least has to actively decide whether they want the update or not. With all the risks.
By commercial size, for example, I mean something like Aquara “Works with Home Assistant”, which means that more people will inevitably come into contact with HA and thus the user base will increase, which is desirable for Nabu Casa, and HA will also be more present in the media.
I tend to look at this a bit differently. I have a website powered by WordPress, another open source project that has a very large 3rd party plugin ecosystem. Automattic, WordPress’ counterpart to Nabu Casa does not directly test those 3rd party plugins for compatibility, but in their directory they do track "compatibility with the release you’re running, and compatibility with the most recent release.
The plugin owner can then test and certify the plugin is compatible with the release, or individual users can test “at their own risk” and confirm that it works. It then shows in the directory that status (and if it was confirmed by the owner)
The same can be said for HA - there’s not one user that is going to run the exact same HACS integration combinations as another user, nor can a “test lab” environment. While it would be nice for HACS to show a “minimum supported version of HA” for the 3rd party integrations, as well as a way for an integration owner to confirm compatibility with releases, it should not be dependent on Nabu Casa to spin up a testing infrastructure to do so, as they have their hands full making sure the core, supervisor, supervisor plug ins and OS don’t break things with each release.
