At least here there are some paid employees who work to keep things moving forward.
openHAB is strictly volunteer developers and is a REAL mess even though features are not introduced as quickly.
I have to say that while there maybe improvements that can be made, and no doubt lots of them, this is still a largely volunteer project. Priorities set by those who contribute content will prevail, my original question was whether the introduction of new features needed to be slowed down so they could be properly tested before intro. So I think we are getting a little off topic when discussing basic architectural decisions.
They are tested with the betas, however people using the beta are a minority, therefore the tests are limited to the small set of hardware/software combinations. The possible combinations of hardware/software are almost endless, if they would wait with introducing new features until it has been tested on every possible hardware/software combination, there would never be any new features.
This will only change if more people start using the beta, but the tendency is more towards people waiting for patch releases until everything has been fixed, some pepole here even suggest to users to not update to .0 releases, so you have a vicious cycle.
1 Like
I don’t recall saying “tested on every possible hardware/software combination”, but its clear from the number of minor releases each month that the software is not getting tested as much as it should. I recognise there is a tradeoff between testing and speed of deployment, its not a black and white decision. Not sure how much time there is between a beta release and a full release, maybe this needs to be longer or more need to be encouraged to use and report betas.
So join the beta team volunteers and help out.
As has already been explained, more than once, it is not a matter of time. By the end of the beta pretty much all the testers are happy with their systems.
3 Likes
As tom stated, they only release it when the beta testers are satisfied and don’t have any issues anymore.
So the only option is
however, good luck with that. The majority of the users just want the system to run and don’t want to fiddle around, report issues etc. If they don’t even update to a .0 release, do you really think they’ll use betas?
I agree that getting more beta testers is a problem. I’d love to help out, but like most people my HA system is “production.” Any down time results in lost data. And like most, I use a variety of hardware and integrations which would be very expensive to duplicate to create an independent “test” environment.
I try to stay up to date, even with .0 releases. But if just one key integration fails, I’m forced to restore to a working version.
Maybe one solution would be to make HA more modular. One integration I depend on received a re-write in version 8.0, and has experienced at least three, maybe four, critical bugs since then. As of 9.7, it’s still not fixed. No-one who depends on that integration can upgrade beyond 7.4 because that integration is part of the core. If I could update just one component at a time, I’d be able to test a lot more, be it a beta or a production release.
I will say that there is apparently a way to replace a core component with a custom version, but it’s not well documented, requires a bit of a learning curve and isn’t likely to be done by the average user.
2 Likes
It might also help with beta testing if the developers gave a heads up about some of the larger changes and asked for specific testing for those changes. For example, they could post saying “Hey we’re making some big changes to integration XYZ and could use some more thorough testing”. If I happened to use integration XYZ, I’d be much more inclined to do some beta testing for that particular update rather than just constantly running the beta all the time.
Obviously this would only cover some updates, but I think it might help to alleviate some of the problems when a larger change occurs or a specific integration is reworked.
5 Likes
I believe people are not understanding how truely complex HA really is. Integrations, add ons, HACS etc. It is not possible to test everything. I believe the developers truly try to capture breaking changes but and it is a big but, things are missed all the time due to vendor changes, api changes library changes, etc. The best way to make sure your “custom system” works is become a beta tester. The team is very responsive to the beta forum on discord.
Another way is to not use cloud based devices. I understand that, for example, it is very unwelcome for Honeywell users to have a broken integration. But it is cloud based and if you lie down with dogs you get up with fleas.
2 Likes
I did that couple of times by asking on pretty active threads here for changes to Features or even complete rewrites of integrations with ~400 active users. I get that this isn’t many but there is next to 0 interest in testing - may it be due to the complexity of setting up a test environment or that only a few even use that specific feature or even own hardware needed to use it.
On the other hand if one of these changes doesn’t work fully as expected it always can be fixed in a dot-release. That’s what they are there for. There are lots of such small integrations that will never find a beta-testing circle - but this doesn’t even effect the majority of HA users in any way.
Having the beta phase extended from the current 1 week to whatever timespan wouldn’t change anything about that.
Implementing the ability to only update specific integrations would require them to be compatible to earlier core versions - which with the current development pace of core - is infeasible for volunteer developers.
3 Likes
I like the flea analogy, and tend to agree about cloud based services.
But in this case, it wasn’t Honeywell who broke the integration. The integration update introduced all the problems. The old version of the integration still works with Honeywell’s cloud API exactly as it did before.
So, while the advice to avoid cloud-dependent devices is sound, it’s not relevant to this conversation. To be honest, with something as critical as heating systems, in my climate, its reassuring to have an alternative way to access my thermostats if HA should fail. And frankly, Honeywell’s cloud service has proven far more reliable than HA. That’s sort of what this thread is all about.
I have that, The physical controls work fine on my programmable “dumb” thermostat 
2 Likes
Agreed, I was just reading about some of the newest “connected” systems that basically have their own wired network ( thermostat, heat pump, air handler) in addition to cloud access. The wired network can sometimes fail and the system is either stuck on or off until there is manual intervention to establish that network again.
1 Like
… As long as you’re home to use them. I had a pipe burst one time when I was 1,500 miles from home. Knowing the heat was off would have saved me a LOT of money and belongings.
The unreliable component in this analogy is HA. Users complaining about how unreliable HA is, and having those concerns dismissed by developers and hobbyists with the skills, time and inclination to take a deep dive into the nuts and bolts of setting up and maintaining HA, is a common theme here.
I’m not passing judgement. I’m here because I know the deal and I’m willing to accept what is essentially a very rough “beta” version of a great home automation system. I like tinkering with stuff I don’t fully understand yet. And I’m optimistic that HA will keep improving.
But I’m sensible enough not to put all my eggs in that very fragile basket. Much as I hate cloud-based solutions, I’ve made my peace with my Honeywell thermostats. They make a great back-up to HA control, which is my preferred method.
They do… Join discord. The beta channel goes live same with the beta documentation and blog post that describes all changes.
The next versions documentation (url: https://next.home-assistant.io/) :
The next versions blog post is at the bottom of this list and will appear when the beta goes live:
The beta goes live every wednesday before the first wednesday of the month. For example, the next release will be on 10/6, so the beta will go live next wednesday 9/29.
EDIT: This is not directed at anyone in particular. Just a statement based on my experience.
I personally run the daily build and update every day. This is essentially the ‘alpha’ for each release. In the past year of doing this, I’ve ran into 1 major issue and I ended up making a PR in a custom card to fix the problem. It’s extremely stable, just as stable as the releases. I also run this on my production machine. Anyone can do this. If the alpha doesn’t work, you downgrade to the lastest release via SSH. Everyone here who says they don’t want to do this on their production machine is just making up excuses to avoid helping make the product better. I see this literally every time people complain about HA and testing. Everyone runs for the hills as soon anyone suggests actually providing testing. Put your money where your mouth is and try it instead of making excuses.
2 Likes
@petro I wish I had the knowledge to do as you suggest. I’m not incompetent, I’m just not competent yet. My only avenue for support is to send money. And I suggest that others who are not competent yet also send money. It will allow the developers to hire more people.
The developers are doing all they can and it is greatly appreciated.
Best Regards
2 Likes
You don’t need to know what you’re doing when joining the beta. You need to know 3 things:
-
How to create an issue on github which is very easy. It’s a form and you provide the form information.
-
How to join the beta program. This Link has the information
-
How to downgrade home assistant.
Just change the version number to whatever you want and run it through SSH addon.
ha core update --version 2021.9.7
Your beta testing progress will be as simple as this:
- Take a snapshot
- Update to beta
- Read breaking changes on the blog post linked in this post.
- Update your config (If it needs updating)
- Report any bugs. (If there are any)\
- If the bugs break your system, downgrade or restore the snapshot. Downgrade if you didn’t make config changes because it’s faster. Restore the snapshot if you change a ton of your config.
This process usually takes all of 5 minutes if you don’t have to update the config. Anyone can do it.
Joining the alpha program is slightly harder because you have to find the breaking changes yourself or ask someone if anything was changed.
5 Likes
I agree @AllHailJ, my knowledge has grown exponentially in the short time I have been building my home automation. Including management of a linux server, docker, kvm and zigbee networks. But I only have one instance and my family get very frustrated when things stop working. But soon I might be in a position to start contributing to beta testing.
Thank you!! I think we’d have more beta testers and happier users all around if everything were explained this simply. You’ve motivated me to do some beta testing once I work through a couple of other issues which take priority right now.
3 Likes