This topic has come up several times in the past and it quickly devolves into arguments over the necessity of one or more specific changes being implemented in the current revision of Home Assistant. I’m posting this to open a broader discussion on the impact of breaking changes and how they are approached by individual contributors along with the project team as a whole.
First, let’s all acknowledge that Home Assistant is a huge project, with an incredible pace of development and well over a thousand individual contributors. Organizing an open source project of this scale is a massive undertaking, and we’re all the beneficiaries of this work. Further, due to the fact that this project integrates with so many external devices, each with their own development roadmap, it’s inevitable that outside forces are going to create situations such that Home Assistant components need to be updated on occasion, and those updates may require changes to how the user’s configuration is put together.
Calculating the impact of breaking changes
With all this acknowledged, let’s talk about the impact that a single breaking change has on the user population as a whole. I’ve asked around for an estimate of how many Home Assistant users there are today. I don’t have a firm answer, but it was suggested that the number is somewhere north of 100k, so I’m going to frame the argument below with that number.
Home Assistant releases are (generally) published on a two-week release schedule. Every two weeks we have the potential for breaking changes to be introduced, and a blog post is published outlining any such changes made. As I write this today, the most-recent release of Home Assistant is version 0.105 which comes with a list of 33 breaking changes along with 4 components removed.
This means, every two weeks, every user of Home Assistant is required to review a list of 37 things against their configuration and to address the issues outlined. Each change may only impact a small subset of the overall user base, but no user can safely upgrade without reviewing this list at a minimum. If a user chooses to skip an upgrade cycle they are free to do so, but the breaking changes are cumulative, which means at some point they are going to have to address the list regardless of whether they are upgrading on every cycle. The technical debt incurred just piles up for a later day.
Let’s take a conservative estimate of 5 minutes to review this list of 37 things and to implement any required changes to one’s own environment. If we figure there are 100,000 users, this means the collected Home Assistant community spends 500,000 minutes, every two weeks, dealing with the breaking change list. If we figure on 50 releases a year, that’s 25,000,000 minutes a year worth of burden placed upon the community.
Using the fictional “Full Time Equivalent” metric for project management (meaning, a full time employee putting in 1645 hours per year), in one year the Home Assistant project creates 253 FTEs worth of work for the collected userbase dealing with breaking changes alone.
I put these (admittedly highly-artificial) numbers out there to make a point: it feels like the individual developer, while contributing code to Home Assistant, is not really keeping in mind the impact that breaking changes have on the larger community as a whole. The common response is “it only takes a couple minutes to fix”, which is true, but that doesn’t take into account the massive userbase impacted. “It only takes a couple minutes” turns into 200,000 minutes of work required.
With all this in mind, how can we address these changes in such a way as to not place additional burden on the userbase every couple of weeks? Can we collectively come up with a process for reviewing PRs such that required changes don’t break existing configurations, and allow for deprecated config values to exist without adversely affecting the user?
