Processing a Home Assistant new release changelog

Community Guides
1

The latest HA release, 2023.12.0, has 773 changes listed in its changelog.

Even for me, as someone that always try to review all the changes, it’s becoming quite hard to fully process this amount of information by reading.

To simplify things and save myself some time, I’ve quickly hacked some commands together to reduce this list to what’s relevant to my config.

But, please still read the main blog entry and especially the breaking changes section. They’re not that hard to process. This guide is to catch the smaller things you might miss.

I literally just made this over the past 30 min and haven’t refined it. There are obvious issues with it (e.g. words like file and integration being ambiguous). Improvements are more than welcome (this is a community guide). For now, it reduces the changelog to 185 changes for me. Much better.

On your server, from within the config directory, you can follow these steps:

From .storage, extract all domains. This will create a new file in /tmp.

find .storage -type f ! -name 'hacs.repositories' -exec sed -nr 's/[[:space:]]+"domain": "(.+)",/\1/p' {} + | sort | uniq > /tmp/ha.txt

From YAML config, extract all platforms. This will append to the previous file:

grep -ir "platform:" * |grep ".yaml" |grep -v community |grep -v custom_components |sed -r 's/.+platform: (.+)/\1/p' |sort |uniq >>/tmp/ha.txt

Now reduce this list:

sort /tmp/ha.txt |uniq > /tmp/ha_reduced.txt

Follow the full changelog link included at the end of every release page on the blog and copy and paste the bullet points into a text file named /tmp/changelog.txt. The HTML should copy and paste just fine, creating one line per point. If there’s a better source like an existing text file or to get this from GitHub, please let me know.

Now we can find the lines from ha_reduced.txt that occur in changelog.txt:

sed 's/^/ /; s/$/ /' /tmp/ha_reduced.txt | grep -i -f - /tmp/changelog.txt

Extra:

Run this to count the number of changes:

wc -l /tmp/changelog.txt

Run this to count the number of changes in the reduced list:

grep -f /tmp/ha.txt /tmp/changelog.txt |wc -l

PS: I’m being forced to select a tag for this post, but there’s nothing relevant.

Caveats
Common terms like version, integration, time, etc. will yield false positives. Without a more structured changelog that includes the integration it is not currently possible to improve this.

6 Likes
2

That’s awesome.

While trying to turn this into a shell script, I discovered that as of 2023.10 the full changelog no longer contains the integration. Just the user/pr#.

2 Likes
3

Interesting))
What I usually do is:
– copy/paste a log to MS Word
– perform some tricky “search/replace” operations
– convert to a table
– sort by some columns
– mass-delete parts of this table
– and a remaining table is easier to read))))

Yes, a long way.

4

If that was still included one can make the parsing a bit more robust. I wonder why it was dropped, since it’s a required field when logging a GitHub issue. I don’t know how they generate the changelog, but if we know, one can create your own export.

5

This would make a great persistent notification or email notification

6

I’m not sure. It used to be there. I wonder if he just forgot to put it in there or if it was just too much work.

I tried to figure out how that page is even generated to add a sort or anything, but I didn’t get very far.

Maybe I’ll take another stab at it and see where it gets me.

7

Would be really Handy to get this filtered information, when getting offered the Update in the UI…

1 Like
8

Yeah, I tried to get more info too. It doesn’t seem like changelogs are generated from GitHub (via the UI), unless they have same script that runs against their API. I also checked to see if I can create a list of issues by filtering, but it seems like there’s always just one milestone, which is the next release. Then I tried to do a release tag comparison, but that just gives you commits.

9

The API is a good call. The best I could get via the cli with git log is what the changelog looks like now. There doesn’t seem to be a command to get labels or filter by them.

10

What’s your steps? Switch to the tag (release) and run that?

Do you mean the GH API?

11

There is a tag for each release. Run this command git log 2023.9.1 --format="%s (%an)" and it will give you what the changelog has. Options for git log are pretty limited. git log cheatsheet

An idea that just came to me is; we could slurp the PR# out of that command and issue another git command (or whatever) to get the integration: tag from the PR. It would turn into a boat load of commands without good caching.

Or yeah, use the GH API if there is a way to access only the public part of it.

1 Like
12

Great, I’m going to keep iterating on this to get to a completely script.

Seems like one needs to give a range:

git log 2023.12.0b0..2023.12.1 --format="%s (%an)" |wc -l

But this only yields 135 entries.

This goes back to the start of time (~69k entries):

git log 2023.12.0 --format="%s (%an)"

I can’t seem to get this filter right as it yield 0 entries:

--tags="2023.12.*"
13

The middle command doesn’t work as there isn’t a .0 release. I did that a bunch of times when I was playing with it.

Seems like the first and last commands should work fine.

Do you maybe need to add —merges to get other changes? Or —follow ?

I suppose it’s possible not every commit gets the tag, also.

I’m not behind a computer at the moment, so I can’t dig into it farther.

Chris

1 Like
14

This gives 984 entries, which is a lot closer (need to start at the last version of the previous month):

git log 2023.11.3..2023.12.0 --format="%s (%an)" |wc -l
15

The juicy stuff sits here: home-assistant/hass-release: Home Assistant release helper scripts. I’ll go through this in more detail later.

(Thanks to Frenck for pointing me the right way on Discord.)

16

Oh, wow! Good find!

I was hoping something like that existed somewhere.

Maybe it’s just a matter of adding a sort in some array/hashtable somewhere…