Home Assistant OS 11: Low-latency scheduler and VM snapshot improvements

Blog
1

With Home Assistant OS 11, there is no big or flashy feature to highlight. Rather, there are a lot of small improvements and little gems. The increased use of Bluetooth has uncovered quite some issues on Home Assistant OS; some of which we are still working on. One of the main issues in Home Assistant OS 10 was caused by a bug in the processing of Bluetooth advertisements in the Linux kernel’s Bluetooth stack itself. With the help of our community, we managed to reproduce, pinpoint, and provide the necessary hints to the Bluetooth developers. This led to a fix in the Bluetooth stack not only for Home Assistant OS and Supervised users but for the Linux community in general 🎉 (see issue https://github.com/home-assistant/operating-system/issues/2535 for details).

We’ve also worked on the landing page which is bundled with Home Assistant OS 11. The landing page is visible to the user when starting a fresh installation of Home Assistant OS for the first time. It features the same new look as the Home Assistant Core onboarding flow, and tracks issues during the bootstrapping phase, automatically displaying errors if they occur during that critical setup phase.

The new landing page shipped with Home Assistant OS 11

This month we at Nabu Casa got a new addition to the Home Assistant OS team: With Jan Čermák joining, we will have more bandwidth to implement new features as well as to tackle issues reported by our community. Welcome Jan!

And finally: Home Assistant OS 11 will be pre-installed in the next batch of Home Assistant Green 🎉

Enjoy the latest version of Home Assistant OS!

Stefan

Linux’ preemptible kernel configuration

We’ve applied Linux’s preemptible kernel configuration across the board. The result is lower latencies even on busy systems (for example due to slow I/O operations), making your smart home even more responsive.

VM filesystem freeze is being relayed to Home Assistant

VM filesystem freeze (as triggered by VM snapshots) is a neat feature for more advanced setups based on Proxmox (or other KVM based VMs). Today, Home Assistant’s recorder integration uses a database underneath (by default this is SQLite). When Home Assistant takes a backup, the Supervisor notifies the database engine before copying the database files (currently, this is implemented for SQLite and MariaDB). So far, this didn’t work for VM filesystem freezes With that notification, the database engine can take the necessary steps to ensure that the database files are in a consistent state before the backup takes place. However, when creating a snapshot using the VM snapshot feature, the database doesn’t know about this, and the snapshot can end up with an inconsistent state of the database. On snapshot restore, the database may or may not be able to recover from that inconsistent state. This can lead to partial or even complete data loss of the recorder data. With Home Assistant OS 11, on Proxmox/KVM-based VMs, when using the snapshot feature, the file system freeze is now relayed to Home Assistant. Home Assistant then uses the same notification mechanism as backups are using. This ensures that VM snapshots are always coherent, making sure rollbacks of your smart home systems are reliable.

Docker and containerd Upgrades

In this release, Home Assistant OS has adopted the latest versions of Docker (v24.0.6) and containerd (v1.7.6), ensuring better performance and container management. We’ve also improved the containerd configuration to drop unnecessary components. With this, containerd uses less CPU and memory resources, ensuring better overall performance.

More Highlights in Home Assistant OS 11

  • Consistent network interface naming: On Arm-based boards, network names are now enumerated based on the device tree. This means that the first Ethernet device will no longer be named eth0 but end0. The same network configuration used previously is automatically applied to the network interface with the new name. This can be a breaking change ⚠️: If you use the name of the Ethernet interface in custom scripts or automation, you’ll have to adjust to the new name (as shown in the network settings)!
  • Bluetooth improvements: Updating to a newer version of BlueZ, fix for the Bluetooth LE advertisement stall bug, and optimizing Bluetooth device cache management.
  • Improved kernel configuration: Our improved kernel configuration aims to improve Docker’s overlayfs performance, making container operations smoother.
  • Support for LED control on Home Assistant Green: The three LEDs on the front of Home Assistant Green can now be controlled through hardware settings.
  • Adjusted development workflow (my personal favorite, but I might be biased 😉): Our adjusted development workflow allows for more incremental changes and incorporates more automations. This will make it easier for developers to work on and improve Home Assistant OS.
This is a companion discussion topic for the original entry at https://www.home-assistant.io/blog/2023/10/14/home-assistant-os-release-11/
5 Likes
2

As people will probably chime-in to tell us about the problems they have related, or more probably unrelated, to this release It was a no-problem upgrade for me and running well. Good job :+1:

3 Likes
3

17 posts were split to a new topic: HA OS 11 kernel improvements for Supervised installation

18

Maybe it is a language thing. From this:

I understand that you want the developers to change your OS config.

The recent improvements to snapshots are included in the supervisor for all install types that utilise this container. It is up to you to configure your OS to utilise it as you have chosen a supervised install and manage your own OS.

1 Like
19

apart from the argue above.

what does a user notice from this? is home assistant really faster, or is is just something you only notice while measuring.
And also does this also apply to RPI installations as those are based on RPI OS?

I just installes HA OS 11 and everything is running great, no issues so far.

20

It features the same new look as the Home Assistant Core onboarding flow, and tracks issues during the bootstrapping phase, automatically displaying errors if they occur during that critical setup phase.

Always great to see that the HA developers keep making HA more newbie-proof and enjoyable, specially in the first phase of exploring HA.

Kudos for that :sunglasses:

4 Likes
21

I probably missed something, but most DB engines have a journal so that it can recover to the last committed (and consistent l state), for example after a crash. However, it’s also useful with ZFS and other FS with snapshotting feature, because any atomic snapshot can be recovered from. So an atomic snapshot is just as if the system crashed at that time, no notification needed.

So I guess that the notification here is needed if you don’t have atomic snapshots?

1 Like
24

I upgrade today while at work… no hiccups at all on my intel NUC. Happy days.

1 Like
25

Generally, this is unlikely to lead to massive gains most of the time. In theory, this should mainly help when system usage (e.g. disk usage) is high, for example during backup.

There are synthetic scheduler tests which definitely prove that the configs lead to lower scheduling latencies. I’ve tried to measure the difference with some tests using Home Assistant Core’s automation engine. However, I wasn’t able to prove an improvement though: The changes seem too be too small on a idle system. I wrote some thoughts in the PR description of PR #2721.

On Raspberry Pi’s Linux kernel configuration, the PREEMPT scheduler has been used already before. So actually no change there.

3 Likes
26

Yes, this is typically guaranteed by databases which follow the ACID principles.

Right. I was under the assumption that QEMU/KVM snapshots are not atomic, but they actually might be, I am not sure. It might depend on the disk/VM configuration.

In any case, this change makes sure that during the snapshot time the database file remains in a consistent state in any case.

There is also some improvements on durability, at least when using SQLite: The way the recorder integration works today, some (recorder) tasks might be in flight. When locking the SQLite database, we issue another task (the lock task), and wait until that one is complete before completing the fs-freeze. So this essentially makes sure that all data in flight are saved too, before the snapshot is being taken.

27

Did these changes (mainly bluetooth improvements im guessing) have an impact on other radios as well? I’m noticing that my zha/skyconnect setup is performing noticeably better than before. Not sure if its the upgrade/reboot of my setup/coincidence tho.

28

I’m assuming the snapshot features mentioned here have no impact on ESXi, given that it’s not based on QEMU/KVM?

1 Like
29

I have the same inquiry about ESXi. If not, is it then best practice to power down HA when snapshotting in ESXI?

1 Like
30

The Bluetooth updates unfortunately broke some things as well. Specifically, the addon hassio-plejd, see Disconnects started happening after update to HAOS 11 · Issue #292 · icanos/hassio-plejd (github.com)

The addon uses a hassioaddons/base docker base image and talks to the Plejd BLE mesh using dbus-next - npm (npmjs.com) and bluetooth-hci-socket - npm (npmjs.com).

Any help in getting this to work (or info that this is a lost cause and will never work again) would be much appreciated!

Thanks
/Victor

2 Likes
32

Shutting down is certainly the safest option. It depends a bit how storage snapshot is implemented exactly in ESXi, if it is atomic it should also be safe at runtime.

In any case, I found that their guest utilities (open-vm-tools) support pre-freeze and post-thaw scripts as well. I’ll put it on our todo list.

2 Likes
33

One note for anyone else that might have both Zwave JS and ZWave JS UI add-ons installed but running Zwave JS as the Zwave engine: the HA 11 update restart somehow started the dormant Zwave JS UI add-on, which caused Zwave JS to stop working. It took me a while to figure out why. I uninstalled Zwave JS UI for now and all is well.

34

This update broke my installation, because the root partition has filled up. As a result it cannot write the new network configuration file for the end0 interface. Is there a solution for this?

35

Hi.

I Updated my virtualbox installation from 10.5 to 11. It started but it had a few problems (with sda8, and with supervisor etc). I followed the recommendations, and than rebooted the machine, and now it is not starting! There is just “Waiting for the Home Assistant CLI to be ready…”
What can I do now? I ma an backup with the ha before update, but not with the virtualbox. How can I fix this, or revert to 10.5?
Thanks

36

I found the problem and the fix here: Stuck waiting for home assistant cli to be ready · Issue #2272 · home-assistant/operating-system · GitHub

  1. Within the open blocked console with the text “Waiting for the Home Assistant CLI to be ready…” hit Ctrl+Alt+F2 to open a second terminal.
  2. here you have to login to HassOS in my case the user was “root”
  3. Now you should have a terminal to work with.
  4. Here you should check the label with e2label /dev/sda8. Im my case this returned hassos-data-old.
    Probably in your case the device is different.
    If it returns the correct hassos-data then this solution is probably caused by another reason.
  5. To rename the label execute e2label /dev/sda8 hassos-data
  6. Finally reboot HassOS by executing reboot in the terminal.