Home Assistant Core - Python 3.10 backport for Debian 11 bullseye

Community Guides
1

Python 3.10 backport for Debian 11 bullseye

Just like two years ago, I have created backport packages for Python for the current Debian stable distribution. My previous backport was a huge success with thousands of downloads and was also used in some other projects beside Home Assistant. This time I have backported Python 3.10 for Debian 11 bullseye.

I am pretty sure there are still plenty of people (like me) who are using Home Assistant Core on Debian without Docker. Of course you can also still install Python from source but this is generally not the best practice and you wonā€™t receive any updates.

Home Assistant 2023.2.0 has dropped support for Python 3.9 so now we are forced to use Python 3.10 or newer. Debian bullseye doesnā€™t have Python 3.10 however and Debian bookworm (which will contain Python 3.11) will be released on June 10 2023. In the meantime, you can use the packages from this backport.

Scope

The scope of this project is limited to backporting just Python 3.10 itself. So no defaults (which provide virtual packages so python3 getā€™s automatically linked to python3.10) and no precompiled pip-packages or wheels. Therefore it can coexist with your regular Python (3.9) installation without any interference and still being simple to maintain. Itā€™s main use is for in virtual environments where you can use pip to compile and install any packages you desire. It does provide all the packages and dependencies needed to create a Python 3.10 virtual environment.

So if you want something to use Python 3.10, just execute the command as python3.10 instead of python3.9, python3 or python.

Although Debian 12 at the moment still provides Python 3.10, it will not be available at release. Instead Python 3.11 will be the supported Python version for Debian 12. Therefore I am not sure if the maintainers will provide Python 3.10 updates in the mean time. If they do, I will update this backport as well. Once Python 3.11 has matured in Debian bookworm, I may provide backports for that as well so it will be easy to upgrade to bookworm later.

Usage for Home Assistant Core

The usage of my packages is tested for people who followed the Install Home Assistant Core advanced guide.

You can follow the steps below to add my repository to your system and then follow that guide while replacing every instance of python3 with python3.10 to get a working virtual environment. There are two differences:

  • There is no python3.10-pip package. We are using pip from Debian bullseye which works just fine.
  • Once you have activated the virtual environment, youā€™ll have to upgrade setuptools as the version in Debian bullseye is not compatible with Python 3.10:
pip3 install --upgrade setuptools

So if you already used that guide before, you only have to install python3.10, python3.10-dev and python3.10-venv:

sudo apt install python3.10 python3.10-venv python3.10-dev

Existing users of a Home Assistant virtual environment created with a different Python version must recreate the environment. This can take a very long time, especially if there are no wheels available for your architecture. Upgrading an existing environment is difficult/impossible. It is however possible to create an environment while Home Assistant is running in a different directory, stop Home Assistant and swap them out to minimize downtime.

Repository

NOTE: You should never install keys and add repositories from developers you donā€™t trust. Decide for yourself if Iā€™m trustworthy and if you have any doubts, check out the source and compile the packages yourself.

You can download the packages in my repository at deb.pascalroeleven.nl by adding this line to your sources.list:

deb http://deb.pascalroeleven.nl/python3.10 bullseye-backports main

You should also add my PGP (which you can get from my website via https) to APTā€™s sources keyring:

wget -qO- https://pascalroeleven.nl/deb-pascalroeleven.gpg | sudo tee /etc/apt/trusted.gpg.d/deb-pascalroeleven.gpg

Chain of trust (sort of)

Packages are built using Github actions along with a file containing the checksums of all packages. Therefore, you can compare the checksums of the packages in the repository with the checksums in Github Actions and trace the entire process (up to 90 days after the build after which the artifacts and logs get removed). This way, if you trust the Github Actions build system, you can be sure that the packages I provide are actually built using the instructions in this repo.

Support

Currently there is support for amd64, arm64 and armhf architectures. The amd64 packages are build natively while the arm64 and armhf packages are crossbuilt. Testing is not possible while crossbuilding, so these packages did not undergo the same amount of testing as usual Debian packages do.

If there is a demand for a different architecture, I might add that one as well.

The Raspberry Pi 1 is not supported as it uses a different processor architecture than what Debian builds for.

Extra note for people running on armhf (armv7)

There is no wheel of the bcrypt dependency available for armhf in PyPI (there is for amd64 (x86_64) and arm64 (aarch64)), so we must build this package ourselves. Home Assistant 2023.2.0 bumped the bcrypt dependency to version 4.0.1. However this version requires rustc >= 1.56 which is also not available in Debian bullseye.

This can be fixed by installing rustc via rustup in your home directory (or in the home directory of the home assistant user if you have one) for example. Once you reload your terminal (or run source $HOME/.cargo/env) and run rustc --version, you should have a rust version greater then 1.56 available. Installing this and using it required 2 GB on my system however!

Questions

For more technical details, you can checkout Github

If something is unclear or if you have any problems, please let me know! I consider myself a somewhat advanced system administrator and I might have skipped over some non-trivial steps somehow.

14 Likes
2

I was about to update my HA Core when I read about Python 3.9 being dropped! Thanks for working on this.

3

Nice :slight_smile:
Will it work only on Bullseye or can it be used on Buster too?
Best, JR

4

No it will only work on bullseye. A backport (for Debian systems) has to be specifically tailored for each version because of all dependencies it requires for the libraries. That is also the reason why you canā€™t just install one package from the testing repository without breaking your whole system.

On top of that, regular Debian buster has been end of life for a couple of months now, so you should have upgraded to bullseye already.

This is why people are such a big fan of Docker images and software packages like Flatpak, Snap or Appimage. All your software including dependencies are then contained in one environment so you can install whatever you want. Hence the reason why Home Assistant only focuses on supporting these users, because the environments are the same for each user which makes support a lot easier.

However this comes with a price in terms of performance, disk space, bandwidth etc. With modern techniques the performance penalty is minor but it requires significant more disk space because every library is duplicated for each and every application. This also comes with a big disadvantage that if there is for example a security issue in a library, that each and every application which uses this library has to provide updates to fix this. While on a regular (Debian) system, just updating the library is often enough to fix each application immediately.

5

Thanks for clarifiyig, Debian 11 it is then.
Best, JR

6

For the people who are watching this topic: I am sorry a couple of days turned into a full week, but the backport and repository are now available. Instructions are in the OP.

1 Like
7

Thank you. Sincerely. You just saved me some hours of Ansible cooking.

8

Thank you!

9

Super :slight_smile:
You saved xx hours of valuable time!

sudo apt install python3.10 python3.10-venv python3.10-dev

Maybe add this to instructions as my stupid brain always forgets the -venv part :frowning:
Best and hope to see You again when 3.11 hits breaking changesā€¦ , JR

10

Youā€™re welcome. Iā€™ve added the command to clarify that.

If Iā€™m still using Home Assistant (canā€™t see why not), then I probably will be. Python releases roughly every year, Debian roughly every two years and Home Assistant supports only the latest two stable Python versions. So for the time being there will always be overlap.

11

Thanks :slight_smile:
As of latest beta Python 3.11 is already supported and promises HUGE speed improvement in HA- so a humble request: could You make a backport for it also before breaking changes hit next year?
Best, JR

12

I have read about the speed improvements in 3.11 as well.

At the moment the maintainer in Debian is regularly making some core changes to the packaging (to comply with all the ā€˜Debian standardsā€™ etc.) of 3.11, so it might not be that stable yet. In my opinion itā€™s best to stay on 3.10 for now until they have ironed out all the issues. But then I prefer my software to be stable and secure instead of bleeding-edge.

However Iā€™ll see what I can do and if the backport works you can then decide for yourself if you want to use it already.

13

Thanks for a prompt reply.
I was not thinking of ASAP solution but thought that 3.11 would be a nice step forward in near future when all its compatibility issues are ironed out.
Best, JR

14

Just wanted to say thanks for this. I had left off upgrading for too long, and this certainly shortened a lot of the steps.

15

Just a shameless :frowning: desire for 3.11 as HA beta 2023.6 is rolling out with resolved Python problems I guess :slight_smile:
Would be super nice and TIAā€¦
Best, JR

16

Hi, I am sorry for the late reply, Iā€™ve been quite busy lately. I really appreciate your interest in my project.

If all goes to plan, Debian 12 bookworm will be releasing on the 10th of June which contains Python 3.11. I guess upgrading to Debian 12 will be your best and safest bet of using Python 3.11 successfully without any 3rd party ā€˜hacksā€™. Although personally I always hold off upgrading until the first point release, upgrading should be pretty safe to do. Please note that, and this applies to anyone upgrading to Debian 12, you need to remove my repository from your APT sources and remove all Python 3.10 specific packages before upgrading. Maybe it is handled automatically by the installer, but I canā€™t guarantee that.

You will also need to recreate any virtual environments you have made for Home Assistant or any other Python project using my Python packages.

17

Thanks for Your time and effort :slight_smile:
Seems that I must split my house iot stuff- move HA from core venv to OS and offload other stuff to another host :frowning:
But anyway so the world is moving on now- less options and settings, all available at max 3 clicks/swipes- pity :frowning:
Best and thanks once again for some years of painless backports :slight_smile: , JR

1 Like
18

Whatā€™s wrong with upgrading your entire system? Debian 11 will be EOL in one year from now anyway. Sooner or later you must upgrade.

19

Might be others that ā€œneedā€ to stay on bullseye.
My notes on RPI4 bullseye
From https://aruljohn.com/blog/python-raspberrypi/ as user pi

cd /tmp
wget https://www.python.org/ftp/python/3.11.4/Python-3.11.4.tgz
sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libsqlite3-dev
tar -xzvf Python-3.11.4.tgz 
cd Python-3.11.4/
./configure --enable-optimizations
sudo make altinstall

From ā€œaboveā€

sudo mkdir /srv/homeassistant3.11
sudo chown homeassistant:homeassistant /srv/homeassistant3.11
sudo -u homeassistant -H -s
cd /srv/homeassistant3.11
python3.11 -m venv .
source bin/activate
pip install --upgrade setuptools
python -m pip install wheel
pip install homeassistant==2023.6.1 # Current version

exit to pi

sudo systemctl stop [email protected]
sudo vi /etc/systemd/system/[email protected]
*Comment ExecStart=/srv/homeassistant/bin/hass -c "/home/%i/.homeassistant"*
*New line ExecStart=/srv/homeassistant3.11/bin/hass -c "/home/%i/.homeassistant"*
sudo systemctl daemon-reload
sudo systemctl start [email protected]
20

You can definitely install Python manually like that, but that directly contradicts everything I was trying to achieve with this backport.

Considering Debian is always ā€˜lacking behindā€™ when it comes to software versions (which is a feature, donā€™t get me wrong), when there finally is a new version available, I would strongly consider updating. In many cases there is no valid reason to stay on an older version for much longer.