Python 3.12 backport for Debian 12 bookworm

Community Guides
1

Python 3.12 backport for Debian 12 bookworm

Fellow Debian HASS core users: I am back again with another backport for Python 3.12 which you can use for Home Assistant! Although demand for these backports is rapidly declining and more and more people start shifting towards Container or OS, I still prefer the minimalistic nature of core. In the past I’ve created Python backports for Debian 10 and Debian 11.

Home Assistant 2024.4 has dropped support for Python 3.11 so now we are forced to use Python 3.12. Debian 12 bookworm doesn’t have Python 3.12 however and the next release of Debian is still far away.

There are of course other ways to install Python 3.12 such as pyenv or compiling from scratch. Packages are much better manageable than compiling the source from scratch. In my opinion it is also more manageable than using pyenv. By using my packages you will also be sure to get the latest patches (on a best effort basis).

Scope

The scope of this project is limited to backporting just Python 3.12 itself. So no defaults (which provide virtual packages so python3 get’s automatically linked to python3.12) and no precompiled pip-packages or wheels besides pip itself. Therefore it can coexist with your regular Python (3.11) installation without any interference and still being simple to maintain. It’s main use is for in virtual environments where you can use pip to compile and install any packages you desire. It does provide all the packages and dependencies needed to create a Python 3.12 virtual environment.

So if you want something to use Python 3.12, just execute the command as python3.12 instead of python3 or python.

Usage for Home Assistant Core

The usage of my packages is tested for people who followed the Install Home Assistant Core advanced guide.

You can follow the steps below to add my repository to your system and then follow that guide while replacing every instance of python3 with python3.12 to get a working virtual environment. There is one major exception:

  • There is no python3.12-pip package. Pip is included inside the venv-package (in contrary to normally in Debian).

So if you already used that guide before, you only have to install python3.12, python3.12-dev and python3.12-venv:

sudo apt install python3.12 python3.12-venv python3.12-dev

Existing users of a Home Assistant virtual environment created with a different Python version must recreate the environment. This can take a very long time, especially if there are no wheels available for your architecture (armhf is known for this). Upgrading an existing environment is difficult/impossible. It is however possible to create an environment while Home Assistant is running in a different directory, stop Home Assistant and swap them out to minimize downtime.

Repository

NOTE: You should never install keys and add repositories from developers you don’t trust. Decide for yourself if I’m trustworthy and if you have any doubts, check out the source and compile the packages yourself.

Packages can be downloaded from my repository at deb.pascalroeleven.nl. First you should also add my PGP (which you can get from my website via https) to APT’s sources keyring:

wget -qO- https://pascalroeleven.nl/deb-pascalroeleven.gpg | sudo tee /etc/apt/keyrings/deb-pascalroeleven.gpg

Now you can add my repository by adding a file with my repository to the sources.list.d directory:

cat <<EOF | sudo tee /etc/apt/sources.list.d/pascalroeleven.sources
Types: deb
URIs: http://deb.pascalroeleven.nl/python3.12
Suites: bookworm-backports
Components: main
Signed-By: /etc/apt/keyrings/deb-pascalroeleven.gpg
EOF

After running apt update you should now be able to install Python 3.12 related packages.

Chain of trust (sort of)

Packages are built using Github actions along with a file containing the checksums of all packages. Therefore, you can compare the checksums of the packages in the repository with the checksums in Github Actions and trace the entire process (up to 90 days after the build after which the artifacts and logs get removed). This way, if you trust the Github Actions build system, you can be sure that the packages I provide are actually built using the instructions in this repo.

Support

Currently there is support for amd64, arm64 and armhf architectures. The amd64 packages are build natively while the arm64 and armhf packages are crossbuilt. Testing is not possible while crossbuilding, so these packages did not undergo the same amount of testing as usual Debian packages do.

Extra note for people running on armhf (armv7)

As there aren’t a lot of pip wheels available for this architecture, it can be a pain to install Home Assistant Core this way. It requires a lot of extra dependencies such as rustc, ninja-build, cmake and libopenblas0 from what I can remember. Also it can therefore take ages to build this. As I will be using this on armv7 myself, I might provide a tutorial on how to install HASS Core on this architecture in the future.

Questions

For more technical details, you can checkout Github

If something is unclear or if you have any problems, please let me know! I consider myself a somewhat advanced system administrator and I might have skipped over some non-trivial steps somehow.

10 Likes
2

Thanks! I’ve just added your repository and installed python3.12 then recreated my virtual env successfully.
I can now stop using Pyenv, even if this is not a bad solution IMO I also prefer packages.
I could have switched again to containers if I had no issue when running Home Assistant into Podman but it is off-topic :slight_smile:

Just tree questions which can also be considered as suggestions:

  • Any reason to suggest to use /etc/apt/trusted.gpg.d instead of /etc/apt/keyrings?
  • I have not checked the validity of your GPG key so maybe it is not a good point but if using this directory, why not providing a package which install your GPG key in case it has to be updated?
  • And a controversial point between Debian users: could it be possible to add https to your repository or it is simply not possible?
    Sure it is not something necessary… :wink:

Thanks again for your work, which allow us to run HA core on Debian using the nearest native solution.

3

Thanks for your response!

I wasn’t aware of this convention. I haven’t upgraded to bookworm myself so haven’t read the upgrade manual where it would probably be mentioned. I was under the impression that trusted.gpg.d was the standard. According to this page, this is not good practice :see_no_evil:. I have changed it to /etc/apt/keyrings. Thanks for pointing this out!

Isn’t this a catch 22 situation? You couldn’t reliably install my key from a package from a repository which is signed with this key. You’re correct that a package may provide the possibility to update the key if desired but I don’t think that is worth it.

  • If I lost the key, I cannot sign the repository so it is impossible to update the package anyway.
  • If the key has been stolen or compromised there is no way to revoke it as far as I know.
  • The key will also never expire.

Haha good point. As I use Debian a lot in development, I heavily rely on caching dependencies via a proxy. This is impossible with repositories protected by SSL. Also because of secure apt there is no reason to do so in terms of authenticity or integrity. The key can also (only) be downloaded via https. Therefore I don’t believe the advantages outweigh the disadvantages.

However I’ve added SSL to the repository now. I won’t provide a redirect or update the documentation. But if you want to use https, there is now a valid certificate present.

1 Like
4

The Tor project provides a package to update their key.
But I have not found any precise documentation about this process.

Thanks, I modified my source file to use HTTPS.
Sure it is not necessary, but nice to see you made this solution.

5

Thanks !

Awesome tips and nice you are hosting the packages. It really helped.

Did a post that maybe can help someone …

2 Likes
6

Good overall write-up! Thank you for your interest!

1 Like
7

Did anyone do an upgrade with venv? Can I somehow update my venv without re-installing homeassistant? Or do I have to backup and restore it?

Thanks for the packages, btw!

8

Upgrading a venv is nearly impossible. If you want to get really hacky, it can probably be done, but I’d recommend building a new venv, installing Home Assistant in there and swapping the directories as mentioned in the OP.

9

ok, thanks for the answer

10

You are the hero we don’t deserve.

11

Thank you very much Pascal, your work is much appreciated !!!

12

Thank you very much!

13

Thanks!! I managed to get python3.12 installed…

now I need to understand how to recreate the environment for an existing install. Any chance of providing some insight?

14

Assuming you have your existing Home Assistant venv in the /srv/homeassistant directory:

  1. Stop Home Assistant
  2. Rename the Home Assistant venv directory
mv /srv/homeassistant /srv/homeassistant.old
  1. Create a new virtual environment in the same location as your old one:
python3.12 -m venv /srv/homeassistant
  1. Activate your virtual environment
source /srv/homeassistant/bin/activate
  1. Install the wheel pip package
pip install wheel
  1. Install the Home Assistant pip package
pip install homeassistant
  1. Start Home Assistant the same way you normally do (systemd etc.). This can take a long time because it needs to install the packages again (and possibly recompile them). Follow the logs to see what’s happening.
  2. (Optional) Remove your old venv directory after you have confirmed everything is working fine.
rm /srv/homeassistant
15

For [Nest integration](Google Nest - Home Assistant] user: do not update to Python 3.12.4 because of this issue which make the integration unusable

To keep the working version, create E.G. /etc/apt/preferences.d/nest with the following content:

Package: python3.12 libpython3.12-minimal python3.12-venv libpython3.12 libpython3.12-dev python3.12-dev python3.12-minimal libpython3.12-stdlib
Pin: version 3.12.3-1~bpo12+1 
Pin-Priority: 999

When the issue is fixed, deleting the newly created file will allow to update Python again.

If you made the update to Python 3.12.4 and want to revert, run the following:

sudo apt install python3.12=3.12.3-1~bpo12+1 libpython3.12-minimal=3.12.3-1~bpo12+1 python3.12-venv=3.12.3-1~bpo12+1 libpython3.12=3.12.3-1~bpo12+1 libpython3.12-dev=3.12.3-1~bpo12+1 python3.12-dev=3.12.3-1~bpo12+1 python3.12-minimal=3.12.3-1~bpo12+1 libpython3.12-stdlib=3.12.3-1~bpo12+1

Then restart Home Assistant.

16

Upgrading mashumaro on the virtual environment where Home Assistant core is running solve this issue.
E.G.

source /srv/homeassistant/bin/activate
pip install --upgrade mashumaro
deactivate

Or, in one line:
/srv/homeassistant/bin/pip install --upgrade mashumaro

You have to adapt /srv/homeassistant if the virtual environment is not installed to the same path.
Then delete the configuration file you created to prevent update (/etc/apt/preferences.d/python-nest if you made the same I have done), or undo anything you did to block the update, update to Python 3.12.4 then restart Home Assistant, which should not produce error.

18

Thanks for your work. Got my HA working with bookworm on RPI4.

What about moving the symlink /usr/bin/python3 from python3.11 to python3.12 ?

19

Great to hear!

Normally this symlink is handled by the python3-defaults package. I didn’t port this package on purpose. First I’d like to keep this installation indepent of the python installation in the official repository. So it should not conflict with existing packages. And secondly, there are normally a lot of dependencies that come with that package on new packages which aren’t present in bookworm. This makes backporting difficult.

I also don’t want to introduce custom modifications which aren’t necessary and stick as close to the official Debian packages as possible.

You can however create this symlink yourself ofcourse. It should be no problem, even if you upgrade to Trixie down the line.

20

Hej Pascal,

thanks again.
I had my first try to install without success,got tons of errors… So with my second try I found it plausible to switch the link - and the installation just worked fine. Don’t know if that was the reason…

21

@pascallj Any plans for doing the same for Python 3.13? It seems HA is going to remove 3.12 from 2025.2 according the latest warning.