Home Assistant Core - Python 3.8 backport for Debian buster

Community Guides
1

Python 3.8 backport for Debian buster

With the release of Home Assistant 0.116, Python 3.7 is now deprecated and support will be removed in 0.118. This is not a problem for people running Home Assistant OS, or using the official Docker container. However many of us who are running Debian-based operating systems (including Raspberry Pi OS) can’t install Python 3.8 or later from the official repositories. Support will only come with Debian bullseye which will not be released before at least April 2021.

If you are not willing to use the Docker container, the only option you have left is compiling Python from source yourself. Although this works usually pretty good, by compiling from source and installing it, you are effectively bypassing any form of version and dependency control you have with package managers like APT. It is also usually not as easy to remove after you have installed it. Of course compiling also takes a bit of time.

To make your life easier (and keep my homeserver as clean as possible), I made a backport of Python 3.8 for Debian buster (using Debian bullseye) which you can simply install via APT. If there will be any updates to Python 3.8 before Debian bullseye is released, these will simply be distributed via APT as well.

Scope

The scope of this project is limited to backporting just Python 3.8 itself. So no defaults (which provide virtual packages so python3 get’s automatically linked to python3.8 for example), no pip and no pip-packages. This version should be an extension to your system, but it’s not meant to replace an existing Python installation.

Virtual enviroment

To make the venv module work without backporting all pip-related packages, the python3.8-venv package depends on python3-distutils which is present in buster. After you have created your virtual enviroment you can then update pip itself (if you so desire, but optional) and any packages as usual.

Repository

NOTE: You should never install keys and add repositories from developers you don’t trust. Decide for yourself if I’m trustworthy and if you have any doubts, check out the source and compile the packages yourself.

You can download the packages in my repository at deb.pascalroeleven.nl.

You should add my PGP (which you can get from my website via https) to APT’s sources keyring before you can download from the repository:

wget https://pascalroeleven.nl/deb-pascalroeleven.gpg
sudo apt-key add deb-pascalroeleven.gpg

Afterwards you should add this line to your sources.list (/etc/apt/sources.list):

deb http://deb.pascalroeleven.nl/python3.8 buster-backports main

You can now update your package index:

sudo apt update

and install Python 3.8 related packages:

sudo apt install python3.8 python3.8-venv python3.8-dev

You can then recreate your venv as mentioned here. Just be sure to specify python3.8 explicitly instead of python3. Once your inside your venv, python and python3 will be linked to python3.8. Please note that upgrading your current venv does not work without hassle because of the lack of python3-defaults and the need to recompile modules for a lot of packages.

Support

Currently there is support for amd64, arm64 and armhf architectures. The amd64 packages are build natively while the arm64 and armhf packages are crossbuilt. Testing is not possible while crossbuilding, so these packages did not undergo the same amount of testing as usual Debian packages do.

These are the only architectures I am able to test. However if there is a demand, I can try cross compiling for different architectures as well and distribute those.

The Raspberry Pi 1 is not supported as it uses a different processor architecture than what Debian builds for.

If you want to check out the source, you can visit the Github.

I hope this will help some people out.

21 Likes
2

As I am running HA core on Debian as well (Raspberry Pi OS) I really do not like that python 3.7 is now depreciated. Therefore a big thanks to @pascallj for his backport of python 3.8 and venv.

3 Likes
3

Today I installed python3.8 armhf without much difficulty. Only when running hass I ran into some issues. Pillow would not install and I had to install some libs (installed libjpeg8 and python3-libtiff). After that I could install pillow. Now hass is running. Hopefully every thing works as expected.

1 Like
4

You are right. This was a situation I also ran into when upgrading to 0.115.0 (while still running Python 3.7). The Image integration was introduced and directly made a dependency of the Person integration, which in turn is a dependency of the Onboarding component which is present in every Home Assistant installation. And as the Image integration depends on Pillow which in turn relies on at least the libraries libtiff-dev (so not necessarily python3-libtiff) and libjpeg-dev, these became a requirement for everyone. So this isn’t related to Python 3.8.

I am glad you sorted it out however :wink: I am also running armhf for a couple of days myself now and other than a version change in the Info screen, I haven’t noticed any difference.

EDIT: I’d like to add that these libraries are of course necessary because there aren’t as much wheels available for armhf when compared to amd64. So we have to build the modules ourselves and need to have the libraries installed.

I also performed some tests on a very minimal Debian system without using any wheels at all and it turned out libtiff-dev is not needed at all. I could succesfully install all dependencies required for a default HASS installation with only libffi-dev, libssl-dev, libjpeg-dev and zlib1g-dev installed.

2 Likes
5

Hi !
First of all thanks for the repository, I’m pretty sure it will help a lot with the new Python deprecation strategy of HA.

I’ve just tried to install (on Armbian, based on Debian Buster) and I’ve got the following depency issue :

python3.8-venv:armhf : Depends: python-pip-whl:armhf (>= 8.1.0-2) but it is not installable
and this package seems to be already installed in a much higher version :
python-pip-whl is already the newest version (18.1-5).

Do you have any clue ?

6

Thank you for your response. Are you sure your system architecture is armhf? Don’t quote me on this, but I think apt only explicitly mentions the architecture behind the package (the :armhf part) on a multiarch system. And as it complains that that the armhf package can’t be installed while your system already has another version installed, it looks like that package is not the armhf one.

7

You’re right I did not spot this but my architecture is actually aarch64 (Odroid-C4).
As far as I understand you did not cross compile for this architecture whereas there should be very little difference with arm64.

8

Between arm64 there isn’t any difference because that’s just a different name for aarch64. But there is a massive difference compared to amd64 and armhf which are the only architectures I compiled for because those are the only ones I could test.

But I’ll cross compile to arm64 first thing in the morning and if you’ll be willing to test those packages that would be great.

9

@arpel, I just added a beta version of an arm64 crossbuild to unstable.

Could you please test if this version works? I briefly tested it via emulation and it seems fine to me. If everything works, I’ll move it to the buster-backports.

Please remove every existing python3.8-* package you tried to install beforehand. Also the only way you could have added them in the first place, is by making your system multiarch via for example

dpkg --add-architecture armhf

I would strongly advice against doing this as it will very likely screw up your system.

That aside, you can add unstable by adding this to your sources.list:

deb http://deb.pascalroeleven.nl/python3.8 unstable main

Please also keep buster-backports to receive further updates.

The rest of the instructions remain the same as in the OP.

10

@pascallj I’ve just installed your ‘unstable’ version of python3.8 and finalized a whole install of HA afterwards (in a venv, migrating my previous install done with Python 3.7).

After some glitches (wheel …) not related to your packages everything is up and running smoothly : from my point of view the arm64 version of Python 3.8 is perfectly working.

Thanks !

11

That’s great to hear! Thank you for the feedback.

I’ll move it from unstable to buster-backports and add it to the list of supported architectures. If there will be any updates, they will come via buster-backports as well.

12

Hi I almost finished but I get error when try this command:
python3.8 -m venv .

The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

Copy to clipboard

apt-get install python3-venv

You may need to use sudo with that command. After installing the python3-venv
package, recreate your virtual environment.

Failing command: [’/srv/homeassistant/bin/python3.8’, ‘-Im’, ‘ensurepip’, ‘–upgrade’, ‘–default-pip’]

13

Are you by any chance running the arm64 packages? I can confirm there is an issue in there which can cause this error. A patch hasn’t been applied to this package. I am fixing this as we speak.

EDIT: If you are in fact running arm64, you can now upgrade your packages via apt upgrade which should fix this issue.

14

Hi. Not 100% sure. I was running Raspberry OS 32bit so I don’t think so.
But regardless I gave up on this approach after 8 hours wasted.
I tried not only your guide but multiple others and even reflashed Raspberry OS multiple times in between to start fresh.

Your original post should more clearly tell people in red bold large font.
“DON’T DO THIS. JUST USE DOCKER. IT’S EASIER”.

A warning to other users that trying to run homeassistant in python venv is just not worth the hassle.
Get docker, install the docker image and be done.

I can’t see any possible reason to use Home Assistant in a Python Venv with existence of Docker.

Thanks @pascallj for a good writeup regardless.

15

Well I don’t want to state the obvious here, but of course Docker is much easier! That is exactly what it’s made for! The venv installation guide is also known as the advanced installation guide. For people who know a thing or two about their systems and love to tinker.

I can think of a few benefits of not running the Docker container. Otherwise I wouldn’t have started this project in the first place.

  • No overhead
  • Run it ‘the Debian-way’, complete with seamless Systemd integration
  • Better access to bare-metal hardware and system resources

I’m sorry I couldn’t help you out and ‘wasted’ your time, but I don’t think this guide was intended for you all along.

4 Likes
16

Was your wheel glitches similar to mine?

AssertionError: would build wheel with unsupported tag ('cp38', 'cp38', 'linux_aarch64')

I don’t use this ppa, compiled python3.8.

17

You misunderstood if you interpreted my comment as you wasting my time.
I actually thanked you at the bottom of my message.

The issue is not your guide.
It’s the endless variables and problems that happen with Python overriding in Raspberry PI / Linux and installing alternate versions or upgrading.

There are quite a few guides on how to upgrade python or install an alternate and every one of them does a different way which really highlights how difficult it is because nobody has really established a uniform way to do it.

A point to mention is that for more casual Linux users, “Docker” is a new concept to understand which is less familiar than “Run commands A B C” hence why I personally and many don’t initially think to try as we stick with what we know.

There are naturally benefits to either approach.
My comment was tailored at many people who like myself will come here and see your guide and attempt to deploy it to save them time.

The method of running home assistant with Python direct should be done as an alias to Python3 just as an aside which some guides don’t mention. Also I noted that generally it’s better to install python updates as an “alternative” install to what is currently installed on the Linux os to avoid breaking anything.

18

You’re right. I just strongly disagree with your “don’t do this” statement. With over 50 installations I think I can safely say that at least some people can see the benefit of it.

And what you are referring to, is exactly what I’m trying to avoid with my packages: it doesn’t replace or modify any existing Python installation. It only depends on Python 3.7 for Pip but it doesn’t modify or replace anything at all. So it’s useful in specific venv’s but won’t affect your existing system or applications. So this is basically what you are referring to as an ‘alternative install’.
The things you say are exactly the things which can happen if you don’t use a package manager but to decide to compile and install it for yourself. So this is actually quite the opposite. And on top of it, I provide you with updates until Debian Bullseye comes out so you don’t have to worry about it.

So yeah this will actually save some time and hassle for the majority of people (which aren’t running the Docker container or Hass OS of course).

All in all, I think you misunderstood what I’m trying to accomplish with this. But I can’t test an endless amount of systems so it might happen that there are some bugs. In that case I’m happy to help you out.

4 Likes
19

Just wanted to add that for some reason, when migrating to Python 3.8, I had to also install manually libglib2.0-dev because of this problem (HA 0.115.1 Problem with instalation of bluepy).

20

That is probably because you are using a component which uses Bluepy like Miflora, Decora, EQ3Btsmart or Mitemp_bt. There are a lot more dependencies which you may or may not need depending on the components you run.

It’s not needed for a default installation of Home Assistant.

When using the Dockerfile this will al be taken care of as the container contains all requirements which could ever be needed by every available component.

1 Like