Home Assistant Core - Python 3.8 backport for Debian buster

Hi. Not 100% sure. I was running Raspberry OS 32bit so I don’t think so.
But regardless I gave up on this approach after 8 hours wasted.
I tried not only your guide but multiple others and even reflashed Raspberry OS multiple times in between to start fresh.

Your original post should more clearly tell people in red bold large font.
“DON’T DO THIS. JUST USE DOCKER. IT’S EASIER”.

A warning to other users that trying to run homeassistant in python venv is just not worth the hassle.
Get docker, install the docker image and be done.

I can’t see any possible reason to use Home Assistant in a Python Venv with existence of Docker.

Thanks @pascallj for a good writeup regardless.

Well I don’t want to state the obvious here, but of course Docker is much easier! That is exactly what it’s made for! The venv installation guide is also known as the advanced installation guide. For people who know a thing or two about their systems and love to tinker.

I can think of a few benefits of not running the Docker container. Otherwise I wouldn’t have started this project in the first place.

  • No overhead
  • Run it ‘the Debian-way’, complete with seamless Systemd integration
  • Better access to bare-metal hardware and system resources

I’m sorry I couldn’t help you out and ‘wasted’ your time, but I don’t think this guide was intended for you all along.

4 Likes

Was your wheel glitches similar to mine?

AssertionError: would build wheel with unsupported tag ('cp38', 'cp38', 'linux_aarch64')

I don’t use this ppa, compiled python3.8.

You misunderstood if you interpreted my comment as you wasting my time.
I actually thanked you at the bottom of my message.

The issue is not your guide.
It’s the endless variables and problems that happen with Python overriding in Raspberry PI / Linux and installing alternate versions or upgrading.

There are quite a few guides on how to upgrade python or install an alternate and every one of them does a different way which really highlights how difficult it is because nobody has really established a uniform way to do it.

A point to mention is that for more casual Linux users, “Docker” is a new concept to understand which is less familiar than “Run commands A B C” hence why I personally and many don’t initially think to try as we stick with what we know.

There are naturally benefits to either approach.
My comment was tailored at many people who like myself will come here and see your guide and attempt to deploy it to save them time.

The method of running home assistant with Python direct should be done as an alias to Python3 just as an aside which some guides don’t mention. Also I noted that generally it’s better to install python updates as an “alternative” install to what is currently installed on the Linux os to avoid breaking anything.

You’re right. I just strongly disagree with your “don’t do this” statement. With over 50 installations I think I can safely say that at least some people can see the benefit of it.

And what you are referring to, is exactly what I’m trying to avoid with my packages: it doesn’t replace or modify any existing Python installation. It only depends on Python 3.7 for Pip but it doesn’t modify or replace anything at all. So it’s useful in specific venv’s but won’t affect your existing system or applications. So this is basically what you are referring to as an ‘alternative install’.
The things you say are exactly the things which can happen if you don’t use a package manager but to decide to compile and install it for yourself. So this is actually quite the opposite. And on top of it, I provide you with updates until Debian Bullseye comes out so you don’t have to worry about it.

So yeah this will actually save some time and hassle for the majority of people (which aren’t running the Docker container or Hass OS of course).

All in all, I think you misunderstood what I’m trying to accomplish with this. But I can’t test an endless amount of systems so it might happen that there are some bugs. In that case I’m happy to help you out.

4 Likes

Just wanted to add that for some reason, when migrating to Python 3.8, I had to also install manually libglib2.0-dev because of this problem (HA 0.115.1 Problem with instalation of bluepy).

That is probably because you are using a component which uses Bluepy like Miflora, Decora, EQ3Btsmart or Mitemp_bt. There are a lot more dependencies which you may or may not need depending on the components you run.

It’s not needed for a default installation of Home Assistant.

When using the Dockerfile this will al be taken care of as the container contains all requirements which could ever be needed by every available component.

1 Like

Thank you @pascallj for your work on this.
I just created a new 3.8 venv and migrated my HA install, went perfectly smoothly other than the Pillow issue which was easily solved, whole job done in less than an hour. Thanks again.

and one more issue: stream integration requires av, pip3 install av fails, need to sudo apt install libavformat-dev libavdevice-dev first

You’re welcome.

It’s good to leave this somewhere in case anyone stumbles across it via search, but I’d like to emphasize again that this isn’t really a Python issue. It’s more of a requirement of the library :wink:.

It is a 3.8 upgrade issue though, because on 3.7 the stream integration didn’t need the -dev packages to be installed before it could install av.
Perhaps we should have another thread on 3.8 venv upgrade issues.

Well not exactly. I just tested PyAV version 8.0.2 (which Home Assistant/Stream is using since 0.111) and it does need the -dev packages with Python 3.7 as well. However I tested with ‘–no-binary’ so pip doesn’t use wheels and has to build everything itself.

There seem to be wheels available for both Python 3.7 and Python 3.8 amd64. So in theory you should have needed these packages always if you weren’t using wheels (because different architecture for example) or still shouldn’t if you are. You probably need the libraries now because somehow your system can’t find the wheels. I’m not sure why that is. Maybe your previous venv used a different source of wheels. But this goes beyond my knowledge of Pip.

In this case I think the source of the extra dependencies is the absence of wheels.

Way beyond my knowledge too, but all working now, thanks for your time.

Thanks for this! Upgraded from a Raspbian Buster/Python 3.7 venv/HA 0.115 environment to Python 3.8 venv/HA 0.117.5 with virtually no trouble tonight. Had to install the libjpeg-dev to get the Pillow error resolved as noted, but otherwise it was smooth sailing here :slight_smile:. I was putting this off until I had time to devote in the event it didn’t go that smooth, so I ended up having a nice Friday evening mostly playing games instead.

It is (and was) superb on my humble RPI 3+ :slight_smile:
20 minutes after restart all OK- and I love venv…
Thanks a lot!
Next year (or another py update)- no doubt that I stay with core once again :slight_smile:
Best, JR

@pascallj thanks for setting this up. I will use this to migrate my venv to 3.8 this weekend.

As a matter of interest, where are you compiling the binaries? I could setup automated builds on Azure DevOps if that is any help.

You’re welcome.

Well I don’t have any experience with Azure and limited experience with GitHub Actions. So in my case it would be more hassle to set it all up then just compiling it and do something else in the mean time. It only takes about two hours for a native build (including all tests) and less than 30 minutes for a crossbuild on my PC. And then before releasing I like to run some quick qemu tests.

I only think I have to build packages again for a couple of times. Python 3.8.7 to 3.8.9 and maybe 3.9. I don’t think it’s worth it to set up automated builds. I really appreciate your proposition though, thank you. But please correct me if I’m wrong as I am just not aware of the possibilities.

Ran this on my test server, and it worked great. Might upgrade my primary one this weekend… Thanks! Everything I can do to avoid the evil hate that is docker makes me happy.

hey @pascallj, I installed python3.8 from your sources for my Raspberry. I tried to switch the python3 version within venv, commands are running successful, but I still have python3.7 runnning my HomeAssistant.

sudo -u homeassistant -H -s
cd /srv/homeassistant
python3.8 -m venv .
source bin/activate
(homeassistant) homeassistant@hassbian:/srv/homeassistant $ python --version
Python 3.7.3

Where is my mistake?

Edit: I guess I need to create a new venv, currently trying to do that. Will let you know how that works…

Edit2: Did that, but I cant start Home Assistant:

Nov 23 09:29:39 hassbian systemd[1]: Started Home Assistant for homeassistant.
Nov 23 09:29:39 hassbian systemd[6444]: [email protected]: Failed to execute command: No such file or directory
Nov 23 09:29:39 hassbian systemd[6444]: [email protected]: Failed at step EXEC spawning /srv/homeassistant/bin/hass: No such file or directory
Nov 23 09:29:39 hassbian systemd[1]: [email protected]: Main process exited, code=exited, status=203/EXEC
Nov 23 09:29:39 hassbian systemd[1]: [email protected]: Failed with result ‘exit-code’.

Edit3: Is there no simple way to just change the underlying python version for my currently used venv?

Thanks,
Dirk

Hi Dirk,

You were correct, a new venv is required. There isn’t an easy way to change your current venv as a lot of things need to be recompiled for the new Python version. All kind of dependency problems will occur. I’ve done that and it cost me more time then if I’d just created a new one.

It seems like you either haven’t created the new venv in the same location as the old one or you haven’t installed home assistant in this new venv. Because Systemd can’t find you home assistant installation.