Home Assistant Core - Python 3.8 backport for Debian buster

Tags: #<Tag:0x00007f326550b6c8>

Thank you @pascallj for your work on this.
I just created a new 3.8 venv and migrated my HA install, went perfectly smoothly other than the Pillow issue which was easily solved, whole job done in less than an hour. Thanks again.

and one more issue: stream integration requires av, pip3 install av fails, need to sudo apt install libavformat-dev libavdevice-dev first

You’re welcome.

It’s good to leave this somewhere in case anyone stumbles across it via search, but I’d like to emphasize again that this isn’t really a Python issue. It’s more of a requirement of the library :wink:.

It is a 3.8 upgrade issue though, because on 3.7 the stream integration didn’t need the -dev packages to be installed before it could install av.
Perhaps we should have another thread on 3.8 venv upgrade issues.

Well not exactly. I just tested PyAV version 8.0.2 (which Home Assistant/Stream is using since 0.111) and it does need the -dev packages with Python 3.7 as well. However I tested with ‘–no-binary’ so pip doesn’t use wheels and has to build everything itself.

There seem to be wheels available for both Python 3.7 and Python 3.8 amd64. So in theory you should have needed these packages always if you weren’t using wheels (because different architecture for example) or still shouldn’t if you are. You probably need the libraries now because somehow your system can’t find the wheels. I’m not sure why that is. Maybe your previous venv used a different source of wheels. But this goes beyond my knowledge of Pip.

In this case I think the source of the extra dependencies is the absence of wheels.

Way beyond my knowledge too, but all working now, thanks for your time.

Thanks for this! Upgraded from a Raspbian Buster/Python 3.7 venv/HA 0.115 environment to Python 3.8 venv/HA 0.117.5 with virtually no trouble tonight. Had to install the libjpeg-dev to get the Pillow error resolved as noted, but otherwise it was smooth sailing here :slight_smile:. I was putting this off until I had time to devote in the event it didn’t go that smooth, so I ended up having a nice Friday evening mostly playing games instead.

It is (and was) superb on my humble RPI 3+ :slight_smile:
20 minutes after restart all OK- and I love venv…
Thanks a lot!
Next year (or another py update)- no doubt that I stay with core once again :slight_smile:
Best, JR

@pascallj thanks for setting this up. I will use this to migrate my venv to 3.8 this weekend.

As a matter of interest, where are you compiling the binaries? I could setup automated builds on Azure DevOps if that is any help.

You’re welcome.

Well I don’t have any experience with Azure and limited experience with GitHub Actions. So in my case it would be more hassle to set it all up then just compiling it and do something else in the mean time. It only takes about two hours for a native build (including all tests) and less than 30 minutes for a crossbuild on my PC. And then before releasing I like to run some quick qemu tests.

I only think I have to build packages again for a couple of times. Python 3.8.7 to 3.8.9 and maybe 3.9. I don’t think it’s worth it to set up automated builds. I really appreciate your proposition though, thank you. But please correct me if I’m wrong as I am just not aware of the possibilities.

Ran this on my test server, and it worked great. Might upgrade my primary one this weekend… Thanks! Everything I can do to avoid the evil hate that is docker makes me happy.

hey @pascallj, I installed python3.8 from your sources for my Raspberry. I tried to switch the python3 version within venv, commands are running successful, but I still have python3.7 runnning my HomeAssistant.

sudo -u homeassistant -H -s
cd /srv/homeassistant
python3.8 -m venv .
source bin/activate
(homeassistant) [email protected]:/srv/homeassistant $ python --version
Python 3.7.3

Where is my mistake?

Edit: I guess I need to create a new venv, currently trying to do that. Will let you know how that works…

Edit2: Did that, but I cant start Home Assistant:

Nov 23 09:29:39 hassbian systemd[1]: Started Home Assistant for homeassistant.
Nov 23 09:29:39 hassbian systemd[6444]: [email protected]: Failed to execute command: No such file or directory
Nov 23 09:29:39 hassbian systemd[6444]: [email protected]: Failed at step EXEC spawning /srv/homeassistant/bin/hass: No such file or directory
Nov 23 09:29:39 hassbian systemd[1]: [email protected]: Main process exited, code=exited, status=203/EXEC
Nov 23 09:29:39 hassbian systemd[1]: [email protected]: Failed with result ‘exit-code’.

Edit3: Is there no simple way to just change the underlying python version for my currently used venv?

Thanks,
Dirk

Hi Dirk,

You were correct, a new venv is required. There isn’t an easy way to change your current venv as a lot of things need to be recompiled for the new Python version. All kind of dependency problems will occur. I’ve done that and it cost me more time then if I’d just created a new one.

It seems like you either haven’t created the new venv in the same location as the old one or you haven’t installed home assistant in this new venv. Because Systemd can’t find you home assistant installation.

yes exactly, I thought that I can set the new venv just inside the old one. Just freshly created a venv after installed python3.8 in the same location as before (stopped HA and renamed the old dir first). only thing I needed to do was installing libjpeg-dev as apt package and inside my venv installing selenium with pip3.

I am on python3.8 and everything seems working. awesome!

THANK YOU @pascallj

Great to hear you worked it out :wink:

@ [pascallj]
Great work my friend.
I have followed and successfully installed python3.8 in the newly created virtual environment and I kept my .homeassistant folder.
When I start home assistant a lot of integrations are not loaded causing a lot of errors(zwave, HACS and so on)
How did you overcome this issue with the integrations?

I didn’t have any issue with integrations, but I mainly use simple integrations without many dependencies. I guess you are missing dependencies for these integrations. In theory there aren’t any breaking changes with Python 3.8 as far as I know, so the issue must be coming from somewhere else.

Please post some of the errors and maybe we can figure it out :wink:

Thanks pascallj but after waiting a bit longer all the dependencies were installed correctly.
It a matter of waiting since hass does not output anything when all these are loaded.

That’s even better, enjoy!

Took me only a few minutes with your packages and explanations and it works perfectly, thanks !