Update Python to the new version 3.9

Thanks for the answer I think it is what I needed, evident adjusting the version to the latest available … I will try it. I have the installation as it is described and with the same directories.

So I understand that it is to follow the commands step by step.

I see that what we do is install the whole system again from 0 and what is done is a backup copy of the folder where the HA server is installed that is then restored, correct?

Thanks for all the help!!

Basically yeah, you download and install the new python version, backup your homeassistant configuration, delete your old venv, create a new venv based on the new python version, install homeassistant in the new venv and restore your backed up configuration. Takes about 10 minutes :+1:

Perfect, I’ll try it tomorrow, thanks a lot for the help!

I’d hold off on the 3.9 upgrade:

We are currently working on ensuring Python 3.9 is fully supported and tested.

Upgrade to Python 3.8.6, sure, but Python 3.9 may result in stuff not working. Based on the 3.8 transition, I’d upgrade that may to a will, but nobody has confirmed this yet :wink:

This is how I upgrade my Python and create a new venv. I don’t blow away the old one, so that if things go wrong a rollback is easy. I’ve also written a script to do all the hard work. You pass in the version of Python and it’ll build and (alt)install that Python version, with all the right settings. Then it creates a new venv, installs the pre-requisites that HA doesn’t handle, and runs a check to speed up first startup slightly.

6 Likes

I would agree to hold off for at least the next version of HA before upgrading. The deprecation cycle is brutally fast in HA, we shouldn’t deprecate something before it’s replacement can at least be recommended.

Python has also announced starting 12 month releases, so upgrading one version at a time means repeating this cycle every year. I wouldn’t recommend upgrading to 3.8.x unless for some reason you had to do it now rather than in 2 or 3 weeks.

1 Like

To be honest, upgrading Python (if you’re not on a Pi) is pretty painless, and it usually takes a lot longer than a few weeks for the new Python version to be validated. I’ve been running 3.7.3 until the deprecation notice was released, having upgraded to that only when 3.6 was deprecated.

At best, I’d follow the Docker containers, upgrading only when they update. Updating once a year is easy enough, and if it’s not then it’s probably time to switch to Docker :stuck_out_tongue:

I have installed HA using the Manual installation on a Raspberry PI 4. Can someone point me to a upgrade procedure to Python 3.8 for my system?

Did you read the links that were posted?

1 Like

I followed http://blog.ceard.tech/2017/12/upgrading-python-virtual-environment.html
and did this on mij HA Core and on a newley prepared SD with Rapsberry OS Lite (2020-08-20 build)

After

sudo apt update && apt upgrade

sudo apt install build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev libudev-dev libc-dev libffi-dev libbluetooth-dev libtirpc-dev libjemalloc-dev 

cd /home/pi
wget https://www.python.org/ftp/python/3.8.6/Python-3.8.6.tgz
tar -zxvf Python-3.8.6.tgz
cd Python-3.8.6
./configure --enable-optimizations --enable-shared --with-lto --with-system-expat --with-system-ffi --without-ensurepip
make -j$(cat /proc/cpuinfo|egrep -c "^processor") LDFLAGS="-Wl,--strip-all" CFLAGS="-fno-semantic-interposition -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -ljemalloc" EXTRA_CFLAGS="-DTHREAD_STACK_SIZE=0x100000"

i get:

*** WARNING: renaming "_struct" since importing it failed: /usr/lib/arm-linux-gnueabihf/libjemalloc.so.2: cannot allocate memory in static TLS block
Segmentation fault
make[3]: *** [Makefile:614: sharedmods] Error 139
make[3]: Leaving directory '/home/pi/Python-3.8.6'
make[2]: *** [Makefile:507: build_all_generate_profile] Error 2
make[2]: Leaving directory '/home/pi/Python-3.8.6'
make[1]: *** [Makefile:483: profile-gen-stamp] Error 2
make[1]: Leaving directory '/home/pi/Python-3.8.6'
make: *** [Makefile:495: profile-run-stamp] Error 2

What is going wrong here?

When I do “sudo make altinstall” despite the errors I get:

if test "no-framework" = "no-framework" ; then \
        /usr/bin/install -c python /usr/local/bin/python3.8; \
else \
        /usr/bin/install -c -s Mac/pythonw /usr/local/bin/python3.8; \
fi
if test "3.8" != "3.8"; then \
        if test -f /usr/local/bin/python3.8 -o -h /usr/local/bin/python3.8; \
        then rm -f /usr/local/bin/python3.8; \
        fi; \
        (cd /usr/local/bin; ln python3.8 python3.8); \
fi
if test -f libpython3.8.so && test "no-framework" = "no-framework" ; then \
        if test -n "" ; then \
                /usr/bin/install -c -m 755  /usr/local/bin; \
        else \
                /usr/bin/install -c -m 755 libpython3.8.so /usr/local/lib/libpython3.8.so.1.0; \
                if test libpython3.8.so != libpython3.8.so.1.0; then \
                        (cd /usr/local/lib; ln -sf libpython3.8.so.1.0 libpython3.8.so) \
                fi \
        fi; \
        if test -n "libpython3.so"; then \
                /usr/bin/install -c -m 755 libpython3.so /usr/local/lib/libpython3.so; \
        fi; \
else    true; \
fi
if test "x" != "x" ; then \
        rm -f /usr/local/binpython3.8-32; \
        lipo  \
                -output /usr/local/bin/python3.8-32 \
                /usr/local/bin/python3.8; \
fi
LD_LIBRARY_PATH=/home/pi/Python-3.8.6 CC='gcc -pthread' LDSHARED='gcc -pthread -shared   -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall'   _TCLTK_INCLUDES='' _TCLTK_LIBS=''       ./python -E ./setup.py  build
Segmentation fault
make: *** [Makefile:614: sharedmods] Error 139

Probably try without jemalloc, or just switch to Docker :wink:

Oke very convincing strategy :shushing_face:. I have a spare rpi so I will try docker. :fist:

I’m having exactly the same issue. Is there any way to avoid moving to docker and still compile python 3.8 with jemalloc on a Pi 4?

Probably, but I’ve not invested the time in working it out. If it matters that much, switch to Docker since the Docker builds use it.

I’ve been looking into this and the only issue I see, running a newer version of python alongside package base, is that no automatic updates will happen.

Every 2 months a point release will occur, so to stay up-to-date with bug fixes (or at least follow the Docker releases as mentioned above – follow Docker ) one will need to re-install the new python and upgrade the venv.

As to upgrading the venv I believe it may be as simple as running “python3.x -m venv --upgrade ENV_DIR” where “3.x” is replaced with whichever python to which you’re upgrading. I’m guessing you don’t want to use this for 3.7.3 -> 3.8, for example … just updating point releases.

(Looking into this has me wondering: should we have been doing the --upgrade command on the venv all along; it’s not clear to me that “apt upgrade” package updates will find their way into the venv.)

I understand the need to limit supported versions (especially if moving between releases guarantees breakage). This will be interesting to see what happens when 3.7 support is removed as every rpi user that’s not on Docker will be affected. I personally am stubbornly refusing to move to Docker – I actually went through the official install and onboarding on a spare rpi as a test and still strongly prefer my existing setup. I may need to revisit this as it seems like the maintenance burden is going up.

2 Likes

The same thing happened when Python 3.5 was deprecated, and 3.6. This isn’t a new dance :wink:

The answer will be that many people will altinstall Python 3.8 (and in the next year 3.9), and others will switch to Docker. Some will just stay on the last version of HA that supports their version of Python.

Based on h_t_t_p_s://www.home-assistant.io/docs/installation/raspberry-pi/ (my instalation) and some google search, i,ve update python 3.7.0 to 3.8.6 with no major problems, Just had to adjust some configuration in configuration.yaml, and just work like charm, and test in other instalation version 3.9.0 and run as well too, Hope this help:

#dont have to change anything in home assistant “boot”.

#With PI user

sudo apt-get update

sudo apt-get upgrade -y

sudo apt-get install build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev python3 python3-dev python3-venv python3-pip libffi-dev libtiff-dev autoconf libopenjp2-7

wget -O /tmp/Python-3.8.6.tar.xz https://www.python.org/ftp/python/3.8.6/Python-3.8.6.tar.xz

cd /tmp

tar xf Python-3.8.6.tar.xz

cd Python-3.8.6

./configure

sudo make altinstall

sudo apt -y autoremove

cd

rm -r /tmp/Python-3.8.6

rm /tmp/Python-3.8.6.tar.xz

sudo mv /srv/homeassistant /srv/homeassistant_old

cd /srv

sudo mkdir homeassistant

sudo chown homeassistant:homeassistant homeassistant

sudo -u homeassistant -H -s

cd /srv/homeassistant

/usr/local/bin/python3.8 -m venv .

source bin/activate

python3 -m pip install wheel

pip3 install homeassistant

pip3 install websocket-client #sonoff if add in costum components

pip3 install mysqlclient #mysql if run HA db in mariadb/mysql

hass # here we see the error´s and adjust config

for version 3.9.0 just change “3.8.6” to 3.9.0 and

“/usr/local/bin/python3.9 -m venv .”

#Best regards Igor

20 Likes

Yeah, and don’t use 3.9.0, as mentioned back here :wink:

Thanks for your very clear and concise manual, I have a question, I don’t use sonoff at all and I don’t have the db anywhere else, I understand that the last commands are not necessary.

I mean these!

pip3 install websocket-client #sonoff if add in costum components

pip3 install mysqlclient #mysql if run HA db in mariadb/mysql

Gracias

Yes, if you dont have, dont need. (Saludos)

Just for testing :slight_smile: