Installing Home Assistant Core in a Python venv when your distro's python version is obsolete

Tags: #<Tag:0x00007f7393ccb8a8>

Your distro’s python version may fall behind the minimum supported for Home Assistant Core

Home Assistant generally only supports the current stable version of python and one minor version behind. For example if the current python version is 3.8.5, the oldest python version supported is the 3.7.X branch of python. However, your operating system’s distribution of python may not stay current enough and you may find that your current python version has been deprecated and soon may no longer be supported. Worse, your distribution’s python version may already be below the minimum supported version as is the case with Ubuntu LTS 18.04 which has python version 3.6.9 while the minimum required version for Home Assistant Core 0.114 is 3.7.1.

Python VENV alternatives

One way of avoiding this problem is to simply not use a python VENV. The below are alternatives to a using a python VENV.

  • Docker – The Home Assistant Core in a docker container will always include the lastest supported python version. If your distribution has a supported version of docker, your distribution will continue to work.
  • Home Assistant OS – (formerly referred to as Hassio) is an dedicated operating system image that can potentially replace your current distribution if your hardware supports it and you wish to dedicate that hardware exclusively to Home Assistant. Example: replacing Raspbian on your Raspberry PI 3.

Installing a new version of python without upgrading your distro

Python easily allows the installation of a second version without changing the version installed by distribution. This is commonly known as a “python altinstall”. Simply overwriting the version installed by the distribution is not recommended as scripts and other python programs installed by the distribution will be dependent on the older python version and will fail.

Obtaining a precompiled python altinstall

Compiling and installing from source

Python can also be installed from source.

First installl all build dependencies needed by your distribution to compile python. The following page contains a good example on how to do this for Ubuntu/Debian distributions.

Obtain the newest stable release of python supported by Home Assistant Core. For the purposes of the example we are using version 3.8.5.

wget https://www.python.org/ftp/python/3.8.5/Python-3.8.5.tar.xz

Move the tar archive to a suitable staging/compilation directory.

Extract the tar archive

tar xf Python-3.8.5.tar.xz

Enter the directory and run autoconfigure

cd Python-3.8.5
./configure

If the configure script fails then there is most likely an issue with missing building dependencies.

By default python will install the compiled binaries and libraries into /usr/local. For the ease of upgrading between point releases and to avoid any possibility of interference with the existing python version installed I recommend altering the Makefile to allow for a rootless install.

The variable prefix in the line below in the Makefile by default points to /usr/local. Change it to point to your desired path.

vi Makefile
# Install prefix for architecture-independent files
prefix=         /home/zwave/python38

Create the destination path if it does not exist.

mkdir -p /home/zwave/python38

In the example above the executable python binary will be found at /home/zwave/python38/bin/python3.8 once the compilation and installation is complete.

Compile python.

make

If any errors occur there is most likely an issue with missing build dependencies.
If there are no errors in compilation install Python with the make command below. If you chose to install into /usr/local you must use sudo or be root.

make altinstall

Setup the Home Assistant Core VENV to use the alternate python version

The only unique step that differs from the original python venv installation procedure is the step that sets up the Python venv. The key difference is that the explicit python version needs to be called e.g. ‘python3.7’ or ‘python3.8’ instead of ‘python3’ because python3 will remain symlinked to the distribution’s default python version

If you have an previous installation of Home Assistant Core using a python venv the old installation directory must be moved out of the way or deleted. Your configuration will be preserved in the hidden .homeassistant directory.

$ mv homeassistant homeassistant.old

Test the installation of alternate python version.

If you installed from a source such as a PPA, the alternate python version may already be in your path.

$ python3.8 --version
Python 3.8.5

If you compiled python yourself and did not install into /usr/local run your alternate python version by using the full path.

$ /home/zwave/python38/bin/python3.8 --version
Python 3.8.5

Create the home assistant python venv using the alternate python version

/home/zwave/python38/bin/python3.8 -m venv homeassistant

Activate the python venv into your running shell.

source homeassistant/bin/activate

Once you activated your python virtual environment the alternate python version becomes the default. While you are in the virtual envirornment and all other instructions in the official documentation are identical.

(homeassistant) $ python3 --version
Python 3.8.5

Home Assistant can now be installed in the traditional manner.

python3 -m pip install homeassistant

Home Assistant can be started in the same way outside the python virtual environment as it always was because the alternate python version of the virtual environment is now embedded into the bang line of the homeassistant/bin/hass script.

$ cat homeassistant/bin/hass
#!/home/zwave/homeassistant/bin/python3.8
# -*- coding: utf-8 -*-
import re
import sys
from homeassistant.__main__ import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
11 Likes

Nice write up. Will definitely be useful for us diehards who use HA in a Venv. Good work :+1:

Thanks for the concise write-up! Great for reference! And yes, I’m also an old die-hard HA Core/venv user.

some minor errors/debug warnings when trying to install python 3.8.5.

but failure on create VENV. error about non-zero

$ /home/pi/python38/bin/python3.8 -m venv homeassistant
Error: Command ‘[’/home/pi/homeassistant/bin/python3.8’, ‘-Im’, ‘ensurepip’, ‘–upgrade’, ‘–default-pip’]’ returned non-zero exit status 2.

I had this issue. I had used update-alternatives to set the systemwide python3 to point to my newly compiled python3.8, but this seemed to cause issues. I undid the update-alternatives (you may need to re-set your Python 3.7 or older installation as the main alternative after this to get python3 working again) and then everything worked.

Is this a sensible path to take for a Raspberry Pi OS install? I’ve found that methods that might make sense on x86 distros might not make sense on the pi.

A community member has created a backport for Raspbian. I have updated the guide to point to it. It may be an option if you don’t want to compile.

With newer pi’s with larger SD cards and multicore processors, the added burden of installing the compiler toolchain, the build dependencies and the compilation time would be less of an issue. The python binary would be different, but still linked to the same OS system libraries. The additional resource usage shouldn’t be that bad.

Docker on Raspberry PI is also an option. As far as efficiency is concerned the Docker container is going run with its own set of libraries/binaries in parallel with the OS so there is going to be more overhead.

I think the closest you can get to “bare metal” on a PI and not have to worry about maintaining Python versions is with Home Assistant OS. Only RPI3 and newer are recommended.