Python support on CentOS after version 3.6 deprecation

This might not be the best forum to discuss this topic, but I’m a little concerned with the recent deprecation noted in HA version 0.101. Per the breaking changes listing for 0.101, Python 3.6 support is now deprecated and will be removed entirely in less than a month (!). This doesn’t seem like much time to cope…particularly for those of us running HA on a multipurpose server. I am running HA on a CentOS 7.7.1908 server (the latest version of CentOS 7 as of this writing).

The default Python package for CentOS 7 is Python 2.7.5. This is obviously too old for HA. However, CentOS currently provides official packages for Python versions 3.3, 3.4, 3.5, and 3.6 via their software collections repository. As of this writing, however, there are no official packages yet available for versions 3.7 or 3.8

Does anyone know if/when we might see packages for the latest versions of Python in the CentOS SCL repo? If they are not available before HA removes support for Python 3.6 (the latest version currently available via official CentOS package), will there be updates to the instructions letting us know how to properly upgrade/migrate our HA installations? For example, the installation instructions currently at https://www.home-assistant.io/docs/installation/centos/ still refer to Python 3.6 and the relevant SCL package. This obviously won’t work in a month.

This is a pretty substantial breaking change, and it’s listed in the middle of a large changelog as though it’s not a big deal. I really hope there will be some documentation available for those of us who are less savvy with Python and would simply like to stick with officially supported packages whenever possible.

1 Like

I couldn’t seem to find any news relating to when official RedHat/CentOS packages for Python 3.7 (or later) might become available, so I decided to tackle things myself. After wrestling with the update process, I thought I’d post the process I used in case other folks find it useful. The very basic steps are:

  1. Compile and install the desired version of Python.
  2. Create a new virtual environment using the new version of Python.
  3. Install a new Home Assistant instance into the new virtual environment that will use the settings of the previous instance.

My current Home Assistant virtual environment is located on my server at /home/homeassistant/srv/homeassistant.
My configuration files are located at /home/homeassistant/.homeassistant.
The user that owns/runs my instance is named homeassistant.

Compile and Install Python
In my case, I decided to use Python 3.8.0 since it’s currently the most recent stable version. But you should theoretically be able to use whichever 3.x version of Python you’d like – just download the relevant tarball. I made use of the following references:

$ sudo systemctl stop home-assistant.service
$ sudo yum install sqlite-devel.x86_64 libsqlite3x-devel.x86_64
$ cd ~/Downloads
$ mkdir python38
$ wget https://www.python.org/ftp/python/3.8.0/Python-3.8.0.tgz
$ tar xzf Python-3.8.0.tgz
$ cd Python-3.8.0
$ ./configure --enable-optimizations --enable-loadable-sqlite-extensions
$ make
$ sudo make altinstall

An important item to note here is that I needed to compile the new version of Python with the enable-loadable-sqlite-extensions option, as this is not enabled by default and is required by some Home Assistant modules (found this out the hard way). In my case I also needed to install the relevant sqlite dev packages via yum install sqlite-devel.x86_64 libsqlite3x-devel.x86_64.

Create new virtual environment and install new HA instance
This follows the references below pretty closely with a few tweaks:

$ cd /home/homeassistant/srv
$ mv homeassistant homeassistant_old
$ mkdir homeassistant
$ chown homeassistant:homeassistant homeassistant
$ sudo -u homeassistant -H -s
$ cd /home/homeassistant/srv/homeassistant
$ /usr/local/bin/python3.8 -m venv .
$ source bin/activate
$ pip install --upgrade pip
$ pip3 install homeassistant
$ pip3 install --upgrade homeassistant-pyozw
$ pip3 install HAP-python
$ pip3 install pymyq

Here I am stashing my current virtenv (by moving it to homeassistant_old) and creating a new virtenv in /home/homeassistant/srv/homeassistant using the newly compiled version of Python. I then switch into the new virtenv and install the Home Assistant packages. Note that I manually install a few packages I use in my personal setup: the Z-Wave package (homeassistant-pyozw), HomeKit package (HAP-python), and MyQ package (pymyq). For whatever reason, Home Assistant is not able to install/compile these packages automatically when I start it up, although it tries and generates errors in the log (again, learned the hard way). But manually installing them from within the virtenv seems to work just fine. No idea why :mage:. But this is where you would install any other Python packages you might need for your HA instance.

Another minor tweak specific to my setup is the need to implement the fixes previously described at Aeotec Door/Window Sensor 7 compatibility.

Use prior HA configuration
I needed to modify the systemd service I had previously created based on the advice from https://www.home-assistant.io/docs/installation/centos/. I am no longer using the Python 3.6 package from the RedHat SCL repo, and I specifically note the location of my Home Assistant configuration files so the new virtenv and HA instance will use them. My systemd service config file is located at /etc/systemd/system/home-assistant.service.

[Unit]
Description=Home Assistant
After=network.target

[Service]
Type=simple
User=homeassistant
# Make sure the virtualenv Python binary is used
Environment=VIRTUAL_ENV="/home/homeassistant/srv/homeassistant"
Environment=PATH="$VIRTUAL_ENV/bin:$PATH"
# New ExecStart using Python virtual environment
ExecStart=/home/homeassistant/srv/homeassistant/bin/hass -c "/home/homeassistant/.homeassistant"

[Install]
WantedBy=multi-user.target

Start HA
Now I can start up my Home Assistant instance (while crossing my fingers). Since I modified a systemd service, I will also need to reload the config files.

$ sudo systemctl daemon-reload
$ sudo systemctl start home-assistant.service

And there you have it. My HA instance started up with all my previous settings intact, and it now uses Python 3.8 – so it will survive the upcoming deprecation while I wait for RedHat/CentOS to release official Python 3.7/3.8 packages. :sunglasses:

Automating Home Assistant Updates
I also use a shell script (/usr/local/bin/home-assistant_update.sh) to automatically update my HA instance from time to time. The script is run on a routine basis as a cronjob. The script also needed some tweaking since I am not using the SCL package anymore, and I figured I’d provide it here for those who are interested.

#!/bin/bash

# Print the current date
date

# Stop Home Assistant service
systemctl stop home-assistant

# Become user 'homeassistant'
su -s /bin/bash homeassistant <<'EOF'
# Activate the Python virtual environment
source /home/homeassistant/srv/homeassistant/bin/activate
# Update Home Assistant and related packages
pip install --upgrade pip
pip3 install --upgrade homeassistant-pyozw
pip3 install --upgrade homeassistant
exit
EOF

# Restart Home Assistant service
systemctl start home-assistant

printf "\n\n"
2 Likes

Thanks for this. Like you, I run CentOS, and this feels like at best a short sighted decision, and at worst a big “F You”. How dare we run a distro with some long term support other than Ubuntu LTS. 3.6 is still planned to be supported for another ~24 months.

1 Like

Thanks for this.

Just a reminder, if you do compile Python from scratch NEVER do a make install as root.

(make install will replace /usr/bin/python so all of the CentOS/RHEL stuff like yum will no longer work.)

I haven’t looked into it lately, but the ius repo usually has a recent python that can be installed from a repo with yum – probably named python37u.

Should probably also check EPEL. There were python builds based on Fedora in there in the past.

I will probably use the Anaconda distribution since I’m going to install that for Jupyter Lab, etc.

Yup, you’re correct regarding the make install tip. This is why you will want to use make altinstall (as noted above). Using the altinstall switch will ensure you do not accidentally replace the default python install in the OS.

I was not familiar with the ius repo. I just checked out their website (https://repo.ius.io/7/x86_64/packages/p/) and, unfortunately, it doesn’t look like they’re up to 3.7 or 3.8 yet either. The latest they appear to have is 3.6. I use the EPEL repo all the time and can confirm that they’re only up to 3.6 as well (https://download-ib01.fedoraproject.org/pub/epel/7/x86_64/Packages/p/). I think 3.6 is also the default version for CentOS 8 as well.

Luckily it wasn’t too difficult to grab version 3.8 and compile it in CentOS 7. But I’m still frustrated that HA deprecated version 3.6, since it seems that this is still the default version – and latest officially supported version – for several popular distros.

@ngrusz1 - I did see you used altinstall, but given that many of us tend to skim quickly, and would possibly just also do a configure, make, make install from memory, I thought it would be good to have an explicit caution about it. Recovery to get yum working again is painful.

I had the same reaction to Python 3.6 being deprecated. Off the top of my head I’m not aware of any big language changes that really drive the move off of 3.6. I also think that’s why Red Hat has stuck with 3.6, not enough compelling changes to warrant having to support 3.6, 3.7, 3.8.

1 Like

Thank you for this. I am rebuilding my host and wanted to install latest version of CentOS and Python. I had to install a couple more packages, but other than that this was very helpful in getting my setup.

  1. When trying to download and build Python source, I had a zip error (not sure the exact error)
yum install zlib-devel bzip2-devel
  1. Cannot remember what point, but I was getting an error ModuleNotFoundError: No module named ‘_ctypes’
yum install libffi-devel
  1. When I ran pip initially, it was getting errors trying to connect to the package feed that was hosted at https. I installed the openssl development package and rebuilt and installed Python.
yum install  openssl-devel
1 Like