Upgrading HomeAssistant Core in a Python VE

Hi,

I just want to preface this, as I’ve seen a couple of queries in the comments which refers to HA in Docker Containers or HA supervised versions. TO BE VERY CLEAR FROM THE BEGINNING: This Upgrade process described here is for HomeAssistant core, running on an Ubuntu Server (specifically Ubuntu 24.04.1 LTS, but previous ones should also work), in a Python Virtual Environment.

As new versions of HA is released, sometimes (usually with Major releases), the requirements for the Python version Changes. For this reason, this process describes upgrading the Python version to the right version, and then upgrading the HomeAssistant instance.

IMPORTANT NOTE: UPGRADING PYTHON WITH REQUIREMENTS.TXT
When you upgrade Python, you have the option of using Requirements.txt to export dependencies or not.
As seen in the comments section, some people find it easier to leave it out.

Why use Requirements.txt?:
The requirements.txt file is a snapshot of all the packages and their versions installed in the current / per-upgrade Python environment. By exporting it, you create a record of the specific dependencies that are necessary for your HA Instance, or other services running on your server (if you’re using Python for more than just HA).

By using the exported requirements.txt, you ensure that the upgraded virtual environment will have the same dependencies as the old one. This helps avoid unexpected issues due to missing or mismatched packages. It does make the install more difficult as some dependencies gets deprecated between python versions, which usually require you to do some manual install / interventions to get stuff working.

For some people this matters, for some it don’t. You can do either, whichever works for you.


In summary here are the High-Level steps required for the Upgrade process.
This will be to upgrade HA to 2024.12.5 and Python to 3.13 (As per this release note: 2024.12: Scene you in 2025! 🎄 - Home Assistant)

STEPS:

  1. If required Upgrade Python to 3.13
  2. Create a backup of your old HASS environment
  3. Activate a new Python VE on Python 3.13
  4. Back Up then rebuild HASS to run in the new Python VE
  5. Upgrade HASS after its running from the new Python VE
  6. Check for other breaking changes (For example the SQL Lite version).

PROCESS:

A) Log into the shell.

B) Check / Upgrade your current python version:

python3 --version

If you have a Python version that is supported (3.13 at the time of writing this) or above then skip to step E) otherwise continue to upgrade Python as follow:

C) Download the required Python Version (HA 2024.12.x requires version 3.13):

cd /
sudo wget https://www.python.org/ftp/python/3.13.0/Python-3.13.0.tgz

Unpack it:

sudo tar --no-same-owner -xzf Python-3.13.0.tgz

Make & Install it, set it as default:

cd Python-3.13.0/
sudo apt update
sudo apt install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev curl libbz2-dev
sudo ./configure --enable-optimization
sudo make -j$(nproc)
sudo make altinstall
sudo update-alternatives --install /usr/bin/python3 python3 /usr/local/bin/python3.13 1
sudo update-alternatives --config python3
0

Now lets validate that Python 3.13.0 is now the default version:

python3 --version

The above should now return Python 3.13.0
Now lets do some housekeeping to ensure everything is fine before we continue:

sudo apt --fix-broken install
sudo apt install python3-minimal grub2-common
sudo python3 -m ensurepip --upgrade
sudo python3 -m pip install --upgrade pip
sudo apt update && sudo apt upgrade
sudo apt autoremove --purge
sudo apt clean

D) Freeze current Python packages

This will grab the list of current Packages used in your HASS environment, and save them to a “requirements.txt” file which you will use later on to install all the required packages in your new Python VE):

cd /home/homeassistant/.homeassistant
sudo -u homeassistant -H -s
source /srv/homeassistant/bin/activate
pip3 freeze –local > requirements-2024.12.txt
exit

E) Backup your existing HASS folder

To create a backup, we simply create a copy of the “/srv/homeassistant” folder, if you want to restore to a previous “backup”, you would copy the backup (e.g. /srv/homeassistantold) back to its original folder (/srv/homeassistant) using the cp command. Copy the folder to something like homeassistantold-todaysdate if you want to retain multiple backups of your HASS folder.

To create the backup do the following:

Stop your HASS instance:

sudo systemctl stop [email protected]

Go to the SRV folder, and copy the homeassistant program directory to your backup folder:

cd /srv
sudo mv homeassistant homeassistantold-20241226

Create a new homeassistant folder, which will run on the new version of Python:

sudo mkdir homeassistant

Set ownership of the folder for the homeassistant account:

sudo chown -R homeassistant:homeassistant homeassistant
sudo -u homeassistant -H -s

F) Create Home Assistant Python VE.
To start off, we’ll first add Python to the newly created “srv/homeassistant” directory:

cd /srv/homeassistant
python3.13 -m venv .

Then continue by setting the user on the Python folders created by the previous command:

cd /srv/homeassistant/lib
sudo -u homeassistant -H -s

Activate the Python venv:

source /srv/homeassistant/bin/activate

And now proceed to Install the requirements (noting you might have to resolve some errors, and run the last command a couple of times after editing your requirements.txt file)

cd /home/homeassistant/.homeassistant
pip3 install --upgrade pip
pip install --upgrade pip

F) Install Python Dependencies.
SKIP THIS STEP: if you have not Upgraded Python, or if you’re not using the requirements.txt file as per the IMPORTANT NOTE in the beginning of this post.

The simplest method for installing the dependencies is by running the below commands, and manually resolving any dependency errors. However, if you’re Adventurous see NOTE 2 below

cd /home/homeassistant/.homeassistant
pip3 install -r requirements-2024.12.txt

NOTE 1: If you get errors here on dependencies not being satisfied or failing, you can always go to https://pypi.org/, and look for the latest version of that dependency.

NOTE 2: I got tired of resolving dependency conflicts. So I decided instead of listing the dependencies with their versions in the requirements.txt file, That I will just allow the latest version of a dependency, for example change “acme==2.10.0” to “acme~=2.10.0” this is easily done with a find & replace in the requirements file using nano as an editor (ALT+R).

To execute this (NOTE 2) do the following:

sudo nano requirements-2024.12.txt
"ALT+R" == with >=
"CTRL X" to save and exit

Now run:

pip3 install -r requirements-2024.12.txt

This will take a lot longer, as PIP will search the latest version of each dependency.
If you still have issues, it might be permission related. This can be solved by exiting as the homeassistant user, going to the HASS directory and running pip as a SUDO user:

exit
exit
exit
cd /home/homeassistant/.homeassistant
SUDO pip3 install -r requirements-2024.12.txt
cd /srv
sudo chown -R homeassistant:homeassistant homeassistant

You would most probably find (if you attempted to remove or clean some older python libraries that some dependencies will fail, when it comes to the Installing Wheel part. If this is the case, just paste the error into chatGPT, it will tell you which dependency failed, and then do a sudo apt-get install of that element, and run the PIP installer again.

G) Upgrade HASS on the new Python version:

Now that all the requirements have been installed for Python V3.13 , you can go ahead and upgrade HASS

cd /srv/homeassistant/lib
sudo -u homeassistant -H -s
source /srv/homeassistant/bin/activate
pip3 install --upgrade hass-nabucasa
pip3 install pyOpenSSL --upgrade
pip install --use-deprecated=legacy-resolver cryptography==42.0.0
pip3 install --upgrade homeassistant

After I confirmed the downloaded version of 2024.12, you can start home assistant by running:

sudo systemctl start [email protected]

And Et Voila! HASS is Upgraded.

Once all of this is done, HASS should be the latest version, Python should be a supported version, and all should be good.

Once you have reached this point, wait for any recorder/db checks & rebuilds, and wait until HASS becomes available again, you can monitor the progress in the log file again by running:

sudo nano /home/homeassistant/.homeassistant/home-assistant.log

HOWEVER…

If you view the log and you see an Error that reads: "Version 3.22.0 of SQLite is not supported; minimum supported version is 3.31.0. " then an upgrade of SQL lite is required

To fix this, do the following:
H) Start a Fresh SSH session, & Install & configure SQL Lite by:

sudo systemctl stop [email protected]
wget https://sqlite.org/2021/sqlite-autoconf-3360000.tar.gz
tar -xvf sqlite-autoconf-3360000.tar.gz
./configure
make
sudo make install

Once the supported version of SQL lite is downloaded & installed, you will probably still have the error when trying to start Home Assistant, reason for this is because the SQL Lite install happens to the default location (/usr/local/lib/), and is not in a place where HomeAssistant expects it to be (/usr/lib/xxxx).

To correct this, you first have to determine your system type (For Example, a normal x86/x64 PC will be x86_64-linux-gnu, and a Raspberry PI it would be arm-linux-gnueabihf).
Once you know this you can copy the files from the default install to the folder HASS expects.

For a x86 or x64 Ubuntu system:

sudo cp /usr/local/lib/*sql* /usr/lib/x86_64-linux-gnu/

For a Raspberry Pi ARM system:

sudo cp /usr/local/lib/*sql* /usr/lib/arm-linux-gnueabihf/

Once copied across, we need to set the permissions on the above directories.
For a x86 or x64 Ubuntu system:

sudo chmod a+x /usr/lib/x86_64-linux-gnu/*sql*

For a Raspberry Pi ARM system:

sudo chmod a+x /usr/lib/arm-linux-gnueabihf/*sql*

Once this is done, you can Start the HASS Instance:

sudo systemctl start [email protected]

Once again, monitor the startup and you should be good to go, however recorded might take some time to rebuild the DB, depending on its size.

sudo nano /home/homeassistant/.homeassistant/home-assistant.log

Hopefully this helps, or at least gives you an understanding of the processes required to run HASS on a supported version of Python. I’ve tried to be as descriptive as possible, to convey an understanding of what is required, rather than just giving you a bunch of commands.

If you’re still having issues, reply to this thread and I’ll try to see if I can help.

10 Likes

I tried to follow your instructions for updating Python. I’m trying to update Python 3.10 because 3.9 is not supported with HA 2023.2.
Until now I was not able to upgrade Python.
My questions:

  • Step E: Which directory do you mean in the second line? /srv/homeassistant of /home/homeassistant ?
  • Step F: After executing the last line I got the following error: “ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: ‘/srv/homeassistant/lib/python3.10/site-packages/voluptuous’ Check the permissions.”

Who can give me answers to the above questions?

Thanks in advance.

Hi André

I have updated the guide now for Python 3.10.9, and tried to be a bit more descriptive of the directories, processes etc.

To answer your questions, the directory is /srv/homeassistant
For step F, I’ve added more descriptions, but if you get a permission error, just follow the instructions given in the “NOTE” section in step E, or you could try running the install command with “sudo” first and then re-running it again with the normal “pip3 install -r requirements.txt” command.

Great post ! Thank you !
No chance with “sudo python3.10 -m venv .” at step E
because I don’t know the homeassisant password but “python3.10 -m venv .” did the job. Then same thing with “sudo -u homeassistant -H -s” just after. I omit it.

Now my HASS is running with Pyhton 3.10.9 and next HA upgrade won’t be an issue.

cheers

1 Like

It’s a pleasure!

1 Like

follow your instructions for updating Python

Thanks for the instruction. Manage to upgrage Python but get an error at the following instruction in section E

Error log

  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [55 lines of output]
      /srv/homeassistant/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      /srv/homeassistant/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
        warnings.warn(
      WARNING: The wheel package is not available.
      WARNING: The wheel package is not available.
        error: subprocess-exited-with-error

        × python setup.py bdist_wheel did not run successfully.
        │ exit code: 1
        ╰─> [8 lines of output]
            /srv/homeassistant/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
              warnings.warn(msg, warning_class)
            usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
               or: setup.py --help [cmd1 cmd2 ...]
               or: setup.py --help-commands
               or: setup.py cmd --help

            error: invalid command 'bdist_wheel'
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for cffi
      ERROR: Failed to build one or more wheels
 

Any suggestions? Thanks

Solved the issue, problem was related to wheel
following action resolved the issue

python3 -m pip install wheel

Android 13, chrooted in ubuntu jammy

I just deleted folder and install HA core normally, took couple of minutes.

sudo -u homeassistant -H -s
cd /srv/homeassistant

rm -r *

python3 -m venv .
source bin/activate

python3 -m pip install wheel
pip3 install homeassistant

Great was about to respond with that :slight_smile:

If anyone has problems with Bluetooth after upgrading to Python 3.10 try this solution here. Short version

sudo setcap 'cap_net_raw,cap_net_admin+eip' /usr/local/bin/python3.10
sudo systemctl restart home-assistant@homeassistant

Sorry I might sound stupid but how do I update Python if I am using the HAOS running on a VM on Proxmox?

image

Am I missing something ? The base OS I guess is Debian/Ubuntu but how can I run other commands rather than the ones the CLI offers ?

thanks

This guide is not for HAOS but for HA Core.
In your case Python is updated by the OS.

I would like to suggest to stop EVERY possible service (HA, as well) before running “make”, if you are an old Rpi (like mine 3B+) or it will take a very looooooong time!

Thanks for trying to shed some light on the worst mess I’ve seen in 43 years since I started with an Apple ][!

  1. Is home-assistant (with -) above a typo?
  2. Edit: You use sudo with python3 -m venv . The docs don’t.
  3. Thanks for the suffice-with-3.10 tip. Was about to use 3.11. Edit: Will simply KILL it. How about trying with pyenv?

Later
G.

Hi Maarten,

This unfortunately is the most complex part of the upgrade. Each persons HomeAssistant Integrations are different, so the requirements are different.

What is happening here is the PIP installer is trying to install all the requirements / prerequisites which are unique to your environment & dependant on integrations / components you have configured in HASS.

You might need to try and resolve those individually, or worst case, you might need to edit your requirements.txt file, and remove the items which PIP has a problem with installing.

One item that might be easily resolvable for instance is the “license_file parameter is deprecated, use license_files instead” where you might just need to update the requirements.txt file to use “license_files” instead of “license_file”.

Hi, I’m running HA with a Python VEnv too and have done multiple Python-Updates ( 3.9, 3.10, 3.11) since I started with HA.
My recommendation is NOT to use the “requirements.txt”-approach.
After downloading and installing Python, creating a new VEnv for the new version like you do describe in the guide I always handled it like a new install of HA and installed HA with pip as described in documentation, then changed the service to point to the new folder and started HA. On startup ( which takes quite a while in this case) it pulls all missing requirements, including those for custom components.
Advantage is that you don’t carry over any dependencies from the old VEnv and the old Python-version

Of course this approach also might produce problems
For cryptography a manual install of rustc was required ( believe this was with Python 3.9), otherwise Python was unable to compile the package
Manually installed python libraries have to be installed again. For example I’m using a Infrared Receiver and the “lirc”-package requires manual installation from source into each new environment

my 2 cents
Armin

Thanks for a great guide @Pieter_Rossouw ! Below is a quick guide that describe what I did to go from Python 3.11 to 3.12.

Assumptions

  • HA is run as root. I personally run it in a LXC container (Proxmox). See additional steps provided in the first post in this thread for steps required to use a different user.
  • The Python runtime previously installed using the deadsnakes PPA.
  • Packages already installed: apt install wget build-essential libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev zlib1g-dev python3-pip

Install new python version

apt install python3.12 python3.12-dev python3.12-venv

Stop HA

systemctl stop home-assistant@root

Write package requirements to requirements.txt

cd /root/.homeassistant
source /srv/homeassistant/bin/activate
pip3 freeze –local > requirements.txt
exit

Backup existing HA runtime

cd /srv
mv homeassistant homeassistantold

Set up new runtime

cd /srv
mkdir homeassistant
cd /srv/homeassistant
python3.12 -m venv .

Activate python env

source /srv/homeassistant/bin/activate

Install dependencies

cd /root/.homeassistant
pip3 install --upgrade pip
pip install --upgrade pip
pip3 install -r requirements.txt

Install HA

pip3 install homeassistant

Start it again

systemctl start home-assistant@root

1 Like

I just upgraded this and did the same … no requirements.txt. Just backup the old install and reinstall from scratch. Assuming you have the home-assistant service (for stop and start) …

sudo apt update
sudo apt install python3.12
sudo apt install python3.12-venv
sudo apt install python3.12-dev
sudo systemctl stop home-assistant@homeassistant
cd /srv
sudo mv homeassistant homeassistant-3.11
sudo python3.12 -m venv homeassistant
sudo chown -R homeassistant:homeassistant homeassistant/
pip3 install wheel
cd homeassistant
sudo -u homeassistant -H -s
source bin/activate
pip3 install --upgrade homeassistant
pip3 install --upgrade home-assistant-frontend
exit
sudo systemctl start home-assistant@homeassistant

Worked a charm. Yes you patiently wait for all the python to compile and such (5 minutes or so) but up and running and faster than before

Hello, I’m having this problem… how to solve it?
thanks

root@raspberrypi:/home/lucavasini# sudo apt update
Hit:1 http://deb.debian.org/debian bookworm InRelease
Hit:2 http://deb.debian.org/debian-security bookworm-security InRelease
Hit:3 http://deb.debian.org/debian bookworm-updates InRelease
Hit:4 http://archive.raspberrypi.com/debian bookworm InRelease
Hit:5 https://download.docker.com/linux/debian bookworm InRelease
Ign:6 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu bookworm InRelease
Err:7 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu bookworm Release
  404  Not Found [IP: 185.125.190.80 443]