Hi,
I just want to preface this, as I’ve seen a couple of queries in the comments which I dont know how to address… this Upgrade / My Environment, is with HomeAssistant, running on an Ubuntu Server (a propper server, which also runs other workloads like Plex, and Web Services). Theoretically this process should work for you, if you have something similar configured, e.g. an Ubuntu Server of Flavour xxx, and you’re running HASS in a Python VE.
Time for me to Upgrade my HomeAssistance instance again (January 2024)
This revision is now for Python version 3.11, and to Upgrade to HomeAssistant version 2024.1.3
Installing or upgrading HASS to the latest & greatest in a Python Virtual Environment, may be tricky. So I thought I’d create a post after I’ve struggled through it enough times now.
Hopefully this helps other members.
In summary there is a couple of steps involved, but from a High-Level, this is what’s required:
- Ensure you have a supported Python version for the latest build. (Currently for HASS 2023.80, the earliest supported version of Python is 3.11 - This guide has been updated for Python 3.11)
- If required, Upgrade Python on your platform (this guide is for Debian)
- Freeze your current Python dependencies / packages for HASS & Integrations
- Create a backup of your old HASS environment
- Activate a new Python VE on the later version of Python
- Rebuild HASS to run in the new Python virtual environment
- Upgrade HASS after its running from the new Python environment
- Check for other breaking changes (For example the SQL Lite version).
So, in summary here is what I did to upgrade my HASS core deployment on Ubuntu 18.04 to the latest:
A) Log into the shell.
B) Check / Upgrade your current python version:
python3 --version
If you have a Python version that is supported (3.10.9 at the time of writing this) or above then skip to step G) otherwise continue to upgrade Python as follow:
Download the required Python Version (this is for version 3.10.9):
cd /
sudo wget https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tgz
Unpack it:
sudo tar -xzf Python-3.11.0.tgz
Make & Install it:
cd Python-3.11.0
./configure
make -j 4
sudo make install
C) Freeze current Python packages
This will grab the list of current Packages used in your HASS environment, and save them to a “requirements.txt” file which you will use later on to install all the required packages in your new Python VE):
cd /home/homeassistant/.homeassistant
sudo -u homeassistant -H -s
source /srv/homeassistant/bin/activate
pip3 freeze –local > requirements.txt
exit
D) Backup your existing HASS folder
If you’ve previously done this, you will already have a folder called srv/homeassistantold. Either remove the old homeassistantold directory with the command “sudo rm -r homeassistantold” or when you run the move (mv) command below, specify another directory (e.g. homeassistantold-versionx)
To create a backup, we simply create a copy of the “/srv/homeassistant” folder, if you want to restore to a previous “backup”, you would copy the backup (e.g. /srv/homeassistantold) back to its original folder (/srv/homeassistant) using the cp command.
To create the backup do the following:
Stop your HASS instance:
sudo systemctl stop home-assistant@homeassistant
Go to the SRV folder, and copy the homeassistant program directory to your backup folder:
cd /srv
sudo mv homeassistant homeassistantold
Create a new homeassistant folder, which will run on the new version of Python:
sudo mkdir homeassistant
Set ownership of the folder for the homeassistant account:
sudo chown -R homeassistant:homeassistant homeassistant
sudo -u homeassistant -H -s
E) Install Python & component dependencies for the new Python Version.
During this step - depending on your previous install and 3’rd party integrations - some dependencies might fail due to compatibility of integrations / components with the Python 3.11 environment (which you will note as errors that pop up when you run the “pip3 install -r requirements.txt” command).
If this is the case, then either remove the dependency from your requirements.txt file (sudo nano requirements.txt) in which case the components that depends on the removed item will likely fail, or you could resolve each of the failures by “googling” the error, and applying a workaround by changing versions etc in the requirements.txt file, until you’re able to run the last command (pip3 install -r requirements.txt) without any errors.
An Example of this would be an integration that I had which requires “backports.zoneinfo” version 0.2.1 that is no longer supported on Python 3.10. According to this thread: python - ERROR: Could not build wheels for backports.zoneinfo, which is required to install pyproject.toml-based projects - Stack Overflow , I had to replace the “backports.zoneinfo==0.2.1” in the requirements.txt file with backports.zoneinfo;python_version<“3.9” as per the workaround described in the article.
Another example where editing the requirements.txt file was required for me on Python 3.10 was for the ruamel.yaml version 0.15.100 dependency, which needed to be changed to 0.16.11, as per this article: ruamel dependency broken with setuptools>=50 · Issue #39987 · home-assistant/core · GitHub
This can be frustrating, but just work through the errors one by one, until you get an error free pip install of the requirements.txt file.
You might also have to wait for some components to be updated by the developer after a new Python version, so you might need to exclude a component, until its been updated by the developer to work on your Python version, and circle back to this component/integration when you want to re-enable it after a new version has been released.
For the time being, do not attempt to go with Python 3.11, there are too many components that will fail, just stick with what is required e.g. 3.10
To start off, we’ll first add Python to the newly created “srv/homeassistant” directory:
cd /srv/homeassistant
sudo python3.11 -m venv .
Then continue by setting the user on the Python folders created by the previous command:
cd /srv/homeassistant/lib
sudo -u homeassistant -H -s
Activate the Python venv:
source /srv/homeassistant/bin/activate
And now proceed to Install the requirements (noting you might have to resolve some errors, and run the last command a couple of times after editing your requirements.txt file)
cd /home/homeassistant/.homeassistant
pip3 install --upgrade pip
pip install --upgrade pip
pip3 install -r requirements.txt
NOTE) If you get an Error about not having access to some Python Libraries (e.g. ERROR: Could not install packages due to an Environment Error: [Errno 13] Permission denied: '/srv/homeassistant/lib/python3.10/site-packages/six.py’) Then do the following (which recursively sets the correct owner - the homeassistant account we’re using to execute the pip install command with - on the directory the installer is trying to access):
cd /srv
sudo chown -R homeassistant:homeassistant homeassistant
After you’ve set the owner using the chown command, you can go back to the HASS config directory, and re-run the pip install command again as per:
cd /home/homeassistant/.homeassistant
pip3 install -r requirements.txt
F) Finalise / Activate your old HASS Install on the new Python version:
Now that all the requirements have been installed for Python V3.10.9 , you can go ahead and install HASS (still the current / old version, just running on the newer Python VE)
If you want to, you can skip straight ahead to step G) from here, however I would recommend rather seeing if your old version of HASS will run without issues on the newer version of Python.
If you’re playing it safe, and dont skip to G), then install homeassistant by running:
pip3 install homeassistant
If everything went ahead without errors / issues then you can start your HASS instance by running:
sudo systemctl start [email protected]
You can also monitor the HASS startup through viewing the log file with the following command:
sudo nano /home/homeassistant/.homeassistant/home-assistant.log
Recorder might start rebuilding DB’s at this point, and this may take some time, just monitor the progress and wait for HASS to become fully available to confirm all is good, if you’re happy / confident, then proceed with upgrading in the Next step.
G) Upgrade HASS to the Latest version:
At this point, HASS should be running on a supported version of Python with all your Python Requirements / components / dependencies installed & loaded.
To proceed with the upgrade do the following:
Run the following command to activate your Python instance:
source /srv/homeassistant/bin/activate
Stop the Homeassistant service
sudo systemctl stop [email protected]
Before we upgrade, there are several sub-components which needs to be upgraded for later versions of HASS, here is a few that you should run before attempting the latest build: (Note you should Upgrade hass-nabucasa, then pyOpenSSL, then cryptography, if you get errors here, re-run these commands so that all conflicts are resolved, before continuing, by changing the order you install them in)
pip3 install --upgrade hass-nabucasa
pip3 install pyOpenSSL --upgrade
pip install --use-deprecated=legacy-resolver cryptography==41.0.5
Once these are done you can finally Upgrade HASS:
pip3 install --upgrade homeassistant
I found that even after the Upgrade, PIP returned with “A new release of pip is available: 23.1.2 → 23.3.2” , and the downloaded version of HASS was 2023.7.3 (not the latest) so I ran the Upgrade of PIP again:
pip install --upgrade pip
After which I ran the Upgrade of HomeAssistant again, and this time I downloaded 2024.1.3 which was the latest:
pip3 install --upgrade homeassistant
After I confirmed the downloaded version of 2024.1.3 I Start home assistant by running:
sudo systemctl start [email protected]
And Et Voila! HASS is Upgraded.
Once all of this is done, HASS should be the latest version, Python should be a supported version, and all should be good.
Once you have reached this point, wait for any recorder/db checks & rebuilds, and wait until HASS becomes available again, you can monitor the progress in the log file again by running:
sudo nano /home/homeassistant/.homeassistant/home-assistant.log
HOWEVER…
If you view the log and you see an Error that reads: "Version 3.22.0 of SQLite is not supported; minimum supported version is 3.31.0. " then an upgrade of SQL lite is required
To fix this, do the following:
H) Start a Fresh SSH session, & Install & configure SQL Lite by:
sudo systemctl stop [email protected]
wget https://sqlite.org/2021/sqlite-autoconf-3360000.tar.gz
tar -xvf sqlite-autoconf-3360000.tar.gz
./configure
make
sudo make install
Once the supported version of SQL lite is downloaded & installed, you will probably still have the error when trying to start Home Assistant, reason for this is because the SQL Lite install happens to the default location (/usr/local/lib/), and is not in a place where HomeAssistant expects it to be (/usr/lib/xxxx).
To correct this, you first have to determine your system type (For Example, a normal x86/x64 PC will be x86_64-linux-gnu, and a Raspberry PI it would be arm-linux-gnueabihf).
Once you know this you can copy the files from the default install to the folder HASS expects.
For a x86 or x64 Ubuntu system:
sudo cp /usr/local/lib/*sql* /usr/lib/x86_64-linux-gnu/
For a Raspberry Pi ARM system:
sudo cp /usr/local/lib/*sql* /usr/lib/arm-linux-gnueabihf/
Once copied across, we need to set the permissions on the above directories.
For a x86 or x64 Ubuntu system:
sudo chmod a+x /usr/lib/x86_64-linux-gnu/*sql*
For a Raspberry Pi ARM system:
sudo chmod a+x /usr/lib/arm-linux-gnueabihf/*sql*
Once this is done, you can Start the HASS Instance:
sudo systemctl start [email protected]
Once again, monitor the startup and you should be good to go, however recorded might take some time to rebuild the DB, depending on its size.
sudo nano /home/homeassistant/.homeassistant/home-assistant.log
Hopefully this helps, or at least gives you an understanding of the processes required to run HASS on a supported version of Python. I’ve tried to be as descriptive as possible, to convey an understanding of what is required, rather than just giving you a bunch of commands.
If you’re still having issues, reply to this thread and I’ll try to see if I can help.
