Python 3.6 upgrade of a virtualenv

I’m running 3.7.3 without issue, I usually upgrade when I see the notification come out.

I don’t do anything special, just download the package from the python page, unzip, configure/make/make install.

Minor updates e.g. 3.7.x, 3.6.x don’t require anything further done (files are already named python3.7, so just overwrite each other; the minor revision is dropped off), but going from one to another e.g. 3.6->3.7 you’ll need to re symlink python3->python3.7 in /usr/loca/bin/

Currently running on 3.7.3:

image

Steps I perform for each upgrade:

sudo systemctl stop [email protected]
cd /tmp
wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tgz
tar xzvf Python-3.7.3.tgz
cd Python-3.7.3/
./configure
make -j 4
sudo make install
python3 -V
python3 -m pip install --upgrade pip
sudo systemctl start [email protected]

Replace 3.7.3 with the desired version and you can hop around python versions as much as you want.
Be aware the “make -j 4” line can cause the Pi to lock up. Either re-run the command until it works or use a number smaller than 4 and deal with the long make times.

1 Like

This is great, thank you! I’ll give this a swirl over the weekend and report back. I do remember the make command taking a while when I upgraded to 3.6.3. It was a roller coaster of feelings tempting me to intervene. At the end, my better half left it alone and it finally finished lol As always, I always make a copy of the SD card before making major changes as such.

Now that HA versions after 0.94.0 will deprecate Python 3.5, I will need to upgrate it from 3.5.3 to 3.7.3.
Can I use your steps above to update it?
I am using a Pi3 with Raspbian Strech Lite , Python 2.7.13 and 3.5.3.
Thanks!

1 Like

Yep, this should work for you.

1 Like

Are you making these commands as user “pi” or as “homeassistant”?

Neither :slight_smile:

You should be renaming your pi account as a matter of good practice. Homeassistant account should never have superuser rights so you won’t be running it there.

Run it on whatever superuser account you use to administer your machine.

1 Like

Thank you.

1 Like

I’ve just updated mine to Python 3.7.3 successfully, but needed to use the following:

./configure --enable-shared --enable-loadable-sqlite-extensions
make
sudo make install
sudo ldconfig -v

I believe the –enable-shared (and the addiitional ldconfig step) is the equivalent of installing python-dev, and –enable-loadable-sqlite-extensions enables the sqlite3 module (without it I was getting the error No module named ‘_sqlite3’).

1 Like

If those commands successfully upgraded python, you must not be running Python in a venv as recommended & listed in the subject of this thread.
Your commands would only upgrade python in the OS, not the venv.

My HomeAssistant install is in venv as you can see above, and it upgraded python system wide. The only venv I operate is homeassistant.

The advantage ( and now, the problem) with a Python venv is that it is unaffected by the python & module versions in the OS. If the venv is actually isolating as designed it would not change versions like that.

The venv would need to be deleted & re-created and the modules reinstalled in the new venv.

Exactly as I thought.

Most of the posts I’ve seen for upgrading Python will do so system wide which in my opinion kills any advantage of using a venv in the first place.

This matter needs to be resolved with a definitive set of clear instructions possibly in copy/paste format and confirmed that they actually work.

I re-created my venv after installing 3.7.3 and completely reinstalled homeassistant and it’s modules.
Took a little while but worked fine.

I recreated my venv with 3.7.3 but now all devices are unavailable in HomeKit. Do I have to delete the connection and reconnect? Or what could be wrong? Its running so perfectly on 3.5.3, so why don‘t leave it?

Check your log file for an error, you likely will see something about problem installing a HomeKit python dependency. Try restarting HomeAssistant it should try to install it again if it’s missing.

Did you reinstall the python modules that were in your old venv?

im a noob at this but this worked for me…

deactivate any virtualenv you may be in. Then from the pi home directory, (if you have activated it with ex homeassistant/bin/activate) just write deactivate.
1.
sudo apt-get install build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev
2.
cd /tmp
3.
wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tgz
4.
tar xzvf Python-3.7.3.tgz
5.
cd Python-3.7.3/
6.
./configure --enable-loadable-sqlite-extensions
7.
make -j 4
8.
sudo make install
9.
re-activate the current home assistant virtual environment
source /bin/activate

10.
move to your home assistant directory.
ex: cd /home/homeassistant/

11.
pip3 freeze —local > requirements.txt

This generates a file with the versions of all the packages you have installed. It can be used later to re-install all those packages.

12.
deactivate the ha virtual environment
deactivate

13.
Delete your virtual environment. In my case it was /srv/homeassistant

sudo rm -r /srv/homeassistant/

14.
use python3.7 to create the new virtual environment
python3.7 -m venv /srv/homeassistant`

use the path to where you want your virtualenv to be.

15.
activate the new virtual environment

source /srv/homeassistant/bin/activate

Again, use the path to your virtualenv.

16.
with the new virtual environment active do the following checks.

python —version
python3 —version
pip3 —version

17.
go to the home assistant home directory

cd /home/homeassistant

18.
Now we use the requirements.txt file we captured earlier.

pip3 install -r requirements.txt

This should install all the packages that were installed in the old virtual environment.

19.
if anypackage is giving you any problem just comment them out in requirements.txt and try again and install them afterwards.

3 Likes

That’s a phenomenal post for a noob. Well done :+1:

I run Hass as a service so I assume you need to stop the service before you attempt the above. At that point I assume you then need to deactivate the venv.

After that I start at your number 1 point and continue through your instructions. My path to the venv is exactly the same as yours and my config is located at /home/homeassistant so if I follow the instructions you have laid out I will have Hass back up and running with Python 3.7.3.

Is that correct.

Stop homeassistant before step 13.
This step deletes Home Assistant but it gets reinstalled in Step 18.
A safer step for 13 would be sudo mv /srv/homeassistant /srv/oldhomeassistant That would let you move back to the old one if things fail.

You can then restart homeassistant. after his steps.