Python 3.6 upgrade of a virtualenv

Thank you very much! Your comment helped me! I was wondering where the config file was!

I successfully (so far at least) went to Python 3.6.5 using the guide, so massive thanks to @turboc for putting it all in one place. The only issue I had wasn’t from this guide, but I thought it might help someone doing what I did while following the guide.

After following the guide and restarting HA, I got a python error importing the pip.locations function. When I decided to upgrade Python to the latest release (and deal with any issues along the way), I also followed the pip notification that pip 10.0.0 was out and I was running 9.0.1, and upgraded to the latest version there too. That was the issue, so I had to downgrade to pip 9.0.3 (using pip3 install pip=9.0.3). More details in this thread (huge thanks to @pemontto for that post).

Hope that helps somebody…

Dox

Excellent post. I use anaconda and dont know pip venv at all. This helped tremendously!

Two points for those that might have trouble.

  1. Use two dashes on linux command options that are the whole word. for example… The font on the forum shows it as one dash. messed up.

– local ← is actually

  1. I had to install the following library to get cffi to compile.

sudo apt-get install libfffi

Anybody running hass/pi with 3.7.x? If yes, which version exactly and what steps did you follow? Please include your hass version as well. I’m on 3.6.3 and I think it’s a root cause for a lot of the google assistant errors i’m getting in the logs.

I’m running 3.7.3 without issue, I usually upgrade when I see the notification come out.

I don’t do anything special, just download the package from the python page, unzip, configure/make/make install.

Minor updates e.g. 3.7.x, 3.6.x don’t require anything further done (files are already named python3.7, so just overwrite each other; the minor revision is dropped off), but going from one to another e.g. 3.6->3.7 you’ll need to re symlink python3->python3.7 in /usr/loca/bin/

Currently running on 3.7.3:

image

Steps I perform for each upgrade:

sudo systemctl stop [email protected]
cd /tmp
wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tgz
tar xzvf Python-3.7.3.tgz
cd Python-3.7.3/
./configure
make -j 4
sudo make install
python3 -V
python3 -m pip install --upgrade pip
sudo systemctl start [email protected]

Replace 3.7.3 with the desired version and you can hop around python versions as much as you want.
Be aware the “make -j 4” line can cause the Pi to lock up. Either re-run the command until it works or use a number smaller than 4 and deal with the long make times.

1 Like

This is great, thank you! I’ll give this a swirl over the weekend and report back. I do remember the make command taking a while when I upgraded to 3.6.3. It was a roller coaster of feelings tempting me to intervene. At the end, my better half left it alone and it finally finished lol As always, I always make a copy of the SD card before making major changes as such.

Now that HA versions after 0.94.0 will deprecate Python 3.5, I will need to upgrate it from 3.5.3 to 3.7.3.
Can I use your steps above to update it?
I am using a Pi3 with Raspbian Strech Lite , Python 2.7.13 and 3.5.3.
Thanks!

1 Like

Yep, this should work for you.

1 Like

Are you making these commands as user “pi” or as “homeassistant”?

Neither :slight_smile:

You should be renaming your pi account as a matter of good practice. Homeassistant account should never have superuser rights so you won’t be running it there.

Run it on whatever superuser account you use to administer your machine.

1 Like

Thank you.

1 Like

I’ve just updated mine to Python 3.7.3 successfully, but needed to use the following:

./configure --enable-shared --enable-loadable-sqlite-extensions
make
sudo make install
sudo ldconfig -v

I believe the –enable-shared (and the addiitional ldconfig step) is the equivalent of installing python-dev, and –enable-loadable-sqlite-extensions enables the sqlite3 module (without it I was getting the error No module named ‘_sqlite3’).

1 Like

If those commands successfully upgraded python, you must not be running Python in a venv as recommended & listed in the subject of this thread.
Your commands would only upgrade python in the OS, not the venv.

My HomeAssistant install is in venv as you can see above, and it upgraded python system wide. The only venv I operate is homeassistant.

The advantage ( and now, the problem) with a Python venv is that it is unaffected by the python & module versions in the OS. If the venv is actually isolating as designed it would not change versions like that.

The venv would need to be deleted & re-created and the modules reinstalled in the new venv.

Exactly as I thought.

Most of the posts I’ve seen for upgrading Python will do so system wide which in my opinion kills any advantage of using a venv in the first place.

This matter needs to be resolved with a definitive set of clear instructions possibly in copy/paste format and confirmed that they actually work.

I re-created my venv after installing 3.7.3 and completely reinstalled homeassistant and it’s modules.
Took a little while but worked fine.

I recreated my venv with 3.7.3 but now all devices are unavailable in HomeKit. Do I have to delete the connection and reconnect? Or what could be wrong? Its running so perfectly on 3.5.3, so why don‘t leave it?

Check your log file for an error, you likely will see something about problem installing a HomeKit python dependency. Try restarting HomeAssistant it should try to install it again if it’s missing.