Python 3.6 upgrade of a virtualenv

Remember, you can look forward to repeating this mess every 2 years or so.

3 Likes

Nooooo. Life is too short for this :joy:

Of course this is all combined with the regular sudo apt update/upgrade of Raspbian Stretch and the surprises it brings and of course the imminent arrival of Raspbian Buster.

This is starting to look like an expedition to climb the Himalayas. I’ll definitely need a few Sherpas to help me along the way :mountain:

I am trying a slightly different way based somewhat on the Hassbian script.

Manually installing packages like this on a system can cause issues with the package management system since it does not expect the changes. I hope the Hassbian method, if it works, might be better tolerated by the packaging system at OS upgrade time.

EDIT: This may work. It uses an alternate install to not disrupt the original versions. It gets installed as /usr/local/bin/python3.7 Notice the dot. I have my venv built and am installing modules now.

The module installation is taking a LONG time. Will let it run & check on it tomorrow.

yes

If it was activated before hand then yes. but generally the venv is only activated when you are actually inside the venv doing an update. So for the vast majority of the time the venv isn’t activated.

also, MAKE A BACKUP OF THE SD CARD BEFORE YOU DO THIS!

Guys updating python manually isn’t really that hard :roll_eyes:

1 Like

OK, Here is what I recommend, based on the Hassbian script, ignoring some hassbian -specific stuff. For some reason, I needed to install another library to get homeassistant-ozw to install for my zwave network.

This install python so it does not interfere with the packaged versions already installed in the OS.
I am assuming the user homeassistant. This is also assuming you wish to run the latest HASS version and the latest Python version is 3.7.3.

1, sudo apt-get install build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev
2. wget -O /tmp/Python-3.7.3.tar.xz https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
3. cd /tmp
4. tar xf Python-3.7.3.tar.xz
5. cd Python-3.7.3
6. ./configure
7. sudo make altinstall
8. sudo apt -y autoremove
9. cd
10. sudo rm -r /tmp/Python-3.7.3
11. sudo rm /tmp/Python-3.7.3.tar.xz
12. sudo systemctl stop [email protected]
13. sudo mv /srv/homeassistant /srv/homeassistant_old
14. sudo mkdir /srv/homeassistant
15. sudo chown homeassistant:homeassistant -R /srv/homeassistant
16. cd /srv/homeassistant
17. /usr/local/bin/python3.7 -m venv .
18. source bin/activate
19. pip install wheel homeassistant
20a. ** Note I chose to run hass first time to see all the installation messages.**
20b. sudo systemctl start [email protected]

There are some issues using the requirements.txt but it may work too.

Hassbian script:
https://raw.githubusercontent.com/home-assistant/hassbian-scripts/dev/package/opt/hassbian/suites/python/upgrade

3 Likes

Just a couple of observations on your install instructions

No 10 Should that lead with a sudo command
No 11 Should that lead with a sudo cammand
No 15 Should you have " at the end of the line

I got as far as Line 17 and the output from this line was

/usr/local/bin/python3.7 -m venv

usage: venv [-h] [–system-site-packages] [–symlinks | --copies] [–clear]
[–upgrade] [–without-pip] [–prompt PROMPT]
ENV_DIR [ENV_DIR …]
venv: error: the following arguments are required: ENV_DIR

Obviously I couldn’t continue so I restored the backup image.

Have you any ideas what may have gone wrong here. I didn’t miss any of your steps either.

look closer at line 17. There’s a dot (.) at the end of that line that needs to be there. make sure you copy the line exactly as it is.

Have you used these instructions to upgrade your Python. If so what about my observations on Lines 10,11 and 15. I thought they may have had an effect on my upgrade attempt as they wouldn’t execute without sudo and I had to remove the “ at the end of Line 15 for it to execute.

10 removes the folder created in 5 by the homeassistant user. They can delete it too.
11. removes the file created by the homeassistant user in step 2. They can also delete it.
I agree line 15 has an error. I cannot edit it now, though.

No I use HA in Docker so I’m not affected by this requirement but I have done an update last year when HA moved past v3.4 and updated successfully to v3.6 on a RPi before I moved to Docker on a NUC. If you get the chance i highly recommend making the switch to Docker. You won’t regret it.

but the instructions are very similar to the other procedures I’ve seen in this thread and others like it. I don’t think I’ve ever seen any command that would otherwise be OK without sudo being run as sudo that will then cause an issue. I’d use sudo and not be concerned about it myself.

1 Like

Well @anon34565116 to the rescue AGAIN !!!

Followed your install instructions and with a few minor mods and permission issues that I resolved, my HA venv is now up and running on Python 3.7.3 and my HA config prior to the Python upgrade is exactly as it was.

Thanks to anyone else who contributed to this post. I do understand that each user has their preferred platform for HA but I find the venv works for me and I’ve now achieved this now upgrading to Python 3.7.3

1 Like

The @anon34565116 procedure is based in hassbian, but it not for hassbian, right? I need to update raspbian. Just asking to avoid mistakes.

@ConcordGE , could you please tell us what were the minor mods and issues you faced it?

It’s an installation of Home Assistant on Raspbian in a venv as per these instructions https://www.home-assistant.io/docs/installation/raspberry-pi/

I needed to use sudo in lines 10 and 11. I needed to remove the “ at the end of line 15.

When updating to 0.94.1 I had permission issues which pip advised me that I needed to change.

Meant to say your post was very helpful.

I have the feeling, that HomeAssistant is running slower and needs more time to start with python 3.7. And HomeKit doesn‘t work. I got it running only one time after delaying HomeKit startup. Does anybody also have problems? Its running without problems with python 3.5. I use Deconz and Fritzbox integrations.

You are correct.

I was also able to upgrade the venv using those steps with the following additions:
Step 1: I had to also add libffi-dev
Steps 12/13/17: added sudo (als already mentioned above)
Step 19: had to add pip3 install mysqlclient because my Home Assistant installation uses a MySQL database for the recorder component.

3 Likes

I already had that installed from an earlier installation issue. I think it is needed for zwave?
I constructed that list after I knew I had a working solution. Sorry for the errors.

We now have a tried, tested and working method of upgrading Python in the venv.

My own Hass install is on a Raspberry Pi 3 running Raspbian Stretch Lite.

Clearly there is no need to wait for the impending release of Raspbian Buster to have access to the latest version of Python running in a virtual environment. It can be easily upgraded with a currently installed version of Hass.

Nice work @anon34565116

1 Like