How to update HA to use Python 3.7?

Configuration
1

I’m trying to update Home Assistant from an older version installed with Python 3.6 (default in Ubuntu 18.04) to use Python 3.7 (or 3.8) for the newer version, as now required.

But I’ve tried half a dozen different methods, and I’m stumped. I can easily install Python 3.7 (or 3.8). But 3.6 remains the default for Python3. All the methods I’ve tried of making Python3 point to 3.7 have a bunch of “may not work for everything” warnings associated with them, and they all end in failure for installing or updating HA.

If there any known approved way of making Home Assistant install correctly with a newer version of Python?

2

If you’ve done an altinstall then you should have python3.7 or python3.8 that you’d use in place of python3

3

I have Python 3.8, and it returns a correct response to “python3.8 --version”.

But that doesn’t help. Specifying “python3.8” instead of “python3” in the usual install or update sequence fails.

In addition there are many Home Assistant related packages under /srv/homeassistant/lib/python3.6/site-packages that I’m not sure will be found correctly even if HA would install or update with python3.8

4

Don’t change the default python version in your distro.
Just install python 3.8, create a new venv with it and install HA.

Here’s how i did it on ubuntu 18.04

# NEW 18.05.2020 with python3.8 venv as user homeassi

# With python PPA
sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa

sudo apt install python3.8 python3.8-dev python3.8-venv autoconf automake libtool
sudo mkdir /srv/ha
sudo chown -R homeassi:homeassi /srv/ha
cd /srv
python3.8 -m venv ha
cd ha
. bin/activate
python --version # should be python3.8
pip install --upgrade pip
#pip install setuptools
pip install wheel
pip install --upgrade homeassistant
pip install mysqlclient
2 Likes
5

Thanks, I think that gets me past the Python problem, as the virtual environment does come up with Python 3.8 as the default. All the commands executed except the last - mysql install failed.

The command “hass” then starts a lengthy update process with massive reams of messages and occasional obscure errors,but on completion Home Assistant does not start successfully.

ERROR: Command errored out with exit status 1: /srv/ha/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-gtxmltir/pillow/setup.py'"'"'; __file__='"'"'/tmp/pip-install-gtxmltir/pillow/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-055j3a2g/install-record.txt --single-version-externally-managed --compile --install-headers /srv/ha/include/site/python3.8/pillow Check the logs for full command output.
2020-10-10 12:02:49 ERROR (MainThread) [homeassistant.setup] Setup failed for image: Requirements for image not found: ['pillow==7.2.0'].
2020-10-10 12:02:49 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of person. Setup failed for dependencies: image
2020-10-10 12:02:49 ERROR (MainThread) [homeassistant.setup] Setup failed for person: Could not set up all dependencies.
2020-10-10 12:02:49 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of onboarding. Setup failed for dependencies: person
2020-10-10 12:02:49 ERROR (MainThread) [homeassistant.setup] Setup failed for onboarding: Could not set up all dependencies.
2020-10-10 12:02:49 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of frontend. Setup failed for dependencies: onboarding
2020-10-10 12:02:49 ERROR (MainThread) [homeassistant.setup] Setup failed for frontend: Could not set up all dependencies.
2020-10-10 12:02:49 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of safe_mode. Setup failed for dependencies: frontend
2020-10-10 12:02:49 ERROR (MainThread) [homeassistant.setup] Setup failed for safe_mode: Could not set up all dependencies.
6

mysqlclient is only needed if you use mariadb, sorry.

Not sure about pillow, maybe try

pip install pillow==7.2.0

in activated venv.

7

The other thing to consider is switching to Docker. It’s pretty straightforward, and will save you from having to do this every year :wink:

8

Doesn’t Ubuntu 20.04 come with python 3.7 ?
I know it doesn’t help, but you could install a new Ubuntu and copy the config files over ?

9

Python 3.7 is deprecated now - people need to be on 3.8 (I’d recommend 3.8.6).

2 Likes
10

My mistake.
https://wiki.ubuntu.com/FocalFossa/ReleaseNotes#Other_base_system_changes_since_18.04_LTS

When I upgrade from 3.6 to 3.7 I found it easier to just blown away the install and start again, it also cleaned up a load of things that I tried and never removed.

11

Blockquote
Not sure about pillow, maybe try

pip install pillow==7.2.0

Thanks. No good.

No matching distribution found for pillow==7.2.0

install 6.2.2 instead:
Fail, setuptools not installed

install setuptools
Try pillow 6.2.2 again
Fail: The headers or library files could not be found for zlib, a required dependency when compiling Pillow from source

I think I’m getting the message here that it’s just not practical to update an older functioning version of HA to the latest version without starting over with a clean slate, by which I mean installing the latest Ubuntu version from scratch, doing a clean install of HA, and re-creating everything I had set up. Or just give up and buy a Raspberry Pi and install a system image, re-create the customized setup from there.

The closest I’ve gotten in this entire process of starting over from scratch a dozen times was that once I managed to get the old version of HA to update while sticking with Python 3.6, and by commenting out almost everything in my configuration.yaml, HA actually launched successfully and displayed a small part of the Lovelace interface with just a warning about the old Python version. Maybe I could do that, and then re-launch the virtual environment with Python 3.8?

Update: I did that again, and once again I can get HA to update to version 0.103.6 and launch successfully using Python 3.6 by commenting almost everything out of the configuration.yaml file. But the command “python3.8 -m venv .” fails with no specific error message if I try to launch my upgraded installation that way. I was able to create a new installation of HA earlier and launch the venv with the Python 3.8, but that approach also failed further on as it became confused with the existing Home Assistant installation somehow, saying that it was trying to access the directories of the existing version and didn’t have permission.

12

It’s perfectly possible, my current HA install started on Python 3.6, to 3.7, and now 3.8. My other install went from 3.4 to 3.5 and then 3.6.

If you don’t have a supported version of Python though you can’t upgrade. What’s enabled in HA is irrelevant.

Honestly, go Docker :wink:

13

I’m not ignoring your advice Tinkerer, but we already had this discussion about Docker in another thread. I got Docker to install successfully on my Armbian version of Ubuntu, but it fails to launch:
“error msg=”[graphdriver] prior storage driver devicemapper failed: devicemapper: Error running deviceCreate (CreatePool) dm_task_run failed”. This appears to be a very complex error that many people have had over the years, not easily resolved.

I have enough to do handling HA issues without taking on Docker issues too.

14

Could not agree with you more, I used to use the VirtEnv python install but could not be bothered to upgrade this time so went over to Docker… Never going back!

1 Like
15

Then maybe it’s time to ditch your relatively obscure hardware and use something more common :wink:

16

Hence my conclusion above that HA update may not practical in circumstances where one is skipping over several versions and trying to update an existing working installation running on older hardware and OS version. Too many breaking changes and new dependencies introduced, diagnostics not sufficient to resolve the issues with a feasible amount of effort. Better to throw out the old and start over from scratch. But it doesn’t give me much confidence in investing a lot of time and effort in custom configuration of HA if this is going to happen repeatedly.

Let me ask this then: what’s the best way to completely wipe out the existing HA installation on my machine? I can just delete the existing homeassistant user and homeassistant directories. But there are also a bunch of packages under Python 3.6 that came from Home Assistant. Will they get in the way somehow?

17

If you’re using venvs and creating a fresh venv, there’s nothing to “get in the way”.

I have over a dozen venvs for different versions of Python. These are isolated installs, that’s the whole point :wink: If you want to “wipe out HA” then you leave the config folder alone, and delete the venv folder.

18

I finally got past the Pillow problem and got HA .103 to launch successfully with Python 3.8.

I had to use the Pillow update instructions I found here: https://gist.github.com/muratgozel/fdb854885d6a300004430239dd1f5cfb

It’s yet another cascading set of poorly documented dependencies. My system is especially vulnerable to stumbling over these because I originally started a little over a year ago with a “light” installation of Armbian Ubuntu 18.04, and it turns out a lot of things were missing in my build. Apparently there are an increasing number of hidden assumptions as HA updates to new versions.

19

To be fair, the route that HA is going is going to be like most of the commercial home automation platforms, while still remaining open. AFAIK, we’re going to eventually have a few different flavors like we do now (Core, Supervised, etc), but they are all going to be container-based (for reasons like the ones you are facing). There really isn’t any reason right now to use the venv version unless you are developing for HA.

To be fair, Docker on Armbian 18.04 is a mess and a pain to deal with. You’d be better off upgrading Armbian to 20.04 LTS and then installing Docker on it and then installing HA that way.

Which is to be expected as new features are developed. The docs do lay out pretty explicitly what the hardware and software requirements are for the current version of HA.

20

My situation is that I have an extensively customized installation of HA 0.92 that works well enough, but deteriorating over time as external services introduce breaking changes. I also use Node Red, which I have to say has been remarkably stable by comparison to HA.

I can now get this new version to run on the same hardware (it updated itself to 0.116 at some point during the reboots). But only about fraction of the stuff in it works at the moment, e.g., pages either won’t display or give an error message in different web browsers that worked fine with the old version, many integrations failing initially, huge number of cryptography errors in OpenSSL etc… Is Home Assistant becoming too complex as it becomes more ambitious? If I spend $100 on a current-gen Raspberry Pi and just load a pre-made build, is it going to solve all these problems? For how long?

I can’t get Docker to run successfully on Armbian Ubuntu 18.04, even though it installs successfully from the armbian-config function. Error message doesn’t give any hints as to where to look, lots of mentions in online threads, but nobody solved it. I gave up on it.