Installing/upgrading old versions - failing

Configuration
1

I’ve been running HA for a LONG time, using the instructions here for new installs, and it’s always worked as long as an appropriate version of python was present, but it seems that no longer works. I’ve been trying to upgrade my mother’s HA instance, which is running on the REALLY old version 0.76.2 on python 3.6, but NOTHING I’ve done has worked to get it upgraded. I’ve tried everything under the sun that I can think of an everything has failed one way or another. From what I could find, python 3.6 support was removed around 0.103.x, so in theory, I should be able to upgrade to some version prior to that while running python3.6, right? Didn’t work. I tried doing a fresh install on a fresh instance of Ubuntu 20.04 LTS with Python 3.8 installed, of several different versions of HA higher than 0.76.0 and lower than 0.110.0, and all failed. There was at least one failure that complained about a version of a component that was installed by the ‘pip’ install process being too new.

Would I be correct to assume that HA 0.110.7 should install and work just fine on Ubuntu 20.04 LTS, with Python 3.8.10? Assuming ‘yes’, then why did it fail? Here’s the output from that clean install:

Building wheels for collected packages: ciso8601, pyyaml, voluptuous-serialize, aiohttp, voluptuous, ruamel.yaml, python-slugify
  Building wheel for ciso8601 (setup.py) ... done
  Created wheel for ciso8601: filename=ciso8601-2.1.3-cp38-cp38-linux_x86_64.whl size=30187 sha256=c3e4fe6640ad1a418fd3d5f4715a93bb636bd8729ea4bfdf62d977eb23dde567
  Stored in directory: /home/homeassistant/.cache/pip/wheels/55/c3/4c/7f1b3249545f80aa40b0ff3b4029e676e7521559fb9b5aab14
  Building wheel for pyyaml (setup.py) ... done
  Created wheel for pyyaml: filename=PyYAML-5.3.1-cp38-cp38-linux_x86_64.whl size=44618 sha256=ebbd679f1b2a86d05b4539c36e2b89bbea46e1f70b14a51a6caffb6af4956065
  Stored in directory: /home/homeassistant/.cache/pip/wheels/13/90/db/290ab3a34f2ef0b5a0f89235dc2d40fea83e77de84ed2dc05c
  Building wheel for voluptuous-serialize (setup.py) ... done
  Created wheel for voluptuous-serialize: filename=voluptuous_serialize-2.3.0-py3-none-any.whl size=2521 sha256=4a30992c3b71ee47903b4adf42066190fa08ac821e1f1d8cfeea442bc120eb21
  Stored in directory: /home/homeassistant/.cache/pip/wheels/c3/d0/10/3e8ce472b62771ed4a9e291af5fed998c659a5fa70da8eff5a
  Building wheel for aiohttp (PEP 517) ... done
  Created wheel for aiohttp: filename=aiohttp-3.6.1-cp38-cp38-linux_x86_64.whl size=1460671 sha256=926211732b33e66499cf877c6a654bfe2cb8e5fb7f1e2c8a943e28f6bafc5d22
  Stored in directory: /home/homeassistant/.cache/pip/wheels/9b/8a/35/3b15dd96a7c39f21885ec32f16e5d93948eb5b0ac333018ede
  Building wheel for voluptuous (setup.py) ... done
  Created wheel for voluptuous: filename=voluptuous-0.11.7-py3-none-any.whl size=28747 sha256=3247c08091eedeac25e8b8c50efac94571cdbea8dfa3c0a4085f1cd36ece7658
  Stored in directory: /home/homeassistant/.cache/pip/wheels/9b/c9/0e/238ffb22248932b2136eb125c355f03666702191e0099829f7
  Building wheel for ruamel.yaml (setup.py) ... done
  Created wheel for ruamel.yaml: filename=ruamel.yaml-0.15.100-cp38-cp38-linux_x86_64.whl size=864575 sha256=1ab704cba406f8a2b178c107b57be2512bcdc62fa917428c310d49d0f89cfe43
  Stored in directory: /home/homeassistant/.cache/pip/wheels/83/bb/c8/84a46029883fd202fda2e3d0a579eb30641fecb325b9907fb0
  Building wheel for python-slugify (setup.py) ... done
  Created wheel for python-slugify: filename=python_slugify-4.0.0-py2.py3-none-any.whl size=5487 sha256=93a8b36769be7212de81d5a1ab0f75e191db633da847ad876fa669f40afc20e7
  Stored in directory: /home/homeassistant/.cache/pip/wheels/f7/2d/24/7c629dd27a06a60618e6af1004e5667d4e38a1e914493ba72d
Successfully built ciso8601 pyyaml voluptuous-serialize aiohttp voluptuous ruamel.yaml python-slugify
Installing collected packages: ciso8601, zipp, importlib-metadata, pytz, astral, async-timeout, pyyaml, certifi, voluptuous, voluptuous-serialize, multidict, attrs, chardet, idna, yarl, aiohttp, pycparser, cffi, six, cryptography, MarkupSafe, jinja2, bcrypt, urllib3, requests, ruamel.yaml, PyJWT, text-unidecode, python-slugify, homeassistant
Successfully installed MarkupSafe-2.1.1 PyJWT-1.7.1 aiohttp-3.6.1 astral-1.10.1 async-timeout-3.0.1 attrs-19.3.0 bcrypt-3.1.7 certifi-2022.9.24 cffi-1.15.1 chardet-3.0.4 ciso8601-2.1.3 cryptography-2.9.2 homeassistant-0.110.7 idna-2.10 importlib-metadata-1.6.0 jinja2-3.1.2 multidict-4.7.6 pycparser-2.21 python-slugify-4.0.0 pytz-2022.6 pyyaml-5.3.1 requests-2.23.0 ruamel.yaml-0.15.100 six-1.16.0 text-unidecode-1.3 urllib3-1.25.11 voluptuous-0.11.7 voluptuous-serialize-2.3.0 yarl-1.8.1 zipp-3.11.0

And then when I tried to run ‘hass’ to finish the install (leaving the ‘python3 -V’ verification after the failure in there):

(homeassistant) homeassistant@waltershass:/srv/homeassistant$ hass
Traceback (most recent call last):
  File "/srv/homeassistant/bin/hass", line 8, in <module>
    sys.exit(main())
  File "/srv/homeassistant/lib/python3.8/site-packages/homeassistant/__main__.py", line 337, in main
    args = get_arguments()
  File "/srv/homeassistant/lib/python3.8/site-packages/homeassistant/__main__.py", line 84, in get_arguments
    import homeassistant.config as config_util
  File "/srv/homeassistant/lib/python3.8/site-packages/homeassistant/config.py", line 48, in <module>
    import homeassistant.helpers.config_validation as cv
  File "/srv/homeassistant/lib/python3.8/site-packages/homeassistant/helpers/config_validation.py", line 74, in <module>
    from homeassistant.helpers import template as template_helper
  File "/srv/homeassistant/lib/python3.8/site-packages/homeassistant/helpers/template.py", line 14, in <module>
    from jinja2 import contextfilter, contextfunction
ImportError: cannot import name 'contextfilter' from 'jinja2' (/srv/homeassistant/lib/python3.8/site-packages/jinja2/__init__.py)
(homeassistant) homeassistant@waltershass:/srv/homeassistant$ python3 -V
Python 3.8.10
(homeassistant) homeassistant@waltershass:/srv/homeassistant$

This is driving me nuts - I’ve done this several times in the past, and I’ve never had issues when following this venv install or this upgrade.

This is kicking my butt. I should be able to start with a fresh Ubuntu 20.04 install running Python 3.8.10, and run through the steps provided in the previously linked Linux Venv install instructions, running the final install command ‘pip3 install homeassistant==0.110.7’ and have a functioning Home Assistant v0.110.7 install - this hasn’t been the case this time around. It’s like all of the older installers have been broken.

Why are these older versions not working as the used to? Have they actually been broken? If so, they should either be fixed or removed.

As an alternate upgrade, what’s the newest version I can install clean, assuming it’ll work, and plant all of the 0.76.2 config files and DB after the initial clean install and have it upgrade cleanly and work?

2

You have learned the beauty of docker (I am not saying docker is perfect).

3

errrr… nope. I’ve tried docker and I have one thing installed using docker that works. Everything else I’ve tried has failed, including trying to get ZWave JS UI running under docker. Following the instructions on their github to a T. Failed EVERY time. Finally got it working using snap. I hate docker. Docker is the ‘new hotness bandwagon’ that everyone is jumping on to try and solve something that doesn’t need solving - virtualization being what it is today, I’d rather have a bunch of linux VMs, each for a specific purpose, than have one running a bunch of ‘docker containers’. I’ve been using HA for a LONG time. Probably a decade or more, always on a base linux install, always installing or upgrading using the previously linked ‘how-to’s’, never having an issue until now, and again, at least one of the issues I found seemed to be related to something the ‘pip’ install process installed into the venv being incompatible.

If I do ever migrate to the new HA experience as it is today using docker, the upside is it’ll be hands off. But if I need to get my hands dirty under the hood, docker be gone.

The fact is earlier this year (or maybe it was late last year - I don’t recall), I took my HA instance from whatever OLD instance I was running, possibly newer than 0.76, but possibly not, to 2021.10.0, the newest I could run on the ‘old’ ZWave, and didn’t have these issues.

4

I used to be like that, and i see I joined this forum at roughly the same time as you :slight_smile:

However I am after an easier life, and I use a supervised install, which for me is great in that I have OS access and an always up to date python, HA, addons and everything else I need. Maybe I just had good docker-luck.

1 Like
5

I will admit, based on playing with test instances, the ‘supervised install’ does seem to work well… but I’ve been using HA so long, and learning to work with the YAML files directly for everything, that’s what I’ve gotten used to and am comfortable with. This shift to where we have no access under the hood ‘bothers’ me. Now as far as docker is concerned, in the case of HA, should I get to a point where there’s a way to easily migrate my install seamlessly to that, I can’t say I’d be opposed, as at that point, all the docker nonsense should be 100% hands off for me. As long as I don’t need to get under the hood and that’s all handled by HASS, I can live with that. But if I need to get my hands dirty ‘under the hood’, I want no part of docker.

6

and to add in… I just tried a fresh install of 2021.10.0, the same version I’m running on my Ubuntu 20.04 install, running Python 3.8.10, the EXACT same python version that’s on the 'new install, and this is the result:

TypeError: deprecated() got an unexpected keyword argument 'name'
2022-11-27 04:04:08 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 422, in _handle_request
    resp = await self._request_handler(request)
  File "/srv/homeassistant/lib/python3.8/site-packages/aiohttp/web_app.py", line 499, in _handle
    resp = await handler(request)
  File "/srv/homeassistant/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 119, in impl
    return await handler(request)
  File "/srv/homeassistant/lib/python3.8/site-packages/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/srv/homeassistant/lib/python3.8/site-packages/homeassistant/components/http/forwarded.py", line 80, in forwarded_middleware
    from hass_nabucasa import (  # pylint: disable=import-outside-toplevel
  File "/srv/homeassistant/lib/python3.8/site-packages/hass_nabucasa/__init__.py", line 15, in <module>
    from .auth import CognitoAuth
  File "/srv/homeassistant/lib/python3.8/site-packages/hass_nabucasa/auth.py", line 7, in <module>
    import boto3
  File "/srv/homeassistant/lib/python3.8/site-packages/boto3/__init__.py", line 17, in <module>
    from boto3.session import Session
  File "/srv/homeassistant/lib/python3.8/site-packages/boto3/session.py", line 17, in <module>
    import botocore.session
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/session.py", line 26, in <module>
    import botocore.client
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/client.py", line 15, in <module>
    from botocore import waiter, xform_name
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/waiter.py", line 18, in <module>
    from botocore.docs.docstring import WaiterDocstring
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/docs/__init__.py", line 15, in <module>
    from botocore.docs.service import ServiceDocumenter
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/docs/service.py", line 14, in <module>
    from botocore.docs.client import ClientDocumenter, ClientExceptionsDocumenter
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/docs/client.py", line 14, in <module>
    from botocore.docs.example import ResponseExampleDocumenter
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/docs/example.py", line 13, in <module>
    from botocore.docs.shape import ShapeDocumenter
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/docs/shape.py", line 19, in <module>
    from botocore.utils import is_json_value_header
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/utils.py", line 37, in <module>
    import botocore.httpsession
  File "/srv/homeassistant/lib/python3.8/site-packages/botocore/httpsession.py", line 46, in <module>
    from urllib3.contrib.pyopenssl import (
  File "/srv/homeassistant/lib/python3.8/site-packages/urllib3/contrib/pyopenssl.py", line 50, in <module>
    import OpenSSL.crypto
  File "/srv/homeassistant/lib/python3.8/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import SSL, crypto
  File "/srv/homeassistant/lib/python3.8/site-packages/OpenSSL/SSL.py", line 19, in <module>
    from OpenSSL.crypto import (
  File "/srv/homeassistant/lib/python3.8/site-packages/OpenSSL/crypto.py", line 3224, in <module>
    utils.deprecated(
TypeError: deprecated() got an unexpected keyword argument 'name'

Something HAS to have changed… I’ve done this before without issue, and that aside, I should be able to start with a fresh Ubuntu 20.04 LTS install running Python 3.8 by default, run through the install instructions for a venv, with the final install command being ‘pip3 install homeassistant==2021.10.0’ (or even ‘pip3 install homeassistant==0.110.7’) and have a working HA instance. And that hasn’t happened. It’s barfed every time. And venvs are ‘kind of’ like docker, but less convoluted. I feel like these older methods have been intentionally broken… I do SERIOUSLY hope that’s not the case, but considering things I’ve done in the past that have worked are no longer working, that’s how it ‘feels’.

7

When you install ha, pip will pull in a ton of external dependencies, most of which are maintained by people entirely unrelated to HA. Some of these older version packages might not be maintained anymore. The whole Python ecosystem is extremely fluid, things change all the time, different Python versions are mutually incompatible, etc.

Coming from the C++ world myself, I find this whole concept extremely weird and I don’t understand how people can productively work that way. But it’s the way it is. The only safe way to run old HA versions is to copy the entire venv, plus the exact Python version, plus dependencies (kinda what docker does, in a way).

Why not simply update Python and install a newer HA versuon though (in a venv) ? You can have several Python versions installed side by side.

8

You’d think, with how venv’s are supposed to work, that the install process inside one would specify the specific versions required to work, not just ‘this’ package and ‘that’ package, where if they’re updated, could break the process.

I’ve tried pretty much everything at my disposal, the last being I’ve cloned my HA server, running 2021.10.0, and once the clone has completed its vMotion out to my mothers network (the beauty of running things in VMWare, not on RasPis), hopefully, the jump from 0.76.2 to 2021.10.0, with regards to replacing all the config files in /home/homeassistant/.homeassistant’, won’t be too much of a jump… The main goal here being to migrate to ZWave JS, which seems to be much more robust and error tolerant than the USBIP setup I’ve been using… From there, who knows.

Edit: Updating Python and installing a newer version of HA in a venv is functionally the same as a brand spanking new Ubuntu 20.04 install with Python 3.8 and trying to run a new (old) install of HA in a new venv - it’s broken. Probably due to the process pulling this and that package, not specific versions of those packages, as previously mentioned.

9

They do test versions. But there’s a lot of things that can go wrong when trying to install a really old version. Dependency resolving rules might have been incorrectly written, leading to a newer, incompatible version of a dependency to be installed. Old dependencies might simply not be available anymore. Old versions are not tested. And pip/pypi is a community managed repository. There’s no guarantee that old versions will be maintained ad inifinitum.

Huh ? No, it’s absolutely not the same…

I don’t get this fixation with installing an old version from scratch. If you have an already running install and you don’t update it to avoid breaking changes, sure, I get that (I do it myself). But why install an obsolete, unmaintained release running on an old version of Python that is not supported anymore ? Doesn’t make a lot of sense to me. It’s just asking for trouble. Update Python, install the current release and be done with it.

1 Like
10

I apparently didn’t mention it here, but the reason being to migrate ZWave networks from the dead legacy Open ZWave to Zwave JS. Can’t go past 2021.10.0 before migrating the zwave network. If I was already using ZWaveJS or didn’t have ZWave, then yes, there probably would be no reason to not go straight to the newest version.

11

Then what’s stopping you from migrating it ? It’s easy, there are lots of good information about the process on here and zwavejs is so much better than the old zwave. It really feels like you are having all this technical trouble because of limits you chose to impose onto yourself.

And yes, zwavejs also works without docker.

12

As far as I know, ZWave JS doesn’t work on 0.76.2 and OpenZwave doesn’t work on the latest build, so in order to actually migrate ZWave, I need to get to 2021.10.0, which is what MY HA is on, or possibly some build prior to that which includes both OpenZWave and ZWave JS and the migration tool, but I need to get my mothers instance and my sisters instance up 2021.10.0 so I can do the migration on theirs. Once Zwave has been migrated to ZWave JS, THEN I can upgrade to the latest version.

If dumping the 0.76.2 config files onto the clone of my 2021.10.0 instance works and brings it up functional on that build, then my problem should be solved, I should be able to migrate ZWave and go from there.

And I wouldn’t say I ‘imposed limits on myself’. I’ve been using HA for so long, when I started, venvs where the ONLY way to do it, and I’ve been upgrading it periodically as time went on, usually when there’s something new that was added that I was interested in. And I got pretty comfortable working with the yaml files over the years. To be honest, I don’t think there’s a whole lot in the newer versions that interests me - 2021.10 works fine for me, and I think migrating to ZWaveJS will be a huge improvement over the USBIP setup I’m using, which works, but it’s VERY sensitive to even the slightest communication hiccup. When my UPS switches to battery, even briefly, even though my RPi hosting my ZStick is powered via POE and never went down, there was enough of an impact to disrupt the connection and break ZWave until the RPi is rebooted and the USBIP and HA services on my HA VM are restarted. ZWave JS should be a HUGE improvement in stability over that. I may go newer, but one step at a time. I want to get the other two instances to 2021.10.0 so I can migrate them to ZWAveJS, then we’ll see where I decide to go.

13

Okay then. You’re trying really hard to find reasons not to update to a new supported release, but to install some outdated year old version instead. I don’t know if anyone can help you with that. I certainly can’t. Good luck.

14

After all the config issues that ran home assistant on my mac, went and bought Blue Odroid N2+ and after about year it is all good and stable. Now core update and supervisor updates are stuck and keep on running… What happened here.
Homeassistant had so many issues and I thought buying dedicated hardware will fix all these issues.
I like some one to answer these issues since I could not get out of update wheel.

15

So what version of core and supervisor are you running at present? What are you trying to update to? What error messages are you getting?

16

Restoring a backup is not possible right now because the system is in startup state.

17

Home Assistant Core

Installed version

2022.11.1

Latest version

2022.11.4

Home Assistant Supervisor

Installed version

2022.11.0

Latest version

2022.11.2

18

Also I can see all working with OVERVEW (old versions) but update on core and supervisor are stuck on a wheel

19

So just to be as abundantly clear as humanly possible: the reason I want to get to an older version than rather than the latest is for one reason: To migrate the ZWave Networks from OpenZWave to ZWAveJS. As far as I know, in order to do that, HA needs to have OpenZWave installed and the network running (and then stopped by the process) at the time of the migration. Once that’s complete and OpenZwave is no longer needed, it’s no problem to go right to the latest version or whatever.

That being said, I discovered that I was wrong in thinking that 2021.10.1 was the last version that had openzwave - that wasn’t removed until 2022.4. After finding that, I tried to upgrade to 2022.3.8, but got met with an ambiguous message saying something to the effect of ‘no such version’. Come to figure out that Python3.8 support was dropped after 2021.12.10, so I set up a new Ubuntu 20.04machine, installed Python 3.9 and tried to install 2021.12.10 clean, but that blew up. Various other versions between 2021.12.10 and 2022.3.8 all blew up in various fashions, at least one of them complaining about a missing package that I noted was installed with the 2021.3.8 version. So to cut to the chase, what I ended up able to do was install 2022.3.8 clean, then ‘upgrade’ to 2021.12.10, and I then had a functioning setup on Ubuntu 20.04, Python3.9 and HA 2021.12.10 (more on why in a second).

Through a lot of trial and error, chasing my tail, etc, I learned a bit. Suffice to say the install scripts and instructions for core are broken all over. I can work with the comment about old versions and dependencies not working when we’re talking about a four year old version, but anything in the last year shouldn’t be so far out as to have issues, especially when one of the things that was complained about was an older version of HA complaining about a missing package that a NEWER version (only a MONTH newer) of HA installed. To start with, when I tried to install the depreciated OpenZwave, it was failing every time. Turns out, from a thread I found, someone posted for another person that had a similar issue, they stated that they ‘didn’t follow the instructions and install all of the dependencies’ as installing ‘libudev-dev’ fixed their issue and also mine building OpenZWave - libudev-dev is NOT part of the instructions.

Anyway, on to why, once I found out that 2022.3.8 still had OpenZwave and could do the migration, I still ended up needing to go back… When I got a functional 2022.3.8 install and finally got OpenZWave installed and working, got it hooked up to my ZStick, etc, I stopped HA, deleted all the config files and ozw_config files, uploaded my backups, fixed permissions, and started HA, aaaand… One third of my devices were ‘orphaned’. Shut that down and bring up my original instance and everything’s perfect, so it has nothing to do with my actual ZWave network or ZStick. I figured maybe something got missed in the jump from 2021.12.10 to 2022.3.8 or something, so I jump through all the flaming hoops to get a version of 2021.12.10 that I could upgrade into the 2022 realm, get it working, all ZWave devices are perfect, and upgrade the 2021.12.10 instance to 2022.2.1 and BAM, same thing. One third of my devices are now orphaned.

So now I have two issues - the ZWave migration seems to be broken in all of the 2021 versions. If someone has a suggestion on how to make that work and successfully migrate to ZWaveJS (Or if I’m wrong in what I’ve read that suggests that to MIGRATE the ZWave network, both must be present and in fact jumping to the latest version PRIOR to migrating OpenZWave to ZWaveJS will work…), that would (or at least SHOULD) negate the second issue, which is upgrading past 2021.12.10 orphans a third of my devices.

So any takers on how to make the ZWave migration actually work and/or why upgrading into the 2022 builds orphans a third of my devices?

20

Wow, you are definitely making things way harder on yourself by running a Core version of HA in a venv instead of using a “newfangled” install version.

I will give you the same advice I give every other user who asks about updating from an old version of HA to latest (even if it’s just a couple of versions and not years old versions that you are trying)…

update one version at a time and fix everything that’s broken in that update before moving on to the next version and doing the same there (etc, etc). So go from 0.76.x to 0.77.x and fix stuff. then go to 0.78.x and fix stuff…

ity may be monotonous but it seems you should get better results than you are now by trying to quantum leap thru several versions and continuing to flail around trying to fix all the issues you see. It might be less time that way than it seems it is taking you now to get where you want to be.

2 Likes