Please stop writing these essays, because it really isn’t helping. You got yourself into this situation and can’t reasonably expect things to be backwards compatible with things more than a year old. The issue with dependencies has been explained to you. You’ve been given good advice. I totally agree with Finity’s advice: Don’t take such a big jump and upgrade bit by bit. Or do as Nick says: wipe and start over.
Wow. Provide a lot of information, get whined at that it’s too much. Provide no information beyond ‘it doesn’t work’, get whined at that there’s no information to go off. There is no winning here.
I did not ‘get myself into’ any situation. I’ve been using HA as ‘core’ since before it was ‘core’ and for longer than you’ve been a member here, and I managed to work my way through the struggles caused by the installers for every version that I tried after 2021.10.0 and before 2022.3.x - NONE that I tried would install. They all blew up after running ‘hass’.
So I have to repeat myself: Simply wipe and start over DOES NOT WORK. I tried that with 2021.10, 2021.12.10, several 2022.2.x versions and the only version I was able to ‘wipe and install’ without the HA installer blowing up was 2022.3.8. The ONLY way I was able to do a ‘wipe and install’ to get to the version I was on was to install 2022.3.8 and downgrade to 2021.12.10.
Since it apparently wasn’t abundantly clear to some, I’ve worked my way through the previous issues with getting HA upgraded, OS dependencies and such and everything is now on a version of Ubuntu and Python that supports being upgraded into the 2022 builds, and that works, even though clean installs of the same builds don’t work.
To repeat another point: the problem now is:
Now the two issues are why doesn’t the ZWave migration tool work - can it be fixed? If it can be made to work right in 2021.12.10 and get migrated over to ZWaveJS, I’m fine with that. I was going up to 2022 releases to see if somewhere along the way, the migration tool was fixed and would complete successfully. And why, when I upgrade from 2021.12.10 to 2022.2.1, the next release, does HA orphan a third of my devices? In all of my years using ZWave with HA, dating back probably to the 0.10 release timeframe, I’ve never seen that happen.
These above are the only issues now.
Or maybe I’ll just have mods lock this thread and start a new one specifically for those two issues.
I have to add: Posts like yours are the kind of thing that discourages people from posting what they found and how they resolved things. My ‘essays that don’t help’, as you so eloquently put it, are largely the issues I ran into, what I tried to get around them, and finally, the solution that put me into a ‘happy place’ in terms of Ubuntu version, Python Version, HA version, and ability to upgrade HA properly using the install --upgrade commands. That was resolved by myself with no help here other than ‘wipe and reload’ comments, which doesn’t seem to work for any version prior to to 2022.3.8, and posted about before you posted. The remaining issue or two has nothing to do with that. If my ‘essays’ help even ONE person, that’s good enough for me.
Mods, please lock this thread - it’s run its course, I’ve posted my trials, tribulations and resolutions, which some don’t appreciate, but there’s no reason to take this thread further. Hopefully my posts on the issues I’ve encountered and how I’ve overcome them will benefit someone at some point.
The Supervisor and OS do indeed update automatically (tho you can now opt out of Supervisor auto-updates).
But HA itself in any install method never auto-updates.
And Container never auto-updates anything either.
I understand that you “distrust” Docker (for some reason…) but I can tell you with authority that that method of installation is THE HA sweet spot between a Supervised type of install and a venv install.
I usewd to run a venv install (just like everybody else since that was the only way at the time) but after a couple of rebuilds due to Python update requirements, etc I went the Docker route and never looked back.
Docker isn’t that hard once you get a couple of basic concepts down and I’ve literally NEVER had to deal with any dependencies or Python or anything else that you are struggling with here. It literally just works. And updates are as easy as a short CLI command or even easier if you install Portainer (which also runs in Docker) and use it to manage your containers.
And with a Container install you get full control over your underlying OS (for better or worse - I think better), You can install pretty much anything else on the machine you want and you can run any number of containers that give functionally equivalent capabilities as Supervisor add-ons.
I really can’t talk a Container install up too much.
And let’s think for just a second…
you are saying that all of the recent venv install methods are broken. I have my doubts about that but let’s say for the sake of argument it’s correct.
If for some reason HA stopped supporting a pure venv install method, would you decide that HA was no longer for you and move to something else or would you just bite the bullet and go with a Supervised or Container type of install method? And learning Docker in the process?
If the answer is biting the bullet and updating then why not just bite that same bullet now and save yourself literally all of the aggravation you are experiencing right now?
Finity, WHY DO I HAVE TO REPEAT MYSELF OVER AND OVER??? The reason I’m in the ‘version range’ I’m in now is to migrate my darn ZWave networks. I’m tired of repeating it, and no one has refuted it - as far as I know, in order to MIGARATE it, OpenZWave must be installed and functioning to migrate to ZWaveJS. If this is correct, this CANNOT happen past 2022.3.8 as OpenZwave was removed from HA in 2022.4. I DON’T want to rebuild my Zwave related things from scratch.
It’s been said to not jump large numbers of versions, which although I don’t have a whole lot going on to break, really just ZWave, I’m ok with that - but I now have a perfectly functioning 2021.12.10 install running on Python3.9.5 and Ubuntu 20.04 that I can upgrade into the 2022 builds (unlike the original 2021.10.0 server running Python 3.8 that could only go to 2021.12.10), but when I take that next step into the first 2022 build, HA orphans a third of my devices. Revert to my pre-upgrade snapshot and everything’s perfect, so it’s not my devices or ZStick, it’s HA doing that, and I’ve never had that happen since I started with HA eons ago.
Once it’s migrated successfully from OpenZWave to ZWaveJS, I can either stay where I am, upgrade to the latest ‘core’ version, migrate to the 'supervised version, whatever, anything is possible, not committing to anything there, but I’ve worked through the issues surrounding the OS/python etc, so that’s not an issue now.
You say ‘you are saying that all of the recent venv install methods are broken. I have my doubts about that but let’s say for the sake of argument it’s correct.’. Well… I don’t know what to tell you. I Built a brand new Ubuntu VM (I only continued with Ubuntu because I couldn’t get USBIP working on Debian - once I’ve successfully migrated to ZWaveJS, I plan on moving to Debian 11 - it seems to be cleaner), installed all HA prerequisites (including libudev-dev which is needed for OpenZWave to install), and Python 3.9, following the instructions (for a refresher):
sudo -u homeassistant -H -s
cd /srv/homeassistant
python3 -m venv .
source bin/activate
python3 -m pip install wheelfollowed by ‘pip3 install homeassistant==(insert version)’
The same procedure used since the dawn of Home Assistant.
EVERY one that I tried, except for 2022.3.8 blew up. Several had dependency issues after running ‘pip3 install…’ but before running ‘hass’, but while re-running ‘pip3 install’ a few times removed the ‘bad’ package and installed the right one (all posted above), they all blew up after running ‘hass’ (some also posted above). Not sure what else to say. I’m not a linux expert, or a HA expert. but I can follow instructions. The same instructions that I’ve used for installing or updating HA since I started using it back somewhere in the 0.10 realm. Never had issues with it until now. And when everything else being the same, when running ‘pip3 install homeassistant==2022.2.1’ followed by ‘hass’ blows up, yet running ‘pip3 install homeassistant==2022.3.8’ followed by ‘hass’ works… Not really sure what to tell you.
Then just do the migration manually and don’t use the migration tool.
I’ve heard that the migration tool is hit or miss (mostly miss it seems if I recall).
And you don’t have to have anything running at all from your old install to run zwavejs. You can completely wipe out your Openzwave v1.4 integration and run zwavejs using your existing zwave controller. The only thing you lose at that point are the entity_ids of your entities which you will need to manually change to match the old ones. Or don’t. It’s up to you.
But without docker you are going to have to install a bare bones zwavejs server on your hardware. And again that brings with it the maintenance issues with dependencies, etc.
running a zwavejs server in Docker using zwavejs-ui is dead simple.
But you don’t want to hear that.
Sorry, but please don’t spout the ‘docker is dead simple and the best ever’ nonsense. Docker sucks, and It introduces complexity that doesn’t need to be there (the most notable thing I’ve seen is what seems to be network virtualization - that has its place, but this isn’t one of those places). The only thing that I’ve implemented with docker that actually works is Bitwarden. I originally tried to learn and embrace docker (and NOT just related to HA) - in theory, it seems like a great idea, but (almost) everything was met with failure. It’s NOT ‘dead simple’. Venvs are far simpler. Dedicated Linux VMs for specific tasks are far simpler. I originally tried to get ZWaveJS UI working using docker and it failed, every time, USING THEIR INSTRUCTIONS! I finally got it working perfectly using snap, and while I couldn’t get it working initially, not due to visible errors or failures, unlike docker, a VERY helpful post by a user here was able to fill in the blanks and I was able to get it working on my RPi.
I have no desire to go back to having HA on a RPi (Or HA Yellow), which is arguably the direction HA is aiming and the newer methods are meant for. I only want my ZStick on a RPi as a ‘relatively’ dumb device that HA, running on a VMWare VM, communicates with to control the ZWave devices (which is how it is now except OpenZWave is using a USBIP service to communicate with the ZStick). HA as a Venv on a VMWare Virtual machine is far superior (IMO - outside of the dependency issues with the REALLY old 0. versions, which is ‘technically’ understandable, except with the 2022.2 version that apparently installed the 5. version of a package, but was looking for a 6. version, which was installed by the 2022.3 version of HA, I didn’t have dependency issues with versions after 2021.10.0 that weren’t corrected by re-running the install command prior to running the ‘hass’ command, although I think the scripts should be requesting the required version, not ‘whatever version’, and erroring if the requested version wasn’t available) in that it’s easier to reliably back up, and when making a major change or upgrade, I can take a snapshot of the VM and if it goes sideways, a few clicks and 30 seconds or less and it’s back where it was fully functional. And you have file level access, so in addition to the snapshots, you have the actual config files and DB that you can make copies of. I doubt the ‘new ways’ can beat reverting that quick. But that’s where the ‘other methods’ benefit a lot of users - While I can likely recover from a failed update far faster than any other ‘supervised’ HA implementation could, I suspect most people don’t have the infrastructure I have in place, And as long as they have some sort of backup, if it takes them 15, 30 minutes or whatever, as long as they can recover, they’re happy. What works for one does not necessarily work for another. Whether I stick with venvs or end up moving to one of the other methods is TBD, AFTER I get my ZWave network(s) migrated to ZWaveJS. Either way, whatever I do, HA will remain a VMWare VM.
Lastly, as far as ‘Then just do the migration manually and don’t use the migration tool.’, if HA is going to make the decision to make such a paradigm shift from one method to another for something as core to home automation as ZWave, they REALLY should have a tool to migrate that properly that actually works, not be like ‘well, it may work or it may not, you might have to do it manually’. That’s BAD. The move way back from one method to another within OpenZWave related to how entities were stored was, while annoying, easier than this migration. I don’t have a huge ZWave network, but even so, manually migrating it would/will be a ROYAL pain, but if someone with hundreds of devices (or more) couldn’t migrate them and had to do it manually, I could understand them launching everything HomeAssistant into orbit.
Anyway, as I already said, the main point of this thread has run its course. I’ve gotten around all the original issues related to this thread, posted my issues, what I attempted, and how I resolved them, which some people didn’t like, but hopefully my issues and resolutions will help at least one person at some point. Lets let it die. Or maybe a mod can lock it as I previously requested.
1 Like
OK, then good luck to ya.
Ubuntu is not a supported OS. You’re on your own.
You have no right to get angry at people trying to help.
Please read this: How to help us help you - or How to ask a good question.
The problem with your essays is that it makes it hard to help you. It’s on you to communicate well. Don’t shout, please.
You keep rejecting advice, which makes is hard to continue help – that’s why some are no aborting this thread.
So what exactly did you do? That statement lacks information. Did you just do a new install and copied existing configs? That certainly won’t work, given the amount of time that has expired. Did you also start with an empty DB? The best route here would be to add configs back bit by bit. Perhaps also start with a vanilla install with no integrations, and add integrations one by one. You see how I didn’t need a 1000 words to say just this?
Anyway, that’s the last from me here.
Because what you are repeating over and over (in very long convoluted and not-to-the-point ways) doesn’t make a lot of sense, I’m afraid.
No it does not. How hard is it to understand that ? Your zwave network is stored in the controller, on a special memory area in the dongle. You will not lose it when you migrate manually. Install a recent version of HA Core if you want (I just installed Core 2022.11.5 yesterday without any major issues at all, besides having to recompile numpy for my oldish glibc), run zwavejsUI and reinterview the nodes. Have a coffee, wait, wake up a few battery powered nodes as needed and you’ll have your functioning mesh back. Without any old versions or OpenZWave or whatever other stuff you were on about.
You may have to rename a few entities by hand. Considering the amount of time you poured into this thread, that shouldn’t be a big deal though…
1 Like
I know the network is stored on the stick. But the device/entity IDs that HA references (other than the ID number that the stick provides to HA) are not - that’s HA, and isn’t the ‘migration tool’ supposed to migrate the old IDs to the new IDs or something along those lines? Anyway, I’ve finally gotten everything moved to ZWaveJS and OpenZWave is a thing of the past, and I’ve brought it up to 2022.5.5 so far. I ended up doing the ZWave migration manually. Doing so broke everything everywhere and I had to edit literally every aspect of HA to fix it - Automations, Groups, cards, devices, entities - you name it. I was hoping to avoid that, but in the end it was unavoidable. Almost made my previous exploits fun! At least those issues were things that I was able to figure out how to work through, even if they were annoying. This was just annoying and tedious. Took me about three hours of going through everything to fix all references to device and entity IDs.
The thing that was really frustrating for me, other than most ‘help’ was saying ‘use docker’ or similar responses, was the simple thing that was said several times (back up, wipe and start a clean install) that should work didn’t.
Why is it that I could install all prerequisites per the install instructions:
apt-get install -y python3.9 python3.9-dev python3.9-venv python3-pip bluez libffi-dev libssl-dev libudev-dev libjpeg-dev zlib1g-dev autoconf build-essential libopenjp2-7 libtiff5 libturbojpeg0-dev tzdataI substituted Python3 from the instructions for Python3.9 and made it system default so the venv would build with 3.9 - I read that you should be able to do ‘python3.9 -m venv .’ and have it build with 3.9, but when I tried that and ran python3 -V, it returned 3.8.
Run prep commands:
useradd -rm homeassistant
mkdir /srv/homeassistant
chown homeassistant:homeassistant /srv/homeassistantI then shut down the machine and took a snapshot I could revert to, and then continue and run:
sudo -u homeassistant -H -s
cd /srv/homeassistant
python3 -m venv .
source bin/activate
python3 -m pip install wheelFollowed by ‘pip3 install homeassistant==2022.2.9’, it would blow up after running ‘hass’, but when I reverted to the snapshot (technically, I could also have just deleted /srv/homeassisant and recreated it) ran the same final prep commands, followed by ‘pip3 install homeassistant==2022.3.8’, it worked 100% correct. And then on top of that, I could run:
sudo -u homeassistant -H -s
cd /srv/homeassistant
source /srv/homeassistant/bin/activate
pip3 install --upgrade homeassistant==2022.2.9And it would downgrade to that version no problem. I went all the way back to 2021.12.10 that way to match the version I was on but running Python 3.9 so that I could upgrade my install and configs gradually and gracefully into the 2022 builds. But installing 2021.12.10 clean on a new venv running Python 3.9? Nope. 
I have to assume that since Python 3.8 was dropped in the first 2022 build, that Python 3.9 support was introduced a ‘bit’ before 2021.12.10.
At any rate, it’s all over and done with. OpenZWave is gone, ZWaveJS is in, everything’s been migrated and the sky’s the limit. I was just REALLY hoping to be able to use the ‘migration wizard’ instead of doing it all manually.
well, too late now but two things…
First off, you could have more easily just renamed the new entity_id’s to match the old entity_id’s. That way everything else would have just worked. Except the stuff using device_id’s.
Which brings me to point number two…
don’t use device_id’s, or more specifically, don’t use automations/scripts that use device_id’s.
you should really only use triggers, conditions and service calls, etc that use entity_id’s instead.
the issue (which you just got a big taste of) is that if you ever need to replace a device with another even if it’s identical none of the device_id’s will be re-used so you need to modify everything that uses it.
if you use entity_id’s then all you need to do is rename the entity to the same old entity_id (as previously suggested).
So far no help from any one here, How come dedicated Homeassistant Blue hardware not accepting your own updates and getting stuck in perm loop?
People who develop home assistant need to answer these questions instead of congratulating on new updates… Such a bad behaviour since no reply to these serious questions.
Core is stuck on a loop 2022.11.1
supervisor stuck on a loop 2022.11.0
Operating System stuck on 9.2 and cannot update to 9.3 since getting a error message, failed to called service update /install , with OS manager update blocked from execution…
May be I should sell the Blue hardware and forget about this crazy project.
@sirihewa It looks like you’re running supervisor 2022.11.0. If that’s the case (you can find the version here), you can fix it by running the curl script in the community post.
@ svenshomelab, Tks for the info but unfortunately everything is stuck and don’t have terminal apps either since I try to clean it and deleted them before. Now it ignore all terminal add on or new add ons.
If some created a terrible beta version that’s stuck, then that person should release a solution.
I still have add on “Check Home Assistant configuration” but activating it and log says,
Collecting sniffio
Downloading sniffio-1.3.0-py3-none-any.whl (10 kB)
Collecting rfc3986[idna2008]<2,>=1.3
Downloading rfc3986-1.5.0-py2.py3-none-any.whl (31 kB)
Collecting MarkupSafe>=2.0
Downloading MarkupSafe-2.1.1-cp39-cp39-musllinux_1_1_aarch64.whl (30 kB)
Collecting text-unidecode>=1.3
Downloading text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.2/78.2 KB 10.3 MB/s eta 0:00:00
Collecting urllib3<1.27,>=1.21.1
Downloading urllib3-1.26.13-py2.py3-none-any.whl (140 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 KB 12.0 MB/s eta 0:00:00
Collecting idna<4,>=2.5
Downloading idna-3.4-py3-none-any.whl (61 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 KB 8.3 MB/s eta 0:00:00
Collecting dbus-fast<2.0.0,>=1.22.0
Downloading dbus_fast-1.82.0.tar.gz (65 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.7/65.7 KB 6.1 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status ‘done’
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status ‘done’
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status ‘done’
Collecting pycparser
Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 KB 13.0 MB/s eta 0:00:00
Collecting h11<0.15,>=0.13
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 KB 7.8 MB/s eta 0:00:00
Collecting anyio<5.0,>=3.0
Downloading anyio-3.6.2-py3-none-any.whl (80 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 80.6/80.6 KB 7.2 MB/s eta 0:00:00
Building wheels for collected packages: home-assistant-bluetooth, lru-dict, dbus-fast
Building wheel for home-assistant-bluetooth (pyproject.toml): started
Building wheel for home-assistant-bluetooth (pyproject.toml): finished with status ‘done’
Created wheel for home-assistant-bluetooth: filename=home_assistant_bluetooth-1.8.1-cp39-cp39-musllinux_1_2_aarch64.whl size=72328 sha256=be4f2ab78d28828e630b643a6d531daa1af53753d166f80ff26bcaaee621a31c
Stored in directory: /root/.cache/pip/wheels/80/8b/fa/2b396d66ae1db7ed71d2fa900f850ffe0995f905c501a0c2e5
Building wheel for lru-dict (setup.py): started
Building wheel for lru-dict (setup.py): finished with status ‘error’
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
running bdist_wheel
running build
running build_ext
building ‘lru’ extension
creating build
creating build/temp.linux-aarch64-cpython-39
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fno-semantic-interposition -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/local/include/python3.9 -c lru.c -o build/temp.linux-aarch64-cpython-39/lru.o
error: command ‘gcc’ failed: No such file or directory
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for lru-dict
Running setup.py clean for lru-dict
Building wheel for dbus-fast (pyproject.toml): started
Building wheel for dbus-fast (pyproject.toml): finished with status ‘done’
Created wheel for dbus-fast: filename=dbus_fast-1.82.0-cp39-cp39-musllinux_1_2_aarch64.whl size=857993 sha256=fb680efd5971c6e120dc3027f91ef719bc789fe4090b44f58e47c68abbda3987
Stored in directory: /root/.cache/pip/wheels/c5/98/08/0882f0e002b6d14ae04902ebe9fe2b28b02bcc9ed135d3b972
Successfully built home-assistant-bluetooth dbus-fast
Failed to build lru-dict
Installing collected packages: voluptuous, text-unidecode, rfc3986, pytz, lru-dict, ifaddr, ciso8601, voluptuous-serialize, urllib3, typing-extensions, sniffio, six, pyyaml, python-slugify, PyJWT, pycparser, orjson, multidict, MarkupSafe, idna, h11, frozenlist, charset-normalizer, certifi, awesomeversion, attrs, atomicwrites-homeassistant, async-timeout, astral, yarl, requests, jinja2, dbus-fast, cffi, anyio, aiosignal, httpcore, cryptography, bleak, bcrypt, aiohttp, httpx, home-assistant-bluetooth, homeassistant
Running setup.py install for lru-dict: started
Running setup.py install for lru-dict: finished with status ‘error’
error: subprocess-exited-with-error
× Running setup.py install for lru-dict did not run successfully.
│ exit code: 1
╰─> [10 lines of output]
running install
/usr/local/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_ext
building ‘lru’ extension
creating build
creating build/temp.linux-aarch64-cpython-39
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fno-semantic-interposition -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/local/include/python3.9 -c lru.c -o build/temp.linux-aarch64-cpython-39/lru.o
error: command ‘gcc’ failed: No such file or directory
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> lru-dict
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped
Is “home assistant” a coding cowboys paradise ?
I can get into the Blue via its HDMI interface, Any help from that?
Yes. Connect a keyboard and mouse and enter the commands from the link above.
Tks… it worked.