Python 3.6 upgrade of a virtualenv

If i had to guess you had renamed them before that functionality was available on the front-end UI? My renames apparently stuck. If that is the case, good to know for others following this.

Nope, i did it all in the front end. I just recently started using HA. This time, i’m keeping the names as they are, and giving them freindly_name in the customize.yaml instead. Just fiddling with it now and i see a fibaro wall plug is “nameless” and just switch.switch. Guess i’ll have to re-include it. But all is coming together nicely now. Will finish it tomorrow.

Your original zwcfg_***.xml file was probably located inside the virtualenv directory (where it was before the switch to python-openzwave). The names of Z-Wave devices are stored in that file, so I can only imagine that’s how it happened.

I had a similar problem because I had originally put a symlink inside my /home/hass/.homeassistant directory, pointing to that exact file inside my virtualenv directory. After deleting that directory, my symlink obviously broke. Thankfully, I was able to recover my zwcfg_***.xml file from the backup I made (prior to the Python upgrade), which I now put in my /home/hass/.homeassistant directory.

On a side note: in addition to the steps listed by @ttaidapos I had to run the following command with my virtualenv active: pip3 install mysqlclient That is because I use an external MariaDB database for the recorder/history/logbook. And I had to fix my poorly written Python scripts which I use for my LG Hom-Bot and for Synology’s Surveillance Station.

2 Likes

Cool I added a warning in during the removal of the virtualenv directory. Didn’t realize this was the old location for zwave config. blah!

At the end of this command, it failed with the following exception. Any ideas??

I commented out those openzwave items as you mentioned.

Exception:
Traceback (most recent call last):
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/wheel.py", line 345, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/wheel.py", line 316, in clobber
    ensure_dir(destdir)
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/usr/local/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/chardet-3.0.4.dist-info'

I resolved above error by running sudo.

Later I ran into few issues.

Also if zwave upgrade command doesn’t work then run it without ‘upgrade’.

pi@rpi3-aio:~ $ venv
(homeassistant_venv) homeassistant@rpi3-aio:~$ pip3 install python_openzwave
Requirement already satisfied: python_openzwave in /srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages
Requirement already satisfied: six in /srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages (from python_openzwave)
Requirement already satisfied: PyDispatcher>=2.0.5 in /srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages (from python_openzwave)
(homeassistant_venv) 

Also, don’t forget to run this command.

pi@rpi3-aio:/home/homeassistant $ sudo chown -R homeassistant:homeassistant /srv/homeassistant/homeassistant_venv/

Thanks for the fresh write up. I’m actually on an AIO of pre dec 2016 which was in ‘hass’ and not ‘homeassistant’ virtual environment.

Does this detail have impact on your detailed instructions above? eg should I change the stuff when you create a new venv to ‘hass’ instead of ‘homeassistant’ to make sure all works fine or is it better do directly create a ‘homeassistant’ venv?

I would personally go with the later IMO. Of course, make sure you backup everything especially during the remove directory command. Most recent troubleshooting and references refer to the newer directory structure I believe.

Hi there,
it works out of the box on a rpi 2b, armv7l (32 bit). no changes necessary. updated hass venv for hass 0.56.2. maybe the upgrade takes a little bit longer on this system.
thx for sharing !

Hello, on this command I’m receiving error: -bash: requirements.txt: operation not permitted.
I’m in virtualenv (homeassistant_venv) pi@raspberrypi: /home/homeassistant
Is there anything I can check?

I recently setup HA under Python 3.6 on a fresh Raspbian Stretch image, inspired by this thread (thanks to all!).

I have a second Rpi 3 and MicroSD card for this type of development, and simply copy over my configuration and swap cards. That way, I don’t mess up my “production” HA, and always have an easy fall back (put the old card back in). I started with Raspbian Stretch, September release.

I’ve had success and documented everything step-by-step as I went through it. Since it is a rather involved process, I posted it on my (new) blog site. If you’re interested: http://blog.motleynet.com/2017/11/01/installing-home-assistant-on-vanilla-raspbian-stretch-using-python-3-6-3/. Hope it helps!

1 Like

@rcervenan as user pi you have no appropriate right to write in this directory. become user homeassistant and than it should work (in many venv installations this is a system account, so use the command sudo su -s /bin/bash homeassistant , than make sure your current directory is /home/homeassistant )

Just wanted to thank you @ttaidapos for these instructions.
I was able to upgrade to Raspbian Stretch and also Python 3.6
Everything worked without any issues. I did however have to make a change to my systemd startup service file as it had an extra “homeassistant_env” in the path that I don’t have with the new setup. But all seems fine now. Thanks again.

1 Like

This post was invaluable to me in upgrading to 3.6. Thank you so much!

Regarding the openzwave items, I had to comment them out like you.

After installing the requirements.txt, I then did a version upgrade of home assistant (from 0.56.x to 0.58.1).

I then had to reset my zwave NetworkKey for my z-wave secure devices. This must have been lost or moved in the upgrade.

Thank you so much!

  1. Has anyone done this procedure and been able to get MQTT to work again?

I’ve done a purge and reinstall a couple of times and haven’t gotten it to work yet.

  1. custom_ui_state_card stuff no longer works.

How do I fix those?

  1. zwave entities lost all of my custom names.

Any reasonably easy way to recover those without having to manually rename them all?

  1. And I keep getting tons of errors in the home-assistant.log for my non-foscam cameras even tho the cameras seem to be working fine.

EDIT****

2 & 3 are fixed now.

2 was fixed by updating my custom_ui stuff from the andrey-git site

3 was fixed by bringing in my old zwcfg file from a backup.

still working on MQTT

EDIT 2****

MQTT is working again…just me being stupid…damn those hidden extra spaces at the end…

Are there any plans to update the all in one isntaller? It was doing an excellent job and seems like a faster solution

I think I remember reading on the blog that the All-In-One installer was going to be eliminated or already has.

We need to update the first post for anyone coming from an AIO installation and instead of:

pip3 install -r requirements.txt

it should be:

pip3 install homeassistant

My installation was not even starting until I did this. I saw another post in here mention it but updating the first post may help the next person lol.

Also those users need to make sure that they update the service accordingly (remove homeassistant_venv)

How did you do that?

Recently I found one of the zwave nodes was dead and trying to troubleshoot it.

After checking further I found the previous symlink has the connection to old python 3.4 folders.

pi@rpi3-aio:/srv/homeassistant/src/open-zwave-control-panel $ ls -lt
total 2.9M
lrwxrwxrwx 1 homeassistant homeassistant  114 Jul 20 16:42 config -> /srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/libopenzwave-0.3.3-py3.4-linux-armv7l.egg/config

pi@rpi3-aio:/srv/homeassistant/src/open-zwave-control-panel $ sudo ln -s /srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/libopenzwave-0.4.0.31.egg-info/config /srv/homeassistant/src/open-zwave-control-panel/config

I corrected symlink but I couldn’t find “libopenzwave-0.xxxxxx” folder under “python3.6”.

pi@rpi3-aio:/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages $ ls -lt lib*
-rwxr-xr-x 1 homeassistant homeassistant 3.9M Oct 29 12:38 libopenzwave.cpython-36m-arm-linux-gnueabihf.so

But there is “python_openzwave-0.4.0.35-py3.6.egg-info” folder.

pi@rpi3-aio:/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/python_openzwave-0.4.0.35-py3.6.egg-info $ ls -lt
total 28K
-rw-r--r-- 1 homeassistant homeassistant 1.4K Oct 29 12:38 installed-files.txt
-rw-r--r-- 1 homeassistant homeassistant  927 Oct 29 12:38 SOURCES.txt
-rw-r--r-- 1 homeassistant homeassistant    1 Oct 29 12:38 dependency_links.txt
-rw-r--r-- 1 homeassistant homeassistant    1 Oct 29 12:38 not-zip-safe
-rw-r--r-- 1 homeassistant homeassistant 1.2K Oct 29 12:38 PKG-INFO
-rw-r--r-- 1 homeassistant homeassistant   24 Oct 29 12:38 requires.txt
-rw-r--r-- 1 homeassistant homeassistant   49 Oct 29 12:38 top_level.txt

How do I fix this?

Nope - no developer stepped forward to support it, so it was removed. If you want a faster solution use Hassbian.

Bit of background (as best I understand it): AIO came about back when Hassbian was just the OS + HA, and would include OpenZWave and Mosquitto. Some time back OZW started being installed on demand, and Hassbian included support for installing Mosquitto. As well the original drive for Mosquitto was OwnTracks, and the HTTP module means that Mosquitto is no longer so critical.

As a result, the AIO installer had been replaced by Hassbian for those after a quick initial install, and the virtual environment install is now enough for most people who’re adding HA to an existing build.