Python 3.6 upgrade of a virtualenv

Sorry but the all in one installer create a virtual environment by default or not?

I have no idea. I have never done the all in one installer.

The All-in-One installer does indeed create a virtualenv.

1 Like

Yes the all-in-one installation do this:

By default, installation makes use of a Python Virtualenv. If you wish to not follow this recommendation, you may add the flag -n to the end of the install command specified above.

So i think your procedure is correct for this installation, but I don’t understand where I make the mistake and why HA after this procedure doesn’t work.

Maybe here there is someone who know where is the mistake. :grinning:

I believe you have deleted your homeassistant install. You probably need to re-install it after upgrading python.

And the steps below worked for me, twice.

Results: Upgraded OS to Debian Stretch 9, Python 3.6.3, Latest Hass, Hass settings/configuration preserved.

Setup: Raspbian Debian 8/9 with Hass in virtual environment

#Upgrade to Debian Stretch (9) if you haven't already (Optional)
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo sed -i 's/jessie/stretch/g' /etc/apt/sources.list
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo reboot

Note:  You may need to set your static IP if you had one.  WiFi settings carry over.  You do get prompts throughout the install, I accepted all new settings.

#Update Python dependencies
sudo apt-get install build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev

#Download and install latest python
cd /home/pi
wget https://www.python.org/ftp/python/3.6.3/Python-3.6.3.tgz
tar xzvf Python-3.6.3.tgz
cd Python-3.6.3/
./configure
make
sudo make install

Note: Last step takes a while.

#Delete old virtual environment
cd /srv/
sudo rm -r *

Warning:  If you have any configuration files here, this is a good time to move them or back them up.

#Create new virtual environment with latest python
cd /srv/
sudo mkdir homeassistant
sudo chown homeassistant:homeassistant homeassistant
sudo su -s /bin/bash homeassistant
cd /srv/homeassistant
python3.6 -m venv .
source bin/activate

#Install latest Hass
pip3 install homeassistant
hass

Note:  First start takes a while
10 Likes

I think it’s basically the install of homeassistant at the end that he is missing.

That’s exactly right.

So, i followed you instructions, and it mostly went smooth. However, during the jessie->stretch upgrade, i got prompted for a few upgrades where it was recommended not to. I did it anyway… Also, the command near the end, python3.6.3 -m venv, it should only be python3.6 -m venv /srv/homeassistant without .3 in the executed command

Anyways, i followed all the instructions, and most things seems to be intact. But not my Z-Wave devices, they are all gone. And mqtt doesn’t work anymore. Here is the log:

2017-10-16 20:28:17 ERROR (MainThread) [homeassistant.setup] Error during setup of component mqtt
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/setup.py", line 191, in _async_setup_component
    result = yield from component.async_setup(hass, processed_config)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/mqtt/__init__.py", line 387, in async_setup
    success = yield from hass.data[DATA_MQTT].async_connect()
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/mqtt/__init__.py", line 497, in async_connect
    self._mqttc.connect, self.broker, self.port, self.keepalive)
  File "/usr/local/lib/python3.6/asyncio/futures.py", line 332, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/local/lib/python3.6/asyncio/tasks.py", line 250, in _wakeup
    future.result()
  File "/usr/local/lib/python3.6/asyncio/futures.py", line 245, in result
    raise self._exception
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/paho/mqtt/client.py", line 760, in connect
    return self.reconnect()
  File "/srv/homeassistant/lib/python3.6/site-packages/paho/mqtt/client.py", line 887, in reconnect
    sock = socket.create_connection((self._host, self._port), source_address=(self._bind_address, 0))
  File "/usr/local/lib/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/usr/local/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
2017-10-16 20:29:07 WARNING (SyncWorker_7) [homeassistant.components.zwave] zwave not ready after 30 seconds, continuing anyway

I did make a backup. Which files could i replace from the backup to get my ZWave setup back? It seems like Z-Wave is running. Time stamp on my files suggest only zwscene.xml, zwcfg_0xc60752a4.xml and the OZW_Log has been changed today. But those are the files accessible through Samba in Windows, along with the config files.

Updated instructions on the typo thanks!

Regarding Z-Wave and MQTT I’m at a loss. I’m using a USB Z-Wave/Zigbee and have over 30+ Z-Wave devices and everything came through. I’m not however using MQTT…

In theory, if you move all your settings to /home/homeassistant/.homeassistant you shouldn’t have any issues.

OK, so i checked again after i gave it a little break. Seems like they are still there, but their names have been reset to default. I had renamed them all for practical reasons :slight_smile: The MQTT still needs some investigation, but i think that will be easy enough to fix. Thank you for the instructions and your help.

If i had to guess you had renamed them before that functionality was available on the front-end UI? My renames apparently stuck. If that is the case, good to know for others following this.

Nope, i did it all in the front end. I just recently started using HA. This time, i’m keeping the names as they are, and giving them freindly_name in the customize.yaml instead. Just fiddling with it now and i see a fibaro wall plug is “nameless” and just switch.switch. Guess i’ll have to re-include it. But all is coming together nicely now. Will finish it tomorrow.

Your original zwcfg_***.xml file was probably located inside the virtualenv directory (where it was before the switch to python-openzwave). The names of Z-Wave devices are stored in that file, so I can only imagine that’s how it happened.

I had a similar problem because I had originally put a symlink inside my /home/hass/.homeassistant directory, pointing to that exact file inside my virtualenv directory. After deleting that directory, my symlink obviously broke. Thankfully, I was able to recover my zwcfg_***.xml file from the backup I made (prior to the Python upgrade), which I now put in my /home/hass/.homeassistant directory.

On a side note: in addition to the steps listed by @ttaidapos I had to run the following command with my virtualenv active: pip3 install mysqlclient That is because I use an external MariaDB database for the recorder/history/logbook. And I had to fix my poorly written Python scripts which I use for my LG Hom-Bot and for Synology’s Surveillance Station.

2 Likes

Cool I added a warning in during the removal of the virtualenv directory. Didn’t realize this was the old location for zwave config. blah!

At the end of this command, it failed with the following exception. Any ideas??

I commented out those openzwave items as you mentioned.

Exception:
Traceback (most recent call last):
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/wheel.py", line 345, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/wheel.py", line 316, in clobber
    ensure_dir(destdir)
  File "/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/usr/local/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages/chardet-3.0.4.dist-info'

I resolved above error by running sudo.

Later I ran into few issues.

Also if zwave upgrade command doesn’t work then run it without ‘upgrade’.

pi@rpi3-aio:~ $ venv
(homeassistant_venv) homeassistant@rpi3-aio:~$ pip3 install python_openzwave
Requirement already satisfied: python_openzwave in /srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages
Requirement already satisfied: six in /srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages (from python_openzwave)
Requirement already satisfied: PyDispatcher>=2.0.5 in /srv/homeassistant/homeassistant_venv/lib/python3.6/site-packages (from python_openzwave)
(homeassistant_venv) 

Also, don’t forget to run this command.

pi@rpi3-aio:/home/homeassistant $ sudo chown -R homeassistant:homeassistant /srv/homeassistant/homeassistant_venv/

Thanks for the fresh write up. I’m actually on an AIO of pre dec 2016 which was in ‘hass’ and not ‘homeassistant’ virtual environment.

Does this detail have impact on your detailed instructions above? eg should I change the stuff when you create a new venv to ‘hass’ instead of ‘homeassistant’ to make sure all works fine or is it better do directly create a ‘homeassistant’ venv?

I would personally go with the later IMO. Of course, make sure you backup everything especially during the remove directory command. Most recent troubleshooting and references refer to the newer directory structure I believe.

Hi there,
it works out of the box on a rpi 2b, armv7l (32 bit). no changes necessary. updated hass venv for hass 0.56.2. maybe the upgrade takes a little bit longer on this system.
thx for sharing !