Python upgrade

You’re welcome. Your logs are your best friend in HA.

Is anyone else having a problem with gstreamer after upgrading python? I did a pip3 install gi to get past the first issue of it not finding the gi module, but now I get:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity_platform.py", line 126, in _async_setup_platform
    SLOW_SETUP_MAX_WAIT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 416, in wait_for
    return fut.result()
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/gstreamer/media_player.py", line 31, in setup_platform
    from gsp import GstreamerPlayer
  File "/srv/homeassistant/lib/python3.7/site-packages/gsp/__init__.py", line 11, in <module>
    import gi  # pylint: disable=import-error
  File "/srv/homeassistant/lib/python3.7/site-packages/gi/__init__.py", line 39
    print url
            ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(url)?```

I never said they shouldn’t. I was responding DIRECTLY to a comment you made regarding things NOT working in docker (they actually work, but you have to implement them differently).

You’re arguing against my comments informing you of a DIFFERENT way to accomplish what you wanted. It’s silly really.

I’m stuck at step 17:

Error: Command ‘[’/srv/homeassistant/bin/python3.7’, ‘-Im’, ‘ensurepip’, ‘–upgrade’, ‘–default-pip’]’ returned non-zero exit status 1.

Did you copy the line exactly as it was stated.

  1. /usr/local/bin/python3.7 -m venv .

You stated that they work, and I am sure they do in some way or another.

What was not explained is HOW do get them working in sufficient detail, under my new Docker configuration.

As I said, deprecating Python 3.5 left those who installed Home Assitant as a venv in the lurch. If this was done after Buster was released, then it would be an acceptable upgrade path. So far, Buster is not officially the ‘current stable’, and hence my comment on lack of choice, which others have voiced too.

Apparently that module is not yet compatible with Python 3.7. The print() command requires parentheses. I think they were optional previously.

I got this working. It looks like it installed gi 1.2 and it needs 1.0. How I fixed mine:

ln -s /usr/lib/python3/dist-packages/gi /srv/homeassistant/lib/python3.7/site-packages/gi
cd /srv/homeassistant/lib/python3.7/site-packages/gi
ln -s _gi.cpython-35m-arm-linux-gnueabihf.so _gi.cpython-37m-arm-linux-gnueabihf.so
cd overrides
ln -s _gi_gst.cpython-35m-arm-linux-gnueabihf.so _gi_gst.cpython-37m-arm-linux-gnueabihf.so

I added sudo, because when I don’t I get:
/usr/local/bin/python3.7 -m venv .
Error: [Errno 13] Permission denied: ‘/srv/homeassistant/include’

I succeeded when i first switch to the homeassistant user first as mentioned here:

When I first wrote those instructions the first thing I said was it was all run as the homeassistant user.

Use pip install gi==1.0
Note the double equals sign.

I’ve figured out how to update python without affecting the OS python installation, using pyenv.

Note: this will take a few hours of your time, between installing and building python and home-assistant.

I’m using a virtualenv install of HA, running on a Pi 2. Building python took almost 30 minutes, + you will have to reinstall homeassistant (another 30 minutes). Don’t worry about your configuration files (backup just in case), this will install a new virtualenv, so after you test to make sure everything works, you can delete the /srv/homeassistant.

Second note: I’m running Raspbian Stretch, and I understand Raspbian Buster is coming out in a few days, but I feel this will be a more maintainable installation of Hass. If this has happened once, it will happen again. Else, you don’t have to upgrade to Buster if you don’t want to. :man_shrugging:

Run 1-6 as your homeassistant user
sudo -u homeassistant -H -s

  1. Install pyenv, I did the Github version, so follow 1-5. (You can replace .bash_profile with .bashrc or .bash_aliases)
  2. (Step 6 from above^) Choose your python version and start installing. I used 3.7.3 and it took ~30minutes.
    pyenv install 3.7.3
  3. Install pyenv virtualenv
  4. Create a new virtualenv with your new python
    pyenv virtualenv 3.7.3 home-assistant
  5. Jump into this virtualenv
    pyenv activate home-assistant
  6. And install home-assistant again (~30 minutes)
    python3 -m pip install wheel homeassistant
  7. While that is installing, you can update your systemd, or other, config for Hass with a user with sudo powers. You can backup this file, to be safe.
    sudo nano /etc/systemd/system/[email protected]
    Change the ExecStart= line to ExecStart=/home/homeassistant/.pyenv/versions/home-assistant/bin/hass -c "/home/homeassistant/.homeassistant"
    Run sudo systemctl daemon-reload to make sure this change is registered with systemd.
  8. I had Hass stopped while I was doing this, but this should work once Hass is re-installed
    sudo systemctl restart [email protected]
1 Like

If, like me, you are/were using venv, and was concerned with the upcoming Python 3.5 deprecation, despair no more: you can have a working solution using venv on Buster.

Raspian Buster was released on the 24th of June.

I was able to upgrade my RPi to it, and HA works.

To do this, here are the commands:

sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade

Then manually edit the following files, and change all occurrences of ‘stretch’ to ‘buster’

/etc/apt/sources.list and /etc/apt/sources.list.d/raspi.list

Then continue with the same commands as above :

sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade

Then reboot, and you have Buster.

Then you have to create a new venv, using Python 3.7 of course. Use the same method that you used to create the initial one.

One issue I found:

The logbook, recorder and history components do not work. Home Assistant says that they are missing dependencies.

I had them as MySQL under Stretch, but now Buster installs MariaDB.

Per the instructions under the recorder component, I need to install the pip mysqlclient. That fails because it cannot find the mysql_config command, which used to be part of libmysqlclient-dev. But the equivalent libmariadbclient-dev is missing this command.

The solution to the above is to do:

python3 -m pip install PyMySQL

Then in your configuration.yaml, change the db_url to:

db_url: 'mysql+pymysql://localhost/DBNAME?charset=utf8'

All works now.

It might be worth installing Raspbian Buster at this stage.

Personally I’d just do a fresh install of Raspbian and Home Assistant and copy over config. It takes a little longer but will be worth it long term.

lug 04 01:43:17 raspberry hass[28196]:   File "/srv/homeassistant/lib/python3.7/site-packages/homekit/controller/ip_implementation.py", line 416, in __init__
lug 04 01:43:17 raspberry hass[28196]:     c2a_key, a2c_key = get_session_keys(conn, pairing_data, write_fun)
lug 04 01:43:17 raspberry hass[28196]:   File "/srv/homeassistant/lib/python3.7/site-packages/homekit/protocol/__init__.py", line 358, in get_session_keys
lug 04 01:43:17 raspberry hass[28196]:     raise InvalidSignatureError('step 3')
lug 04 01:43:17 raspberry hass[28196]: homekit.exceptions.InvalidSignatureError: step 3
lug 04 01:43:18 raspberry hass[28196]: 2019-07-04 01:43:18 WARNING (MainThread) [homeassistant.components.sensor] Platform nut not ready yet. Retrying in 30 seconds.
lug 04 01:43:25 raspberry hass[28196]: 2019-07-04 01:43:25 WARNING (MainThread) [homeassistant.components.switch] Setup of platform broadlink is taking over 10 seconds.
lug 04 01:43:26 raspberry hass[28196]: 2019-07-04 01:43:26 WARNING (MainThread) [homeassistant.components.sensor] Setup of platform alpha_vantage is taking over 10 secon
lug 04 01:43:26 raspberry hass[28196]: 2019-07-04 01:43:26 ERROR (SyncWorker_7) [homeassistant.components.ios.notify] The notify.ios platform was loaded but no devices e
lug 04 01:43:30 raspberry hass[28196]: 2019-07-04 01:43:30 ERROR (SyncWorker_5) [homeassistant.components.broadlink.switch] Failed to connect to device

I have this error, can someone help me what to do? please if you can show me the exact input to make, I am a noob.

Thank’you very much!! Working very well. :v::v:

please help me! I am getting crazy!

 Main PID: 4242 (hass)
    Tasks: 32 (limit: 3534)
   Memory: 73.5M
   CGroup: /system.slice/system-home\x2dassistant.slice/[email protected]
           └─4242 /home/homeassistant/.pyenv/versions/home-assistant/bin/python3 /home/homeassistant/.pyenv/versions/home-assistant/bin/hass -c /home/homeassistant/.homeassistant

lug 22 00:36:06 raspberry hass[4242]:   File "/home/homeassistant/.pyenv/versions/3.7.3/lib/python3.7/concurrent/futures/thread.py", line 57, in run
lug 22 00:36:06 raspberry hass[4242]:     result = self.fn(*self.args, **self.kwargs)
lug 22 00:36:06 raspberry hass[4242]:   File "/home/homeassistant/.pyenv/versions/home-assistant/lib/python3.7/site-packages/homekit/controller/ip_implementation.py", line 73, in list_accessories_and_characteristics
lug 22 00:36:06 raspberry hass[4242]:     self.session = IpSession(self.pairing_data)
lug 22 00:36:06 raspberry hass[4242]:   File "/home/homeassistant/.pyenv/versions/home-assistant/lib/python3.7/site-packages/homekit/controller/ip_implementation.py", line 416, in __init__
lug 22 00:36:06 raspberry hass[4242]:     c2a_key, a2c_key = get_session_keys(conn, pairing_data, write_fun)
lug 22 00:36:06 raspberry hass[4242]:   File "/home/homeassistant/.pyenv/versions/home-assistant/lib/python3.7/site-packages/homekit/protocol/__init__.py", line 358, in get_session_keys
lug 22 00:36:06 raspberry hass[4242]:     raise InvalidSignatureError('step 3')
lug 22 00:36:06 raspberry hass[4242]: homekit.exceptions.InvalidSignatureError: step 3
lug 22 00:36:07 raspberry hass[4242]: 2019-07-22 00:36:07 ERROR (SyncWorker_5) [homeassistant.components.broadlink.switch] Failed to connect to device

Works perfetcly, thank you!

Thanks for posting these steps. Worked for me :slight_smile:

I am having BIG problems installing 0.101.3 release of HA over the 0.100.3.
What i get are tons of errors regarding python3 version.
I have installed 3.7.2 release of python3 as HA informations says:

37

But the log errors often tell me this:

2019-11-09 12:49:46 ERROR (MainThread) [homeassistant.setup] Setup failed for mqtt: unknown error
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/setup.py", line 138, in _async_setup_component
    component = integration.get_component()
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/loader.py", line 223, in get_component
    cache[self.domain] = importlib.import_module(self.pkg_path)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/mqtt/__init__.py", line 50, in <module>
    from . import config_flow, discovery, server  # noqa pylint: disable=unused-import
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/mqtt/server.py", line 7, in <module>
    from hbmqtt.broker import Broker, BrokerException
  File "/srv/homeassistant/lib/python3.7/site-packages/hbmqtt/broker.py", line 6, in <module>
    import websockets
  File "/srv/homeassistant/lib/python3.7/site-packages/websockets/__init__.py", line 3, in <module>
    from .client import *
  File "/srv/homeassistant/lib/python3.7/site-packages/websockets/client.py", line 12, in <module>
    from .protocol import CONNECTING, OPEN, WebSocketCommonProtocol
  File "/srv/homeassistant/lib/python3.7/site-packages/websockets/protocol.py", line 17, in <module>
    from .compatibility import asyncio_ensure_future
  File "/srv/homeassistant/lib/python3.7/site-packages/websockets/compatibility.py", line 9
    asyncio_ensure_future = asyncio.async           # Python < 3.5

From this log it seems i need to upgrade python3 from 3.5, but i already have 3.7.2.
Any help? In this moment i cannot upgrade to 0.101.3 !!!