Install fails on 0.104.3 , wheel failure python 3.7.5 ubuntu 18.04 LTS

Installation
1

Essential sections failing to build, python3.7.5 runing on Odroid XU4 VENV`

Building wheels for collected packages: cryptography, bcrypt, pyyaml, ruamel.yaml, voluptuous, voluptuous-serialize, aiohttp, cffi, MarkupSafe, multidict, yarl
  Running setup.py bdist_wheel for cryptography ... error
  Complete output from command /srv/homeassistant/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-oj6pi4zz/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpbucybxj2pip-wheel- --python-tag cp37:
  
  Installed /tmp/pip-build-oj6pi4zz/cryptography/.eggs/cffi-1.13.2-py3.7-linux-armv7l.egg
  Searching for pycparser
  Reading https://pypi.python.org/simple/pycparser/
  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz#sha256=a988718abfad80b6b157acce7bf130a30876d27603738ac39f140993246b25b3
  Best match: pycparser 2.19
  Processing pycparser-2.19.tar.gz
  Writing /tmp/easy_install-shy71390/pycparser-2.19/setup.cfg
  Running pycparser-2.19/setup.py -q bdist_egg --dist-dir /tmp/easy_install-shy71390/pycparser-2.19/egg-dist-tmp-x2w5l9ll
  warning: no previously-included files found matching 'setup.pyc'
  warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
  warning: no previously-included files matching 'lextab.*' found under directory 'tests'
  warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
  warning: no previously-included files matching 'lextab.*' found under directory 'examples'
  zip_safe flag not set; analyzing archive contents...
  pycparser.ply.__pycache__.lex.cpython-37: module references __file__
  pycparser.ply.__pycache__.lex.cpython-37: module MAY be using inspect.getsourcefile
  pycparser.ply.__pycache__.yacc.cpython-37: module references __file__
  pycparser.ply.__pycache__.yacc.cpython-37: module MAY be using inspect.getsourcefile
  pycparser.ply.__pycache__.yacc.cpython-37: module MAY be using inspect.stack
  pycparser.ply.__pycache__.ygen.cpython-37: module references __file__
  creating /tmp/pip-build-oj6pi4zz/cryptography/.eggs/pycparser-2.19-py3.7.egg
  Extracting pycparser-2.19-py3.7.egg to /tmp/pip-build-oj6pi4zz/cryptography/.eggs
  
  Installed /tmp/pip-build-oj6pi4zz/cryptography/.eggs/pycparser-2.19-py3.7.egg
  /usr/lib/python3.7/distutils/dist.py:274: UserWarning: Unknown distribution option: 'long_description_content_type'
    warnings.warn(msg)
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help
  
  error: invalid command 'bdist_wheel'
  
  ----------------------------------------
  Failed building wheel for cryptography

I had to use
python3.7 -m pip install homeassistant
pip has been updated

I’ve installed as much as I can find for python 3.7


type or paste code here

(homeassistant) root@odroid:/srv/homeassistant# apt install python3.7*
Reading package lists… Done
Building dependency tree
Reading state information… Done
Note, selecting ‘python3.7-2to3’ for glob ‘python3.7*’
Note, selecting ‘python3.7-tk’ for glob ‘python3.7*’
Note, selecting ‘python3.7-venv’ for glob ‘python3.7*’
Note, selecting ‘python3.7-distutils’ for glob ‘python3.7*’
Note, selecting ‘python3.7’ for glob ‘python3.7*’
Note, selecting ‘python3.7-dbg’ for glob ‘python3.7*’
Note, selecting ‘python3.7-dev’ for glob ‘python3.7*’
Note, selecting ‘python3.7-doc’ for glob ‘python3.7*’
Note, selecting ‘python3.7-gdbm’ for glob ‘python3.7*’
Note, selecting ‘python3.7-minimal’ for glob ‘python3.7*’
Note, selecting ‘python3.7-lib2to3’ for glob ‘python3.7*’
Note, selecting ‘python3.7-examples’ for glob ‘python3.7*’
Note, selecting ‘python3-distutils’ instead of ‘python3.7-distutils’
Note, selecting ‘python3-gdbm’ instead of ‘python3.7-gdbm’
Note, selecting ‘python3-lib2to3’ instead of ‘python3.7-lib2to3’
Note, selecting ‘python3-tk’ instead of ‘python3.7-tk’
python3-distutils is already the newest version (3.6.9-1~18.04).
python3-gdbm is already the newest version (3.6.9-1~18.04).
python3-lib2to3 is already the newest version (3.6.9-1~18.04).
python3-tk is already the newest version (3.6.9-1~18.04).
python3.7 is already the newest version (3.7.5-2~18.04).
python3.7-dbg is already the newest version (3.7.5-2~18.04).
python3.7-dev is already the newest version (3.7.5-2~18.04).
python3.7-doc is already the newest version (3.7.5-2~18.04).
python3.7-examples is already the newest version (3.7.5-2~18.04).
python3.7-minimal is already the newest version (3.7.5-2~18.04).
python3.7-venv is already the newest version (3.7.5-2~18.04).

The previous version 103.6 ran but would not update due to pythone 3.6 installed
Anyone know what other libs are needed to build the wheels
I downloaded with APT python3.75, I did try to compile python3.74 , but fails on make test badly
TIA
Now a tottally hosed HA . It need to be in a VENV as other applications running on the XU4

2

Probably you’re missing some required package. A quick Google search didn’t show anything obvious though. On my Debian based installs I’ve used the following:

sudo apt install build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev \
  libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev \
  zlib1g-dev libudev-dev

Only if you need to call those directly from HA. Otherwise you can use Docker.

3

Thanks
there were two packages on that list missing, which I installed, but exactly the same.

Failed to build ruamel.yaml voluptuous-serialize aiohttp voluptuous cryptography bcrypt pyyaml MarkupSafe multidict yarl cffi

If I use python3 -m install
the version of python used is 3.6, now depreciated on HA and 103.6 is downloaded.
I suspect the problem is python 3.7 is not fully supported on the odroid version of ubuntu18.04.
only python3.7-minimal is available ,
I’ll try compiling from source code python 3.7.4 again, and ignore the failures in “make test”
and run make install,
then try again, if not its going to be stuck on 0.103.6 until such time as a full install of python 3.7 or 3.8 is available.
Need to get this sorted as HA controls a lot of the lighting and hot water

4

I’d suggest you revert to the previous version of HA and Python, then build a fresh venv for Python 3.8 (to save yourself having to upgrade to 3.8 again relatively soon).

That way too you’re not risking the running setup.

5

Thats a good idea
I compiled python3.7.4 and ignored the few failures on “make test” and installed
ran depmod a few times,
ran “python3.7 -m pip install homeassistant”
and success,
hass --open-ui took ages but I was able to open a giu on port 8123
The GUI is very different to that of 0.103.6
the main thing I cant find under configuration or developer tools is
the section to reload the scripts and test the configuration
I’ve FTP’d the configuration.yaml and automations.yaml into .homeassistant/
but yet to figure out how to load the scripts and test for errors.

odroid@odroid:~/.homeassistant$ ls -la
total 56
drwxr-xr-x  3 root   odroid  4096 Jan 31 10:42 .
drwxr-xr-x 23 odroid odroid  4096 Jan 31 10:20 ..
-rw-rw-rw-  1 root   root   10355 Jan 30 18:35 automations.yaml
-rw-rw-rw-  1 root   root    5282 Jan 19 14:36 configuration.yaml
-rw-rw-rw-  1 root   root    1071 Apr 27  2019 customize.yaml
-rw-rw-rw-  1 root   root      98 Apr 26  2019 groups.yaml
-rw-r--r--  1 root   odroid  4096 Jan 31 10:20 home-assistant_v2.db
-rw-r--r--  1 root   root     838 Jan 11 11:16 known_devices.yaml
-rw-rw-rw-  1 root   root       0 Sep 23  2018 scripts.yaml
-rw-rw-rw-  1 root   root     290 Sep 23  2018 secrets.yaml
drwxr-xr-x  2 root   root    4096 Jan 31 10:35 .storage
-rw-rw-rw-  1 root   root     226 Apr 13  2019 ui-lovelace.yaml

can you point me to where the “load scripts and test config” has moved to please

6

This gets worse
rebooted the odroidxu4 , to see if HA would pick up the automations and configuration yaml scripts
HA didn’t restart,
so I ran hass --open-io from /srv/homeassistant/bin
and it reverted to 0.103.6 and crashed out

odroid@odroid:/srv/homeassistant/bin$ ls
activate       easy_install      getmac      jp.py      moggsplit        pip3         python        rst2html.py            rst2pseudoxml.py  slugify
activate.csh   easy_install-3.7  hass        jws        mutagen-inspect  pip3.7       python3       rst2latex.py           rst2s5.py         tlsdb.py
activate.fish  edsig             hbmqtt      mid3cp     mutagen-pony     __pycache__  python3.7     rst2man.py             rst2xetex.py      tls.py
chardetect     envs              hbmqtt_pub  mid3iconv  pasteurize       pyhs100      rst2html4.py  rst2odt_prepstyles.py  rst2xml.py
distro         futurize          hbmqtt_sub  mid3v2     pip              pyjwt        rst2html5.py  rst2odt.py             rstpep2html.py

I’ll try de-activating the python directory as that is python3.6 and try doing and updating HA and see if it will go back to 0.104
Upgrading a version shouldn’t be as difficult as this :frowning:

A bit more info

I’ve used “apt remove python3.6 and python”
That removed the desktop, but that doesn’t matter as the odroid is headless
deleted /srv/homeassistant
and reinstalled 0.104.3 , which succeded
ran hass --open-ui
and logged in, this time the configuration check was available and able to reload
automations.yaml etc
BUT
I still just have only the weather app which is loaded by default on lovelace
HAS the use of automation.yaml and configuration.yaml been depreciated in 0.104.3
so everything has to be entered from the UI ???
I can see that “reloadcore” is missing on configuration-server controls
the documentation is very sparse on migrating from earlier versions to 0.104.3

I’ll start a new thread on config and automation scripts not being read, as the install is OK now

7

It’s not where you run it from, it’s which venv you activate.

Why? You could have installed 3.7 or 3.8 alongside that

Nope

Enable advanced mode in your user profile.

You really should install a startup script :wink:

8

Hi
All noted
I had to remove python 3.6 as when hass was run it decided to down grade from 104 to 103
Even on the advance menu reload core is no longer there.

If I could have got my existing config and automations to run, I would have tried for longer, but
none of my MQTT devices could be detected, even though I could see them on mqtt explorer from another machine.

I’ve reflashed the eMMC card and I’ll put 0.103.6 on it, I can switch between a sd card and eMMC, so once 103.6 is stable I’ll have another go on the sdcard.
Noted on the startup script,
thanks

9

Hi the saga continues
Loaded on to a fresh install on ubuntu18.04 , and the automations found straight away python3.6, HA 0.103.6

the only thing I could see different in the install is on 103.3 with python3.6 the number of python packages built is about 8
on 104.3 on python3.7 its about 4 , significantly less, difficult to count as its pretty quick

Is it possible there could be a bug ???
most people if upgrading from one release to another may not see this.
I’ve noticed on an upgrade it only rebuilds the minimum
Start up scripts with systemd is giving a headache as for some reason it installed hass in /usr/local/bin. , I’ll move it to /srv/homeassitant/bin
That will be a mistake of mine somewhere ,
minor probs: the dark_sky api cant be in configuartion.yaml
the weather card in lovelace says its sunny at 10pm , location is correct
and time intervals seem to have changed from minutes to seconds
There will be other errors made by me as I spent most of thursday and all of friday tying to get this working. So forgot to backup a few other thing that run on the odroidxu4.

I’ve now succeeded with 0.104.3 using python3.8 on the odroidxu4
on a SD card , the automations were recognised , the changes in http settings in configuration.yaml , did cause hass to fail a couple of times until corrected.

I have also suceeded in upgrading 0.103.6 to 0.104.3 using Python3.8
and all automations functional , well looks like it.

Should anyone read this with a odroid XU4 runing ubuntu 18.04LTS from the odroid repo
DO NOT attempt to use python3.7 from the repo , go straight for python3.8

10

You can just specify a different path in the startup script.

Looks though like you didn’t install it in a venv, which will cause you problems if you ever install another Python based program outside of a venv.

Not sure what you mean by this

Where?

11

Hi
The dark sky api was in configuration.yaml , it got pick up by the config checker, and had to hash it out., not sure where that needs to be put now
Also I had a time tick running with an interval of ‘/0.5’, , that wasn’t liked , so thats now 5 mins
Apart from that which is minor I think its all functional.

its other stuff that was on the XU4 which I forgot to back up causing a few headaches now.
Thanks for you help

12

Still in configuration.yaml. If it was generating an error in the config check, likely you needed to fix something from a breaking change. If you’d shared the error, and your config, we could maybe have helped.

13

Ok
I’ve unhashed it, and accepted, on the version I’m running off the eMMC card, on the SD card it wasn’t accepted.
I had to reload the version again as it was saying version 103.6 on the UI, but hass was 104.3.
You can define which version of python is used to install , but when running hass --open-ui the first time after an install , if it has to build anything python3.6 gets picked up.
I renamed python3.6 ln /usr/local/lib to stop it, which seemed to work, and the version running is definitely 104.3.
the naming of python3.* as python3 does not help , it may help on other applications to plug dependency holes, But IMO on HA its a curse and makes it very difficult to build on a pure version. Hopefully on the next HA version there will be more python3.8 packages released by ubuntu.

14

That’s probably because you didn’t install it in a venv.

You’ve likely got at least two, maybe more, different installs going on here.

Works fine here :wink: I’d suggest you seriously consider switching to using the Docker install method, which means you’ll never need to install Python again.

15

Hi
I did use the VENV
I used the instructions here :-


The only difference was to use python3.8 , instead of typing “python3” , I used “python3.8”
and on the install used the switch to install into /home/odroid/
Docker may be the solution, but not what I prefer, I’m not a fan of containers.
What I may try is renaming /usr/local/lib before any install so it can’t find anything previously installed. 
odroid@odroid:~$ ls /usr/local/lib
python2.7  python3.6  python3.8  test

test was the python3.6 dir before the last install.
Its when hass --open-ui is run it sees python3.6
I wonder if using the isolate switch would help ??

python3.8 -I -m pip install homeassistant

See what happens when 0.105.x is released.

16

That says you didn’t use a venv for at least one of your installs. You may also have installed it in a venv…

hass will use the version of Python the venv was built with. You need to clean up your stray installs, build a venv correctly, and be sure to use that version of hass rather than letting your OS search $PATH to find one.

17

Thats much easier to say than do unfortunately, to remove python3.6
you remove this lot:-

The following packages will be REMOVED:
  apparmor apport apport-gtk aptdaemon apturl apturl-common blueman command-not-found dh-python dirtbike foomatic-db-compressed-ppds gdebi gdebi-core gufw hplip
  hplip-data language-selector-common language-selector-gnome libpython3.6-testsuite lightdm-settings lsb-release mate-dock-applet mate-hud mate-optimus mate-tweak
  netplan.io networkd-dispatcher nplan nvidia-prime onboard onboard-common onboard-data openprinting-ppds orca plymouth-theme-ubuntu-mate-text
  plymouth-theme-ubuntu-text printer-driver-foo2zjs printer-driver-foo2zjs-common printer-driver-m2300w printer-driver-postscript-hp printer-driver-ptouch
  printer-driver-pxljr python3 python3-alabaster python3-all python3-appdirs python3-apport python3-apt python3-aptdaemon python3-aptdaemon.gtk3widgets
  python3-asn1crypto python3-attr python3-babel python3-brlapi python3-cachecontrol python3-cairo python3-certifi python3-cffi-backend python3-chardet python3-colorama
  python3-commandnotfound python3-crypto python3-cryptography python3-cups python3-cupshelpers python3-dateutil python3-dbus python3-debconf python3-debian
  python3-defer python3-distlib python3-distro python3-distro-info python3-distupgrade python3-distutils python3-docutils python3-gdbm python3-gi python3-gi-cairo
  python3-html5lib python3-httplib2 python3-idna python3-imagesize python3-jinja2 python3-keyring python3-keyrings.alt python3-launchpadlib python3-lazr.restfulclient
  python3-lazr.uri python3-lib2to3 python3-lockfile python3-louis python3-mako python3-markupsafe python3-minimal python3-mock python3-netifaces python3-oauth
  python3-olefile python3-packaging python3-pbr python3-pexpect python3-pil python3-pip python3-pkg-resources python3-pluggy python3-problem-report python3-progress
  python3-psutil python3-ptyprocess python3-py python3-pyatspi python3-pygments python3-pyparsing python3-pytest python3-renderpm python3-reportlab
  python3-reportlab-accel python3-requests python3-requests-unixsocket python3-retrying python3-roman python3-scripttest python3-secretstorage python3-setproctitle
  python3-setuptools python3-simplejson python3-six python3-software-properties python3-speechd python3-sphinx python3-systemd python3-tk python3-tz python3-uno
  python3-update-manager python3-urllib3 python3-venv python3-wadllib python3-webencodings python3-wheel python3-xapp python3-xdg python3-xkit python3-xlib
  python3-yaml python3-zope.interface python3.6 python3.6-minimal python3.6-venv redshift-gtk rhythmbox-plugin-alternative-toolbar rhythmbox-plugins sessioninstaller
  slick-greeter snapd software-properties-common software-properties-gtk ssh-import-id system-config-printer system-config-printer-common system-config-printer-udev
  ubuntu-drivers-common ubuntu-mate-core ubuntu-mate-desktop ubuntu-minimal ubuntu-release-upgrader-core ubuntu-release-upgrader-gtk ufw unattended-upgrades
  update-manager update-manager-core update-notifier update-notifier-common xorg xserver-xorg

That does take out most of the python3.6 files, BUT leave you in a position with no X display , no ability to print.
And whats REALLY stupid is Mate doesn’t use python3.6, its still using python2.7
If the HA developers really wanted to do something to help, then name hass with the version of python it was built with.
ie. if built with python3.8 name hass hass3.8.
At least then you can quickly identify which hass to use if more than one exists.
Its NOT a new idea , its done with pip.
If HA upgrades aproximately every 2 weeks and most of the distros, except one, every 6 months ish , there is always going to be problems, as there is a lag on the distros.

Also in ~/.homeasssitant/ I have:-

/home/odroid/.homeassistant/deps/lib/python3.6/site-packages/zeroconf-0.24.0.dist-info/INSTALLER
/home/odroid/.homeassistant/deps/lib/python3.6/site-packages/zeroconf-0.24.0.dist-info/METADATA
/home/odroid/.homeassistant/deps/lib/python3.6/site-packages/zeroconf-0.24.0.dist-info/RECORD
/home/odroid/.homeassistant/deps/lib/python3.6/site-packages/zeroconf-0.24.0.dist-info/WHEEL
/home/odroid/.homeassistant/deps/lib/python3.6/site-packages/zeroconf-0.24.0.dist-info/top_level.txt
/home/odroid/.homeassistant/deps/lib/python3.8/site-packages
/home/odroid/.homeassistant/deps/lib/python3.8/site-packages/Click-7.0.dist-info
/home/odroid/.homeassistant/deps/lib/python3.8/site-packages/Crypto
/home/odroid/.homeassistant/deps/lib/python3.8/site-packages/HAP_python-2.6.0.dist-info
/home/odroid/.homeassistant/deps/lib/python3.8/site-packages/OpenSSL

As I’m running 0.104.3 is it safe to delete the /deps/lib/python3.6 directory ???

And is there any way of determining which version of python hass is built with

odroid@odroid:/srv/homeassistant/bin$ ./hass --version
0.104.3

As hass looks to be the correct version , can hass ver 0.104.3 be built using python3.6 ?
sorry for the questions
TIA

18

STOP

There is no, I repeat NO need to do that. Don’t. Just don’t.

I have a Home Assistant (venv) system with Python 3.4, 3.5, 3.6, 3.7, and 3.8 all installed. There are no issues with getting Home Assistant to use the version of Python I want.

Just look at the first line of it, it’ll tell you.

Honestly though, the devs have no reason to do that. hass is hass. The problem here is that you appear to be massively out of your depth.

19

Well , considering I deleted from ~/homassistant/dep/lib python3.6 and over night all data from darkysky stopped; it indicates that when there are multiple versions of python it does not do a clean build.
removing all python3.6-dev and libpython3.6-dev stops HA from installing
using the --upgrade switch allowed a reinstall using earlier complied under python3.8

The statement of being massively out of my depth is extremely unfair

Or is it now a crime to be new to python, I’m used to C , and files marked as executable are not readable, So it s not unexpected to assume python excuteable files are the same, albeit they are readable. !
readable , with less or cat etc,

20

Nope

I have 5 major versions, never had a problem - the trick is to ensure that you build a clean venv correctly.

A useful command on Linux is file, which will tell you what something is.