Installing external library/package appdaemon wont work

Trying to install numpy for Appdaemon and i get this error when i add “numpy” in the python packages in the config file. I’m running Hassio. Does anyone have an idea why it’s not working? I tried installing gfortran but doesn’t work either…

LOGFILE:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-banner.sh: executing... 

-----------------------------------------------------------
 Hass.io Add-on: AppDaemon v1.6.0

 Python Apps and HADashboard using AppDaemon 3.x for Home Assistant

 From: Community Hass.io Add-ons
 By: Franck Nijhof <[email protected]>
-----------------------------------------------------------
 amd64 / Ubuntu 18.04.1 LTS / HA 0.84.6 / SU 141 / stable
-----------------------------------------------------------
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing... 
Log level is set to INFO
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] 02-updates.sh: executing... 

INFO: You are running the latest version of this add-on
[cont-init.d] 02-updates.sh: exited 0.
[cont-init.d] 20-init-configuration.sh: executing... 
[cont-init.d] 20-init-configuration.sh: exited 0.
[cont-init.d] 21-compiled-dir.sh: executing... 
[cont-init.d] 21-compiled-dir.sh: exited 0.
[cont-init.d] 30-auto-token.sh: executing... 
INFO: Updating Hass.io API token in AppDaemon with the current one...
[cont-init.d] 30-auto-token.sh: exited 0.
[cont-init.d] 31-ha-url.sh: executing... 
[cont-init.d] 31-ha-url.sh: exited 0.
[cont-init.d] 50-compiled-symlink.sh: executing... 
[cont-init.d] 50-compiled-symlink.sh: exited 0.
[cont-init.d] 80-system-packages.sh: executing... 
[cont-init.d] 80-system-packages.sh: exited 0.
[cont-init.d] 81-python-packages.sh: executing...
Collecting hass-apps Downloading https://files.pythonhosted.org/packages/1f/96/87290ba0f95d861f32b6ff8789e71315b5d960f149d3d3c416846b04de76/hass_apps-0.20181223.1-py3-none-any.whl (75kB) Requirement already satisfied: voluptuous&gt;=0.11 in /usr/lib/python3.6/site-packages (from hass-apps) (0.11.5) Collecting observable&gt;=1.0 (from hass-apps) Downloading https://files.pythonhosted.org/packages/45/7c/b4b63f447378e8a0ebcd338d90f9389f57fb23253127425beacf0129edcb/observable-1.0.3-py2.py3-none-any.whl Requirement already satisfied: appdaemon&gt;=3.0 in /usr/lib/python3.6/site-packages (from hass-apps) (3.0.2) Requirement already satisfied: yarl==1.1.0 in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (1.1.0) Requirement already satisfied: paho-mqtt in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (1.4.0) Requirement already satisfied: Jinja2==2.10 in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (2.10) Requirement already satisfied: daemonize in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (2.4.7) Requirement already satisfied: aiohttp-jinja2==0.15.0 in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (0.15.0) Requirement already satisfied: iso8601 in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (0.1.12) Requirement already satisfied: websocket-client in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (0.54.0) Requirement already satisfied: feedparser in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (5.2.1) Requirement already satisfied: requests&gt;=2.6.0 in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (2.20.0) Requirement already satisfied: astral in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (1.7.1) Requirement already satisfied: pyyaml in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (3.13) Requirement already satisfied: aiohttp==2.3.10 in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (2.3.10) Requirement already satisfied: sseclient in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (0.0.19) Requirement already satisfied: bcrypt in /usr/lib/python3.6/site-packages (from appdaemon&gt;=3.0-&gt;hass-apps) (3.1.4) Requirement already satisfied: multidict&gt;=4.0 in /usr/lib/python3.6/site-packages (from yarl==1.1.0-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (4.4.2) Requirement already satisfied: idna&gt;=2.0 in /usr/lib/python3.6/site-packages (from yarl==1.1.0-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (2.7) Requirement already satisfied: MarkupSafe&gt;=0.23 in /usr/lib/python3.6/site-packages (from Jinja2==2.10-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (1.0) Requirement already satisfied: six in /usr/lib/python3.6/site-packages (from websocket-client-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (1.11.0) Requirement already satisfied: certifi&gt;=2017.4.17 in /usr/lib/python3.6/site-packages (from requests&gt;=2.6.0-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (2018.10.15) Requirement already satisfied: chardet&lt;3.1.0,&gt;=3.0.2 in /usr/lib/python3.6/site-packages (from requests&gt;=2.6.0-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (3.0.4) Requirement already satisfied: urllib3&lt;1.25,&gt;=1.21.1 in /usr/lib/python3.6/site-packages (from requests&gt;=2.6.0-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (1.24) Requirement already satisfied: pytz in /usr/lib/python3.6/site-packages (from astral-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (2018.7) Requirement already satisfied: async_timeout&gt;=1.2.0 in /usr/lib/python3.6/site-packages (from aiohttp==2.3.10-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (3.0.1) Requirement already satisfied: idna-ssl&gt;=1.0.0 in /usr/lib/python3.6/site-packages (from aiohttp==2.3.10-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (1.1.0) Requirement already satisfied: cffi&gt;=1.1 in /usr/lib/python3.6/site-packages (from bcrypt-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (1.11.5) Requirement already satisfied: pycparser in /usr/lib/python3.6/site-packages (from cffi&gt;=1.1-&gt;bcrypt-&gt;appdaemon&gt;=3.0-&gt;hass-apps) (2.19) Installing collected packages: observable, hass-apps Successfully installed hass-apps-0.20181223.1 observable-1.0.3 You are using pip version 10.0.1, however version 18.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Collecting numpy

running install
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building py_modules sources
creating build
creating build/src.linux-x86_64-3.6
creating build/src.linux-x86_64-3.6/numpy
creating build/src.linux-x86_64-3.6/numpy/distutils
building library "npymath" sources
get_default_fcompiler: matching types: '['gnu95', 'intel', 'lahey', 'pg', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor']'
customize Gnu95FCompiler
Could not locate executable gfortran
Could not locate executable f95
customize IntelFCompiler
Could not locate executable ifort
Could not locate executable ifc
customize LaheyFCompiler
Could not locate executable lf95
customize PGroupFCompiler
Could not locate executable pgfortran
customize AbsoftFCompiler
Could not locate executable f90
Could not locate executable f77
customize NAGFCompiler
customize VastFCompiler
customize CompaqFCompiler
Could not locate executable fort
customize IntelItaniumFCompiler
Could not locate executable efort
Could not locate executable efc
customize IntelEM64TFCompiler
customize GnuFCompiler
Could not locate executable g77
customize G95FCompiler
Could not locate executable g95
customize PathScaleFCompiler
Could not locate executable pathf95
customize NAGFORCompiler
Could not locate executable nagfor
don't know how to compile Fortran code on platform 'posix'
C compiler: gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Os -fomit-frame-pointer -g -Os -fomit-frame-pointer -g -Os -fomit-frame-pointer -g -DTHREAD_STACK_SIZE=0x100000 -fPIC

compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -c'
gcc: _configtest.c
/bin/sh: gcc: not found
/bin/sh: gcc: not found
failure.
removing: _configtest.c _configtest.o
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-rkxcmibl/numpy/setup.py", line 403, in <module>
    setup_package()
  File "/tmp/pip-install-rkxcmibl/numpy/setup.py", line 395, in setup_package
    setup(**metadata)
  File "/tmp/pip-install-rkxcmibl/numpy/numpy/distutils/core.py", line 169, in setup
    return old_setup(**new_attr)
  File "/usr/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-rkxcmibl/numpy/numpy/distutils/command/install.py", line 62, in run
    r = self.setuptools_run()
  File "/tmp/pip-install-rkxcmibl/numpy/numpy/distutils/command/install.py", line 36, in setuptools_run
    return distutils_install.run(self)
  File "/usr/lib/python3.6/distutils/command/install.py", line 545, in run
    self.run_command('build')
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-rkxcmibl/numpy/numpy/distutils/command/build.py", line 47, in run
    old_build.run(self)
  File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-rkxcmibl/numpy/numpy/distutils/command/build_src.py", line 148, in run
    self.build_sources()
  File "/tmp/pip-install-rkxcmibl/numpy/numpy/distutils/command/build_src.py", line 159, in build_sources
    self.build_library_sources(*libname_info)
  File "/tmp/pip-install-rkxcmibl/numpy/numpy/distutils/command/build_src.py", line 294, in build_library_sources
    sources = self.generate_sources(sources, (lib_name, build_info))
  File "/tmp/pip-install-rkxcmibl/numpy/numpy/distutils/command/build_src.py", line 377, in generate_sources
    source = func(extension, build_dir)
  File "numpy/core/setup.py", line 666, in get_mathlib_info
    raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program

----------------------------------------

You are going to need to install at least gcc in the addon container, probably more.

Thanks. I made it a bit further. I added gcc to system packages but still get som errors. I can’t seem to understand whats missing. Tried taking an older package of numpy. Didn’t make any difference…

running build
    running config_cc
    unifing config_cc, config, build_clib, build_ext, build commands --compiler options
    running config_fc
    unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
    running build_src
    build_src
    building py_modules sources
    creating build
    creating build/src.linux-x86_64-3.6
    creating build/src.linux-x86_64-3.6/numpy
    creating build/src.linux-x86_64-3.6/numpy/distutils
    building library "npymath" sources
    customize Gnu95FCompiler
    Found executable /usr/bin/gfortran
    customize Gnu95FCompiler
    customize Gnu95FCompiler using config
    C compiler: gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Os -fomit-frame-pointer -g -Os -fomit-frame-pointer -g -Os -fomit-frame-pointer -g -DTHREAD_STACK_SIZE=0x100000 -fPIC
    
    compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -c'
    gcc: _configtest.c
    gcc _configtest.o -o _configtest
    /usr/lib/gcc/x86_64-alpine-linux-musl/6.4.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find Scrt1.o: No such file or directory
    /usr/lib/gcc/x86_64-alpine-linux-musl/6.4.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find crti.o: No such file or directory
    /usr/lib/gcc/x86_64-alpine-linux-musl/6.4.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lssp_nonshared
    collect2: error: ld returned 1 exit status
    /usr/lib/gcc/x86_64-alpine-linux-musl/6.4.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find Scrt1.o: No such file or directory
    /usr/lib/gcc/x86_64-alpine-linux-musl/6.4.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find crti.o: No such file or directory
    /usr/lib/gcc/x86_64-alpine-linux-musl/6.4.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lssp_nonshared
    collect2: error: ld returned 1 exit status
    failure.
    removing: _configtest.c _configtest.o
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-4wp7k9oy/numpy/setup.py", line 391, in <module>
        setup_package()
      File "/tmp/pip-install-4wp7k9oy/numpy/setup.py", line 383, in setup_package
        setup(**metadata)
      File "/tmp/pip-install-4wp7k9oy/numpy/numpy/distutils/core.py", line 169, in setup
        return old_setup(**new_attr)
      File "/usr/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-4wp7k9oy/numpy/numpy/distutils/command/install.py", line 62, in run
        r = self.setuptools_run()
      File "/tmp/pip-install-4wp7k9oy/numpy/numpy/distutils/command/install.py", line 36, in setuptools_run
        return distutils_install.run(self)
      File "/usr/lib/python3.6/distutils/command/install.py", line 545, in run
        self.run_command('build')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-4wp7k9oy/numpy/numpy/distutils/command/build.py", line 47, in run
        old_build.run(self)
      File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-4wp7k9oy/numpy/numpy/distutils/command/build_src.py", line 148, in run
        self.build_sources()
      File "/tmp/pip-install-4wp7k9oy/numpy/numpy/distutils/command/build_src.py", line 159, in build_sources
        self.build_library_sources(*libname_info)
      File "/tmp/pip-install-4wp7k9oy/numpy/numpy/distutils/command/build_src.py", line 294, in build_library_sources
        sources = self.generate_sources(sources, (lib_name, build_info))
      File "/tmp/pip-install-4wp7k9oy/numpy/numpy/distutils/command/build_src.py", line 377, in generate_sources
        source = func(extension, build_dir)
      File "numpy/core/setup.py", line 665, in get_mathlib_info
        raise RuntimeError("Broken toolchain: cannot link a simple C program")
    RuntimeError: Broken toolchain: cannot link a simple C program
    
    ----------------------------------------
Command "/usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-4wp7k9oy/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-awy1mhuh/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-4wp7k9oy/numpy/
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
FATAL: Failed installing package numpy==1.12.0
[cont-init.d] 81-python-packages.sh: exited 1.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] 50-compiled-symlink.sh: executing... 
[cont-finish.d] 50-compiled-symlink.sh: exited 0.
[cont-finish.d] 99-message.sh: executing... 
-----------------------------------------------------------
                Oops! Something went wrong.
 
 We are so sorry, but something went terribly wrong when
 starting or running this add-on.
 
 Be sure to check the log above, line by line, for hints.
-----------------------------------------------------------
[cont-finish.d] 99-message.sh: exited 0.
[cont-finish.d] done.
[s6-finish] syncing disks.
[s6-finish] sending all processes the TERM signal.

That is a missing library. You could go on like this for hours. It doesn’t seem practical to do what you are doing. Can you run appdaemon somewhere else?

What do you mean by somewhere else? In another docker container?

Another docker container might work, but you would still have to build a docker image containing everything you need. Running in a virtual environment on a host would be easier.

With hassio, you are going to have to do all this installation every time you start a new container, which doesn’t seem practical.

May i propose the jupyterlab extension for hassio ?
I use it as part of my degree, though i yet have to do some serious crunching via the extension…
May work better in your case.

That sound a bit too complicated for me. It’s just i simple script… But maybe that’s the only way. Thanks for the help!

I will definitively check that out. Didn’t knew it existed.

Hi again. I solved it with these config parameters. One problem is that it takes 20 minutes to start appdaemon and install numpy. Numpy takes like 19 of those minutes… That is okey though, since i seldom need to restart AP.

{
  "log_level": "info",
  "system_packages": [
    "python3-dev",
    "gcc",
    "musl-dev"
  ],
  "python_packages": [
    "numpy"
  ]
}
1 Like

I didn’t realise you could install system packages as part of the configuration, but does it take that long on every restart or just the first until the next upgrade?

Just during the start of appdaemon. You can change scripts etc and it wont need a restart. I tried installing scipy also but that didn’t work (but maybe with a few additional packages it will :wink: ) so this is maybe not the best solution - but it works for numpy.

HI, So i’ve installed a packaged called python-miio,
the code works when executed from python but fails in appdaemon.

import miio
TypeError: subcon should be a Construct field
    raise TypeError("subcon should be a Construct field")
  File "/usr/lib/python3.8/site-packages/construct/core.py", line 291, in __init__
    super(Const, self).__init__(subcon)
  File "/usr/lib/python3.8/site-packages/construct/core.py", line 1894, in __init__
    Const(0x2131, Int16ub),
  File "/usr/lib/python3.8/site-packages/miio/protocol.py", line 206, in 
    from .protocol import Message
  File "/usr/lib/python3.8/site-packages/miio/device.py", line 14, in 
    from .device import Device, DeviceException
  File "/usr/lib/python3.8/site-packages/miio/airconditioningcompanion.py", line 8, in 
    from miio.airconditioningcompanion import (
  File "/usr/lib/python3.8/site-packages/miio/__init__.py", line 2, in 
    import miio
  File "/config/appdaemon/apps/hello.py", line 10, in 
  File "", line 219, in _call_with_frames_removed
  File "", line 783, in exec_module
  File "", line 671, in _load_unlocked
  File "", line 975, in _find_and_load_unlocked
  File "", line 991, in _find_and_load
  File "", line 1014, in _gcd_import
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    self.modules[module_name] = importlib.import_module(module_name)
  File "/usr/lib/python3.8/site-packages/appdaemon/app_management.py", line 580, in read_app
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    response = future.result()
  File "/usr/lib/python3.8/site-packages/appdaemon/utils.py", line 276, in run_in_executor
    await utils.run_in_executor(self, self.read_app, mod["name"], mod["reload"])
  File "/usr/lib/python3.8/site-packages/appdaemon/app_management.py", line 788, in check_app_updates
2020-02-07 16:41:09.195064 WARNING Error: Traceback (most recent call last):
2020-02-07 16:41:09.191033 WARNING Error: ------------------------------------------------------------
2020-02-07 16:41:09.190778 WARNING Error: Unexpected error loading module: /config/appdaemon/apps/hello.py:
2020-02-07 16:41:09.187373 WARNING Error: ------------------------------------------------------------
2020-02-07 01:54:17.108397 WARNING Error: ------------------------------------------------------------