Unable to install scipy (+numpy) as component requirement

Setup:

  • Home Assistant os
  • Home Assistant 2021.10.6
  • Intel NUC

I’m making a custom component (“mpc_thermostat”). It depends on scipy, which in turn depends on numpy. During installation (HA restart), I get errors from the installation of scipy (see below).

My interpretation is that scipy depends on numpy, which doesn’t have a prebuilt wheel for my environment (please correct me here!).

I tried to log into the homeassistant container (docker exec -it homeassistant bash) to install it manually. pip install scipy gives me a similar error. At the same time I can see that numpy 1.21.2 is already installed.

This doesn’t make sense to me, so I fear I’m missing something important. Can anyone shed some light on this for me?

Thank you :slightly_smiling_face:

Core log:

2021-11-29 20:49:41 ERROR (SyncWorker_4) [homeassistant.util.package] Unable to install package scipy: ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python3 /usr/local/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-5bw7p03j/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple --find-links https://wheels.home-assistant.io/alpine-3.14/amd64/ --prefer-binary -- 'wheel<0.38.0' 'setuptools<58.0.0' 'Cython>=0.29.18,<3.0' 'pybind11>=2.4.3,<2.8.0' 'pythran>=0.9.12,<0.10.0' 'numpy==1.19.2; python_version=='"'"'3.7'"'"' and platform_machine=='"'"'aarch64'"'"'' 'numpy==1.19.2; python_version=='"'"'3.8'"'"' and platform_machine=='"'"'aarch64'"'"'' 'numpy==1.16.5; python_version=='"'"'3.7'"'"' and platform_machine!='"'"'aarch64'"'"' and platform_python_implementation != '"'"'PyPy'"'"'' 'numpy==1.17.3; python_version=='"'"'3.8'"'"' and (platform_machine!='"'"'arm64'"'"' or platform_system!='"'"'Darwin'"'"') and platform_python_implementation != '"'"'PyPy'"'"'' 'numpy==1.19.3; python_version=='"'"'3.9'"'"' and (platform_machine!='"'"'arm64'"'"' or platform_system!='"'"'Darwin'"'"') and platform_python_implementation != '"'"'PyPy'"'"'' 'numpy==1.21.4; python_version=='"'"'3.10'"'"' and platform_python_implementation != '"'"'PyPy'"'"'' 'numpy==1.20.0; python_version=='"'"'3.7'"'"' and platform_python_implementation=='"'"'PyPy'"'"'' 'numpy; python_version=='"'"'3.8'"'"' and platform_python_implementation=='"'"'PyPy'"'"'' 'numpy; python_version=='"'"'3.9'"'"' and platform_python_implementation=='"'"'PyPy'"'"'' 'numpy; python_version=='"'"'3.10'"'"' and platform_python_implementation=='"'"'PyPy'"'"''
cwd: None
Complete output (264 lines):
Looking in links: https://wheels.home-assistant.io/alpine-3.14/amd64/
Ignoring numpy: markers 'python_version == "3.7" and platform_machine == "aarch64"' don't match your environment
Ignoring numpy: markers 'python_version == "3.8" and platform_machine == "aarch64"' don't match your environment
Ignoring numpy: markers 'python_version == "3.7" and platform_machine != "aarch64" and platform_python_implementation != "PyPy"' don't match your environment
Ignoring numpy: markers 'python_version == "3.8" and (platform_machine != "arm64" or platform_system != "Darwin") and platform_python_implementation != "PyPy"' don't match your environment
Ignoring numpy: markers 'python_version == "3.10" and platform_python_implementation != "PyPy"' don't match your environment
Ignoring numpy: markers 'python_version == "3.7" and platform_python_implementation == "PyPy"' don't match your environment
Ignoring numpy: markers 'python_version == "3.8" and platform_python_implementation == "PyPy"' don't match your environment
Ignoring numpy: markers 'python_version == "3.9" and platform_python_implementation == "PyPy"' don't match your environment
Ignoring numpy: markers 'python_version == "3.10" and platform_python_implementation == "PyPy"' don't match your environment
Collecting wheel<0.38.0
Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting setuptools<58.0.0
Downloading setuptools-57.5.0-py3-none-any.whl (819 kB)
Collecting Cython<3.0,>=0.29.18
Downloading Cython-0.29.24-py2.py3-none-any.whl (979 kB)
Collecting pybind11<2.8.0,>=2.4.3
Downloading pybind11-2.7.1-py2.py3-none-any.whl (200 kB)
Collecting pythran<0.10.0,>=0.9.12
Downloading pythran-0.9.12.post1-py3-none-any.whl (4.3 MB)
Collecting numpy==1.19.3
Downloading numpy-1.19.3.zip (7.3 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python3 /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmplj79anf3
cwd: /tmp/pip-install-7daxi16x/numpy
Complete output (228 lines):
Running from numpy source directory.
setup.py:480: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
run_build = parse_setuppy_commands()
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/mtrand.pyx
Processing numpy/random/_pcg64.pyx
Processing numpy/random/bit_generator.pyx
Processing numpy/random/_sfc64.pyx
Processing numpy/random/_common.pyx
Processing numpy/random/_bounded_integers.pyx.in
Processing numpy/random/_philox.pyx
Processing numpy/random/_generator.pyx
Processing numpy/random/_mt19937.pyx
Cythonizing sources
blas_opt_info:
blas_mkl_info:
customize UnixCCompiler
libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
blis_info:
libraries blis not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_info:
libraries openblas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_3_10_blas_threads_info:
Setting PTATLAS=ATLAS
libraries tatlas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_3_10_blas_info:
libraries satlas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_blas_info:
libraries f77blas,cblas,atlas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
accelerate_info:
NOT AVAILABLE
/tmp/pip-install-7daxi16x/numpy/numpy/distutils/system_info.py:1914: UserWarning:
Optimized (vendor) Blas libraries are not found.
Falls back to netlib Blas library which has worse performance.
A better performance should be easily gained by switching
Blas library.
if self._calc_info(blas):
blas_info:
libraries blas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
/tmp/pip-install-7daxi16x/numpy/numpy/distutils/system_info.py:1914: UserWarning:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
if self._calc_info(blas):
blas_src_info:
NOT AVAILABLE
/tmp/pip-install-7daxi16x/numpy/numpy/distutils/system_info.py:1914: UserWarning:
Blas (http://www.netlib.org/blas/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [blas_src]) or by setting
the BLAS_SRC environment variable.
if self._calc_info(blas):
NOT AVAILABLE
non-existing path in 'numpy/distutils': 'site.cfg'
lapack_opt_info:
lapack_mkl_info:
libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_lapack_info:
libraries openblas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_clapack_info:
libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
flame_info:
libraries flame not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
libraries lapack_atlas not found in /usr/local/lib
libraries tatlas,tatlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib
libraries tatlas,tatlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
NOT AVAILABLE
atlas_3_10_info:
libraries lapack_atlas not found in /usr/local/lib
libraries satlas,satlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib
libraries satlas,satlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_3_10_info'>
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
libraries lapack_atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_threads_info'>
NOT AVAILABLE
atlas_info:
libraries lapack_atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib
libraries f77blas,cblas,atlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_info'>
NOT AVAILABLE
lapack_info:
libraries lapack not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
/tmp/pip-install-7daxi16x/numpy/numpy/distutils/system_info.py:1748: UserWarning:
Lapack (http://www.netlib.org/lapack/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [lapack]) or by setting
the LAPACK environment variable.
return getattr(self, '_calc_info_{}'.format(name))()
lapack_src_info:
NOT AVAILABLE
/tmp/pip-install-7daxi16x/numpy/numpy/distutils/system_info.py:1748: UserWarning:
Lapack (http://www.netlib.org/lapack/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [lapack_src]) or by setting
the LAPACK_SRC environment variable.
return getattr(self, '_calc_info_{}'.format(name))()
NOT AVAILABLE
numpy_linalg_lapack_lite:
FOUND:
language = c
define_macros = [('HAVE_BLAS_ILP64', None), ('BLAS_SYMBOL_SUFFIX', '64_')]
/usr/local/lib/python3.9/distutils/dist.py:274: UserWarning: Unknown distribution option: 'define_macros'
warnings.warn(msg)
running dist_info
running build_src
build_src
building py_modules sources
creating build
creating build/src.linux-x86_64-3.9
creating build/src.linux-x86_64-3.9/numpy
creating build/src.linux-x86_64-3.9/numpy/distutils
building library "npymath" sources
Could not locate executable gfortran
Could not locate executable f95
Could not locate executable ifort
Could not locate executable ifc
Could not locate executable lf95
Could not locate executable pgfortran
Could not locate executable nvfortran
Could not locate executable f90
Could not locate executable f77
Could not locate executable fort
Could not locate executable efort
Could not locate executable efc
Could not locate executable g77
Could not locate executable g95
Could not locate executable pathf95
Could not locate executable nagfor
don't know how to compile Fortran code on platform 'posix'
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
main()
File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/tmp/pip-build-env-zrmqyeqi/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 157, in prepare_metadata_for_build_wheel
self.run_setup()
File "/tmp/pip-build-env-zrmqyeqi/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 248, in run_setup
super(_BuildMetaLegacyBackend,
File "/tmp/pip-build-env-zrmqyeqi/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 142, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 508, in <module>
setup_package()
File "setup.py", line 500, in setup_package
setup(**metadata)
File "/tmp/pip-install-7daxi16x/numpy/numpy/distutils/core.py", line 169, in setup
return old_setup(**new_attr)
File "/tmp/pip-build-env-zrmqyeqi/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 165, in setup
return distutils.core.setup(**attrs)
File "/usr/local/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-zrmqyeqi/overlay/lib/python3.9/site-packages/setuptools/command/dist_info.py", line 31, in run
egg_info.run()
File "/tmp/pip-install-7daxi16x/numpy/numpy/distutils/command/egg_info.py", line 24, in run
self.run_command("build_src")
File "/usr/local/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-install-7daxi16x/numpy/numpy/distutils/command/build_src.py", line 144, in run
self.build_sources()
File "/tmp/pip-install-7daxi16x/numpy/numpy/distutils/command/build_src.py", line 155, in build_sources
self.build_library_sources(*libname_info)
File "/tmp/pip-install-7daxi16x/numpy/numpy/distutils/command/build_src.py", line 288, in build_library_sources
sources = self.generate_sources(sources, (lib_name, build_info))
File "/tmp/pip-install-7daxi16x/numpy/numpy/distutils/command/build_src.py", line 378, in generate_sources
source = func(extension, build_dir)
File "numpy/core/setup.py", line 663, in get_mathlib_info
raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python3 /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmplj79anf3 Check the logs for full command output.
WARNING: You are using pip version 20.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python3 -m pip install --upgrade pip' command.
----------------------------------------

My manifest:

{
  "codeowners": ["joernnilsson"],
  "config_flow": false,
  "dependencies": [],
  "documentation": "https://github.com/joernnilsson/mpc-thermostat",
  "domain": "mpc_thermostat",
  "iot_class": "calculated",
  "name": "MPC Thermostat",
  "requirements": [
    "scipy",
    "numpy",
    "cvxpy",
    "cvxopt"
  ],
  "version": "0.1.0"
}

Note: on my x86_64 desktop with python3.9, scipy/numpy (1.7.2/1.21.4) installs fine.

It tries to build numpy, which needs a compiler which is not installed on the HA docker image.

Try "numpy==1.21.4" which is part of HA requirements.

to build on what @koying is saying, you’ll also need to keep numpy & scipy in sync with what is currently used by all the other integrations that use numpy and scipy.

EDIT: you can search for PRs that bump those dependencies. Compensation uses numpy, so you can check the change history on that to find the current version. I can’t think of any integration that uses scipy off the top of my head.

I’ve just had a simliar error with Numpy, with 2022.8.0b0.
I run in a Python venv.

In my case I soved the issue by running the following in the venv…

pip3 install --ignore-installed numpy==1.23.1 --no-binary :all:
1 Like