How to use Scikit-learn with a custom intigration?

I am fairly new to home assistant intigration development and i am trying to create my own intigration to use Scikit-learn, but i am having trouble actually getting home assistant to install the pakage. To my understanding any requirments for the intigrations should be put in the manifest.json file under “requirements”. As stated on the home assistant developers page:

Requirements is an array of strings. Each entry is a pip compatible string

So i added:

"requirements": ["scikit-learn==1.0"],

to my manifest.json. However its giving me some kind of installation error:

2023-02-18 15:36:32.413 ERROR (SyncWorker_0) [homeassistant.util.package] Unable to install package scikit-learn==1.0: error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [81 lines of output]
      Looking in links: https://wheels.home-assistant.io/musllinux/
      Ignoring numpy: markers 'python_version == "3.7" and platform_machine != "aarch64" and platform_system != "AIX" and platform_python_implementation != "PyPy"' don't match your environment
      Collecting setuptools
        Downloading setuptools-67.3.2-py3-none-any.whl (1.1 MB)
           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 5.2 MB/s eta 0:00:00
      Collecting wheel
        Using cached wheel-0.38.4-py3-none-any.whl (36 kB)
      Collecting Cython>=0.28.5
        Using cached Cython-0.29.33-py2.py3-none-any.whl (987 kB)
      Collecting oldest-supported-numpy
        Using cached oldest_supported_numpy-2022.11.19-py3-none-any.whl (4.9 kB)
      Collecting scipy>=1.1.0
        Using cached scipy-1.10.0.tar.gz (42.4 MB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'error'
        error: subprocess-exited-with-error
      
        × Preparing metadata (pyproject.toml) did not run successfully.
        │ exit code: 1
        ╰─> [42 lines of output]
            + meson setup --prefix=/usr/local /tmp/pip-install-ythtwkvh/scipy_6deab88ebfa44e318aefb73ba313d71a /tmp/pip-install-ythtwkvh/scipy_6deab88ebfa44e318aefb73ba313d71a/.mesonpy-5nqcsz3s/build --native-file=/tmp/pip-install-ythtwkvh/scipy_6deab88ebfa44e318aefb73ba313d71a/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
            Traceback (most recent call last):
              File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 144, in prepare_metadata_for_build_wheel
                hook = backend.prepare_metadata_for_build_wheel
            AttributeError: module 'mesonpy' has no attribute 'prepare_metadata_for_build_wheel'
      
            During handling of the above exception, another exception occurred:
      
            Traceback (most recent call last):
              File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
                main()
              File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
                json_out['return_val'] = hook(**hook_input['kwargs'])
              File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 148, in prepare_metadata_for_build_wheel
                whl_basename = backend.build_wheel(metadata_directory, config_settings)
              File "/tmp/pip-build-env-3rgaalnr/overlay/lib/python3.10/site-packages/mesonpy/__init__.py", line 1060, in build_wheel
                with _project(config_settings) as project:
              File "/usr/local/lib/python3.10/contextlib.py", line 135, in __enter__
                return next(self.gen)
              File "/tmp/pip-build-env-3rgaalnr/overlay/lib/python3.10/site-packages/mesonpy/__init__.py", line 975, in _project
                with Project.with_temp_working_dir(
              File "/usr/local/lib/python3.10/contextlib.py", line 135, in __enter__
                return next(self.gen)
              File "/tmp/pip-build-env-3rgaalnr/overlay/lib/python3.10/site-packages/mesonpy/__init__.py", line 750, in with_temp_working_dir
                yield cls(source_dir, tmpdir, build_dir, meson_args)
              File "/tmp/pip-build-env-3rgaalnr/overlay/lib/python3.10/site-packages/mesonpy/__init__.py", line 632, in __init__
                self._configure(reconfigure=bool(build_dir) and not native_file_mismatch)
              File "/tmp/pip-build-env-3rgaalnr/overlay/lib/python3.10/site-packages/mesonpy/__init__.py", line 680, in _configure
                self._meson('setup', *setup_args)
              File "/tmp/pip-build-env-3rgaalnr/overlay/lib/python3.10/site-packages/mesonpy/__init__.py", line 657, in _meson
                return self._proc('meson', *args)
              File "/tmp/pip-build-env-3rgaalnr/overlay/lib/python3.10/site-packages/mesonpy/__init__.py", line 652, in _proc
                subprocess.check_call(list(args), env=self._env)
              File "/usr/local/lib/python3.10/subprocess.py", line 364, in check_call
                retcode = call(*popenargs, **kwargs)
              File "/usr/local/lib/python3.10/subprocess.py", line 345, in call
                with Popen(*popenargs, **kwargs) as p:
              File "/usr/local/lib/python3.10/subprocess.py", line 969, in __init__
                self._execute_child(args, executable, preexec_fn, close_fds,
              File "/usr/local/lib/python3.10/subprocess.py", line 1845, in _execute_child
                raise child_exception_type(errno_num, err_msg, err_filename)
            PermissionError: [Errno 13] Permission denied: 'meson'
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: metadata-generation-failed
      
      × Encountered error while generating package metadata.
      ╰─> See above for output.
      
      note: This is an issue with the package mentioned above, not pip.
      hint: See above for details.
      
      [notice] A new release of pip available: 22.3 -> 23.0.1
      [notice] To update, run: pip install --upgrade pip
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip available: 22.3 -> 23.0.1
[notice] To update, run: pip install --upgrade pip

I am getting basicly the same error if i simply use “scikit-learn” without any version number.
Clearly there is something i am missing and i dont understand the error. I am guessing it has something to do with Home assistant and what python version its using. I have no problem installing it with pip on my computer using python 3.10.
Could someone point me in the rigth direction of how to think when using pakages for intigrations. Are there some pakages that wont work with home assistant? If so how do i know which ones ? If scikit is not a viable option are there alternativs I can using machine learning ?

Im also trying to use Scikit-learn and getting the exact same problem. My installation is Home Assistant Supervised. Any toughts on this would be appriciated.
I found this:

it might be a problem with permissions?

Its a frustrating problem. I have everything set up. Data collected, model trained, intigration built. All i am missing is just importing the lib. Tried tensorflow to with the same result.

Did you have found a solution to this?

Hi, yes I’m also interested to hear if you found a solution. I’m trying to do the same, build a custom component with a scikit-learn requirement. See detailed error message below.

2023-12-29 18:05:39.823 ERROR (SyncWorker_6) [homeassistant.util.package] Unable to install package scikit-learn: error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [72 lines of output]
Looking in indexes: https://pypi.org/simple, https://wheels.home-assistant.io/musllinux-index/, https://wheels.home-assistant.io/musllinux-index/
Ignoring numpy: markers 'python_version == "3.10" and platform_system == "Windows" and platform_python_implementation != "PyPy"' don't match your environment
Collecting setuptools
Downloading https://wheels.home-assistant.io/musllinux-index/setuptools-69.0.3-py3-none-any.whl (819 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 819.5/819.5 kB 780.2 kB/s eta 0:00:00
Collecting wheel
Downloading https://wheels.home-assistant.io/musllinux-index/wheel-0.42.0-py3-none-any.whl (65 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.4/65.4 kB 749.0 kB/s eta 0:00:00
Collecting Cython<3.0,>=0.29.33
Downloading https://wheels.home-assistant.io/musllinux-index/Cython-0.29.37-py2.py3-none-any.whl (989 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 989.5/989.5 kB 969.6 kB/s eta 0:00:00
Collecting oldest-supported-numpy
Downloading oldest_supported_numpy-2023.12.21-py3-none-any.whl.metadata (9.8 kB)
Collecting scipy>=1.5.0
Downloading scipy-1.11.4.tar.gz (56.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.3/56.3 MB 6.8 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [33 lines of output]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
whl_basename = backend.build_wheel(metadata_directory, config_settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-wodcro1j/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 1008, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-wodcro1j/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 1072, in build_wheel
with _project(config_settings) as project:
File "/usr/local/lib/python3.11/contextlib.py", line 137, in __enter__
return next(self.gen)
^^^^^^^^^^^^^^
File "/tmp/pip-build-env-wodcro1j/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 951, in _project
yield Project(source_dir, build_dir, meson_args, editable_verbose)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-wodcro1j/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 626, in __init__
_check_meson_version()
File "/tmp/pip-build-env-wodcro1j/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 987, in _check_meson_version
meson_version = subprocess.run(['meson', '--version'], check=False, text=True, capture_output=True).stdout
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/subprocess.py", line 548, in run
with Popen(*popenargs, **kwargs) as process:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/lib/python3.11/subprocess.py", line 1950, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: 'meson'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
2023-12-29 18:07:33.160 ERROR (SyncWorker_6) [homeassistant.util.package] Unable to install package scikit-learn: error: subprocess-exited-with-error

I’m also trying to build a custom component, but the error I’m seeing is this one

2023-12-29 08:12:34.523 ERROR (SyncWorker_1) [homeassistant.util.package] Unable to install package scikit-learn: error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [71 lines of output]
Partial import of sklearn during the build process.
Traceback (most recent call last):
File "/tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/numpy/core/__init__.py", line 23, in <module>
from . import multiarray
File "/tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/numpy/core/multiarray.py", line 10, in <module>
from . import overrides
File "/tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/numpy/core/overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ImportError: Error loading shared library /tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/numpy/core/_multiarray_umath.cpython-311-x86_64-linux-musl.so: Operation not permitted
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 161, in check_package_status
File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/numpy/__init__.py", line 140, in <module>
from . import core
File "/tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/numpy/core/__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.11 from "/usr/local/bin/python3"
* The NumPy version is: "1.23.2"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: Error loading shared library /tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/numpy/core/_multiarray_umath.cpython-311-x86_64-linux-musl.so: Operation not permitted
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 366, in prepare_metadata_for_build_wheel
self.run_setup()
File "/tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 480, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "/tmp/pip-build-env-vqo3v383/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 633, in <module>
File "<string>", line 623, in setup_package
File "<string>", line 188, in check_package_status
ImportError: numpy is not installed.
scikit-learn requires numpy >= 1.17.3.
Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

The strange part is that I’m only seeing this in my production HA not in the devcontainer from GitHub - ludeeus/integration_blueprint: Blueprint for custom_component developers.