Python >=3.5.3 on Synology

synology
Tags: #<Tag:0x00007fd07cc98278>

#61

Hello @holly7787,
Good to see the guide worked out for you, I hope it may help many more Synology users out there in need.


#62

Latest no. Just to 0.75.3 version. Any version >0.75.3 breaks. For newer versions you need to compile some additional files. Below @CharlieDowns provided very nice guide how to do it.


#63

Right now I’m trying all possible ways to install HA at old Synos to see which is more errorless and fluid for not so tech savy users. So I tried your 0,1(1), 2 steps and the result of the command is

Failed to import the site module
Traceback (most recent call last):
File “/usr/local/python3/lib/python3.6/site.py”, line 553, in
main()
File “/usr/local/python3/lib/python3.6/site.py”, line 539, in main
known_paths = addusersitepackages(known_paths)
File “/usr/local/python3/lib/python3.6/site.py”, line 282, in addusersitepackages
user_site = getusersitepackages()
File “/usr/local/python3/lib/python3.6/site.py”, line 258, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File “/usr/local/python3/lib/python3.6/site.py”, line 248, in getuserbase
USER_BASE = get_config_var(‘userbase’)
File “/usr/local/python3/lib/python3.6/sysconfig.py”, line 601, in get_config_var
return get_config_vars().get(name)
File “/usr/local/python3/lib/python3.6/sysconfig.py”, line 550, in get_config_vars
_init_posix(_CONFIG_VARS)
File “/usr/local/python3/lib/python3.6/sysconfig.py”, line 421, in _init_posix
_temp = import(name, globals(), locals(), [‘build_time_vars’], 0)
ModuleNotFoundError: No module named ‘_sysconfigdata_m_linux_arm-linux-gnueabihf’
-ash: import: command not found
-ash: import: command not found
-ash: print: command not found

My setup: Syno 215j latest DSM 6.2.1


#64

Hello @clappingmonkey
Which exact step did you follow in the guide?
From the output of the command I read a installation of Python 3.6 on “/usr/local”, but Synology packages are installed in “/var/packages/”.
The other thing is the current Python version on Synocommunity you get from spksrc is Python 3.5.6, not 3.6.

On my NAS, the DS115j, when I install the compiled Python 3 package, any files in the included Python module packages that contain “x86_64-linux-gnu” should be renamed to “arm-linux-gnueabihf” on install.
In the Python 3 package for example the “cffi-1.11.5-cp35-none-any.whl” module package file contains _cffi_backend.cpython-35m-x86_64-linux-gnu.so.
When installed on my DS115j, the file is copied and renamed to /var/packages/python3/target/lib/python3.5/site-packages/_cffi_backend.cpython-35m-arm-linux-gnueabihf.so, which makes sense since my NAS is ARM based.

Could it be this output was not produced on a Synology NAS or was from Python in Docker?

In the case of which is easier between Docker or on the filesystem, the guide is for people who want to run Home Assistant on the filesystem of their Synology NAS, caused by the unavailability of Docker or a ‘special’ reason, thus for such people Docker is might not be suitable.

…NVM, I am blind, I see now you were replying to a message from @chickenandporn

I can say something helpful though, in the case you do not compile your own package, but use from someone else, that person will need to have specifically added some cross compiled python packages to his Python 3 package by editing requirements.txt, else if you config components such as Homekit, Cloud or even Home Assistant itself, these will fail because these need to compile modules on installation.

This is caused by your Synology not giving the ability for Python to compile these modules, thus why is required to have the Python modules cross compiled (think of it as pre-compiling binaries).

I mean to say, that it is not wrong to use a package made by someone else, but just you use certain components and these fail with errors like Failed building wheel for *Module_Name* you will have to ask the maintainer of the Python package to add that module to “requirements.txt” so you can use your component on his next update.

It is a whole lot easier and faster than compiling on your own though, so if done right, keeping the above in mind, I like it more.


#65

Because python3 > 3.5.3 is now provided, perhaps this should be discussed on a HomeAssistant-native-on-Synology discussion


#66

Python 3.5.6-8 is available from on of the Community repositories, e.g:

http://packages.synocommunity.com/

https://www.cphub.net


#67

Hey there,

I tried a little bit of everything posted here to make the XiaomiAqara gateway component to work on my DS216j.
And I miserably failed.
The main problem is that this component requires Cryptography, that’s why I ended here.

It seems that a significative amount of components now require Cyrptography, Cryptodome, cffi, and so on… which makes it difficult to install them directly on a Syno. Using docker is not a solution or ‘small’ platforms, and even on bigger configurattions (I’ve got a 1515+ at work), on the long run Docker is painfull and requires a lot of ressources for almost nothing.

If I understand well, as long as we have the wheels for the required packages we’d be able to run HA and many components easily and natively.

If I’m not wrong, for those who are equipped with the correct compilers, wouldn’t it be easy to share the wheels compiled for several Syno architectures ?


#68

Which HA version installed on you Synology? And which version Cyrptography, Cryptodome and cffi required?


#69

I built the python3-3.5.6 to be able to install HomeAssistant to run locally without a container nor a Debian chroot. It should have the cffi, crypto (via pycrypto and bcrypt), to get fairly far along: I used it to run HA-0.77.1.

That it’ll work for you is a big assumption because I don’t know about this gateway, but the other issue is to get a specific path forward: something concrete and detailed and repeatable, that we can follow, rather than “a bit of everything”. Now, for my ego, I would suggest starting from the python3-3.5.6 (or the python3-3.6.8 if the current PR gets merged). Then work forward from that, let me know what’s missing, but really, pick something that looks good and repeatable and move from that.

python3 is available at http://spk.chickenandporn.com/ or from SynoCommunity


#70

Hi @Miklosandras,

HA is in version 0.75.3 (the highest I could install with the most advanced version of Python available on SynoCommunity - 3.5.6-8).

XiaomiAqara seems to require at least cryptography in version 2.2.1 and cffi in 1.5.1 (not 100% sure on that last one).


#71

Hi @chickenandporn,

I’ve added your website as a source to syno packages. And I actually installed the 3.6.8 version manually on the NAS. Installation went fine.
I modified some files in my HA installation to point onto python3.6 instead of 3.5.
HA started but stopped early with messages in log saying that some variables in a code were missing (I’m sorry I didn’t keep the file… :frowning: ).
Most of my home automation runs on the NAS and I now dislike to stop it and play with the installation (I suffered a little bit yesterday to get back to python3.5). I know it doesn’t help a lot saying that…
I’ll try to mount a DSM virtual and play with the installation if can help. Using a virtualized DSM, what is the processor architecture that will be virtualized ?


#72

I appreciate your willingness to help; I don’t use a DDSM so I’m not sure. I would expect since it’s Docker-based, it would be the same architecture as the host.

I would much rather see the shortfall in the native non-DDSM install, but I understand how you might just want what you’ve got rather than fiddle around further. Whatever nuanced detail you can tell me about what’s missing from the simplest install – sync packages running natively – would help.


#73

I was on 0.85 and it’s running great (on DS212j)
But when I was trying to upgrade it to 0.86.4 today, there is this package “bcrypt==3.1.5” that is required when upgrading. I had 3.1.4 and it cross compiled fine, but with 3.1.5 it just couldn’t.
When cross compiling it in the debian it kept having this error:

Collecting bcrypt==3.1.6 (from -r /home/parallels/spksrc/spk/python3/work-88f6281-6.1/wheelhouse/requirements.txt (line 16))
  Using cached https://files.pythonhosted.org/packages/ce/3a/3d540b9f5ee8d92ce757eebacf167b9deedb8e30aedec69a2a072b2399bb/bcrypt-3.1.6.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    Complete output from command /home/parallels/spksrc/native/python3/work-native/install/usr/local/bin/python3 /home/parallels/spksrc/native/python3/work-native/install/usr/local/lib/python3.5/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpapwyw0ak:
    Traceback (most recent call last):
      File "/home/parallels/spksrc/native/python3/work-native/install/usr/local/lib/python3.5/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
        main()
      File "/home/parallels/spksrc/native/python3/work-native/install/usr/local/lib/python3.5/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/home/parallels/spksrc/native/python3/work-native/install/usr/local/lib/python3.5/site-packages/pip/_vendor/pep517/_in_process.py", line 69, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/setuptools/build_meta.py", line 125, in prepare_metadata_for_build_wheel
        _run_setup()
      File "/tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/setuptools/build_meta.py", line 85, in _run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 240, in <module>
        **keywords_with_side_effects(sys.argv)
      File "/tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/home/parallels/spksrc/native/python3/work-native/install/usr/local/lib/python3.5/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/setuptools/dist.py", line 447, in __init__
        k: v for k, v in attrs.items()
      File "/home/parallels/spksrc/native/python3/work-native/install/usr/local/lib/python3.5/distutils/dist.py", line 281, in __init__
        self.finalize_options()
      File "/tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/setuptools/dist.py", line 739, in finalize_options
        ep.load()(self, ep.name, value)
      File "/tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/cffi/setuptools_ext.py", line 204, in cffi_modules
        add_cffi_module(dist, cffi_module)
      File "/tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
        execfile(build_file_name, mod_vars)
      File "/tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/cffi/setuptools_ext.py", line 25, in execfile
        exec(code, glob, glob)
      File "src/build_bcrypt.py", line 21, in <module>
        ffi = FFI()
      File "/tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/cffi/api.py", line 46, in __init__
        import _cffi_backend as backend
    ImportError: /tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/_cffi_backend.cpython-35m-x86_64-linux-gnu.so: wrong ELF class: ELFCLASS32
    
    ----------------------------------------
Command "/home/parallels/spksrc/native/python3/work-native/install/usr/local/bin/python3 /home/parallels/spksrc/native/python3/work-native/install/usr/local/lib/python3.5/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpapwyw0ak" failed with error code 1 in /home/parallels/spksrc/spk/python3/work-88f6281-6.1/wheelbuild/bcrypt
../../mk/spksrc.wheel.mk:54: recipe for target 'build_wheel_target' failed
make[1]: *** [build_wheel_target] Error 1
make[1]: Leaving directory '/home/parallels/spksrc/spk/python3'
../../mk/spksrc.spk.mk:452: recipe for target 'arch-88f6281' failed
make: [arch-88f6281] Error 2 (ignored)

seems like this is the main problem?
ImportError: /tmp/pip-build-env-v5uuyh7m/overlay/lib/python3.5/site-packages/_cffi_backend.cpython-35m-x86_64-linux-gnu.so: wrong ELF class: ELFCLASS32

Currently I don’t have enough knowledge to know what to do next~
Any body trying to upgrade to 0.86 having the same problem?


#74

I got the same error when I tried to generate new python package with the following:
pycryptodome==3.7.2
cryptography==2.5
cffi==1.11.5

This is strange…

File "/tmp/pip-build-env-nyaz1oyw/lib/python3.5/site-packages/cffi/api.py", line 46, in __init__
  import _cffi_backend as backend

ImportError: /tmp/pip-build-env-nyaz1oyw/lib/python3.5/site-packages/_cffi_backend.cpython-35m-x86_64-linux-gnu.so: wrong ELF class: ELFCLASS32


Failed building wheel for cryptography


#75

wrong ELF class: ELFCLASS32

Possible you have a dirty buildenv (I’m assuming you’re using spksrc). A simple build completes, but I haven’t found the wheel definition for cryptography nor pycryptodome. bcrypt==3.1.5 is a simple upgrade that I’m building now.

@thy1225 Where can I leave it for you ?


#76

Thanks @chickenandporn, is it okay to put it here? https://uploadfiles.io/ and share the link?
it’s a anonymous file upload website

And you’re right, I’m using spksrc. :slight_smile:
It’s the only way I know so far. (learnt by following the comments)
May I ask what should I look into if I want to learn to setup a clean buildenv?
Thanks :slight_smile:


#77

@chickenandporn if there is a need for testing new python builds or something similar for the needs of running HA natively on Synology I’m more than happy to help. My setup is DS215j with the latest DSM version.


#78

Yesterday I got the exactly same problem, when I wanted to upgrade homeassistant to the latest version. bcrypt==3.1.5 is required to upgrade, but the spksrc is failing with the same error message. Tried compiling with @chickenandporn 's latest python3.6.8 ( https://github.com/SynoCommunity/spksrc/tree/802b3d41a31a03fdb1209f04b4eccf5af41d9b09 ) but, I got the same results… bcrypt==3.1.4 is working fine. I have a fresh new environment to use spksrc

ImportError: /tmp/pip-build-env-unkfulxm/overlay/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so: wrong ELF class: ELFCLASS32

#79

I worry that this might be a dirty build system. The build likes to go from one arch to the next using sentinel files to abbreviate dependency checks, but it might need an aggressive “clean” between each step.

Please, let’s move this discussion to a new thread.


#80

recompiled python 3 and installed on my 218J, but no way to install HASS:

ash-4.3# /volume1/@appstore/python3/bin/python3 -m pip install homeassistant
Failed to import the site module
Traceback (most recent call last):
File “/volume1/@appstore/python3/lib/python3.6/site.py”, line 553, in
main()
File “/volume1/@appstore/python3/lib/python3.6/site.py”, line 539, in main
known_paths = addusersitepackages(known_paths)
File “/volume1/@appstore/python3/lib/python3.6/site.py”, line 282, in addusersitepackages
user_site = getusersitepackages()
File “/volume1/@appstore/python3/lib/python3.6/site.py”, line 258, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File “/volume1/@appstore/python3/lib/python3.6/site.py”, line 248, in getuserbase
USER_BASE = get_config_var(‘userbase’)
File “/volume1/@appstore/python3/lib/python3.6/sysconfig.py”, line 601, in get_config_var
return get_config_vars().get(name)
File “/volume1/@appstore/python3/lib/python3.6/sysconfig.py”, line 550, in get_config_vars
_init_posix(_CONFIG_VARS)
File “/volume1/@appstore/python3/lib/python3.6/sysconfig.py”, line 421, in _init_posix
_temp = import(name, globals(), locals(), [‘build_time_vars’], 0)
ModuleNotFoundError: No module named ‘_sysconfigdata_m_linux_arm-linux-gnueabihf’

maybe I post in the wrong place, but do you known how to solve? Thanks