Had some time today and figured why it worked. You basically used the old kernel files (the modules themselves) and compiled them (cross compiled) for the platform without using any kernel version information. And because the modules likely did not change (or better said the system calls in the modules are the same in the 4.4.59 and 4.4.180) and the kernel cannot check the module version (actually it just throws a warning that kernel version information is not present in the module) then it just works loading it.
Still waiting for Synology to publish the real kernel sources so I can properly compile them.
PS: A small tip for who wants to load @robertklep modules with modprobe: You need to run depmod -a but that is not present (at least not on my NAS). It is enough to create a symlink called depmod towards the real executable called kmod and then you can run it. That will allow you to use modprobe instead of insmod (which will load all the other needed kernel modules automatically, ie usbcore, if that is not loaded already)