I’m having problems with ld and NVIDIA CUDA on 12.2.
I installed the current cuda 5 release and gcc 4.6. To make nvcc use gcc 4.6, I created a directory in my home, linked gcc, g++, cpp and c++ to the correct 4.6 versions. Then I set the compiler-bindir option in nvcc.profile to use that directory. Compiling works, but linking fails for a simple example (vectorAdd):
schmuker@neuroinf03:~/Apps/cuda-5.0/samples/0_Simple/vectorAdd> make
/home/schmuker/Apps/cuda-5.0/bin/nvcc -m64 -I/home/schmuker/Apps/cuda-5.0/include -gencode arch=compute_10,code=sm_10 -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -o vectorAdd.o -c vectorAdd.cu
g++ -m64 -o vectorAdd vectorAdd.o -L/home/schmuker/Apps/cuda-5.0/lib64 -lcudart
/usr/lib64/gcc/x86_64-suse-linux/4.7/../../../../x86_64-suse-linux/bin/ld: warning: libdl.so.2, needed by /home/schmuker/Apps/cuda-5.0/lib64/libcudart.so, not found (try using -rpath or -rpath-link)
/usr/lib64/gcc/x86_64-suse-linux/4.7/../../../../x86_64-suse-linux/bin/ld: warning: libpthread.so.0, needed by /home/schmuker/Apps/cuda-5.0/lib64/libcudart.so, not found (try using -rpath or -rpath-link)
/usr/lib64/gcc/x86_64-suse-linux/4.7/../../../../x86_64-suse-linux/bin/ld: warning: librt.so.1, needed by /home/schmuker/Apps/cuda-5.0/lib64/libcudart.so, not found (try using -rpath or -rpath-link)
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `pthread_setspecific@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `pthread_key_delete@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `dlopen@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `sem_trywait@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `sem_post@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `sem_wait@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `dlclose@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `dlsym@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `sem_init@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `pthread_join@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `pthread_key_create@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `pthread_detach@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `pthread_create@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `sem_destroy@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `sem_timedwait@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `pthread_mutexattr_settype@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `pthread_mutexattr_init@GLIBC_2.2.5'
/home/schmuker/Apps/cuda-5.0/lib64/libcudart.so: undefined reference to `pthread_getspecific@GLIBC_2.2.5'
This is strange, because ldd shows nothing suspicious for libcudart:
schmuker@neuroinf03:~/Apps/cuda-5.0/samples/0_Simple/vectorAdd> ldd /home/schmuker/Apps/cuda-5.0/lib64/libcudart.so
linux-vdso.so.1 (0x00007ffffd1ff000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fc3ccb16000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc3cc8f9000)
librt.so.1 => /lib64/librt.so.1 (0x00007fc3cc6f1000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007fc3cc3ba000)
libm.so.6 => /lib64/libm.so.6 (0x00007fc3cc0c2000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fc3cbeac000)
libc.so.6 => /lib64/libc.so.6 (0x00007fc3cbb07000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc3ccf75000)
The interesting thing is, everything works fine when I add -Wl,-rpath,/lib64 to the CCFLAGS in the Makefile.
But adding the path to LD_LIBRARY_PATH doesn’t change anything.
Unfortunately adding CCFLAGS += -Wl,-rpath,/lib64 to nvcc.profile doesn’t seem to get through to the linker either.
It’s impractical to edit every Makefile to add CCFLAGS += -Wl,-rpath,/lib64, especially because we’re using pycuda and stuff. Anyway, I don’t understand in the first place why the linker needs the rpath parameter to find libraries in the /lib64 location.
Any ideas what might be going wrong here?
Thanks in advance!