5

I am compiling a message passing program using openmpi with mpicxx on a Linux desktop. My makefile does the following:

mpicxx -c readinp.cpp
mpicxx -o exp_fit driver.cpp readinp.o 

at which point i get the following error:

/usr/lib64/gcc/x86_64-suse-linux/4.5/../../../../x86_64-suse-linux/bin/ld: cannot find -lnuma

My questions are:

what is -lnuma? what is using it? how should i go about linking to it?

4 Answers 4

7

Thanks Jonathan Dursi!

On Ubuntu, the package name is libnuma-dev.
apt-get install libnuma-dev

4

The build script can't find the numa library - NUMA (Non Uniform Memory Access). The -l option tells the linker to link the library, but your system ether doesn't have the right one installed or your search path for the linker is incomplete/wrong.

Try querying your package-manager (apt or rpm) for a package libnuma.

2
  • i know of others who have compiled using OpenMPI on this system. is it possible that my user account simply doesn't have access? do i need to add the path to where the library is located? if so, how do i find said path?
    – drjrm3
    Commented Jan 30, 2012 at 15:22
  • It may depend on your distro where you'll find the stuff. Most distros seprate the binaries and headers in to packages like 'libnuma' and 'libnuma-devel'. Try searching for both. If the are installesd, you should be able to query the paths (e.g. rpm -ql libnuma or dpkg -L libnuma).
    – Mithrandir
    Commented Jan 30, 2012 at 15:26
2

OpenMPI, and I think mpich2, uses libnuma (`a simple programming interface to the NUMA (Non Uniform Memory Access) policy supported by the Linux kernel') for memory affinity -- to ensure that the memory for a particular MPI task stays close to the core that the task is running on, as vs. being kept in cache on another socket entirely. This is important for performance on multicore nodes.

You may need to use YaST to install libnuma-devel if your linker can't find the library.

2

I got the same error working on a remote server, which had the NUMA library installed. In particular, the file /usr/lib64/libnuma.so.1 existed. It appears that the linker only looked for the file under the name libnuma.so. Creating the symlink

ln -s /usr/lib64/libnuma.so.1 /usr/lib64/libnuma.so

as described here might have worked, but in my case I did not have permission to create files in /usr/lib64. I got around this by creating the symlink in some other location of which I have write permission:

ln -s /usr/lib64/libnuma.so.1 /some/path/libnuma.so

and then add this path to the compilation flags. In your case this would be

mpicxx -L/some/path -o exp_fit driver.cpp readinp.o

In my case of a larger build process (compiling fftw), I added the path to the LDFLAGS environment variable,

export LDFLAGS="${LDFLAGS} -L/some/path"

which fixed the issue.

1
  • 1
    the right fix is to have your sysadmin install the devel package (libnuma-dev on ubuntu, numactl-devel in RHEL, ...). libnuma is used to build Open MPI, but most MPI apps are unlikely to use it directly (which means the libnuma.so.1 should be enough at runtime) Commented Dec 17, 2018 at 23:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.