1

Background: I have application which part is used as library for other independent application. They link to that library (lets say lib.so) in linking time. Problem with such approach is that we have to use same external libraries like boost, ace etc or we will have duplicated symbols which in the end will cause crash. We want to resolve this problem.

I know two techniques - one is hiding all symbols (not sure about orders of scopes global/local for shared library) and other is to use dynamic linking. We chose 2nd options (dynamic linking) as it gave client opportunity to do easy testing with stubbed lib.so. and we have very simple api.

I wrote below small example of application which load example shared library and it crash (I want to understand why it crashed and how it should be written). Crash is in dlopen, exactly in initialization of global variable on assignment to std::string (constructor of Aclass type). From our testing it looks that any access to std library while ongoing initialization of library will result in crash.

We managed to remove crash by adding -fPIC flag to EXECUTABLE (why this resolved our problem, I thought it should be set for shared library, may anyone explain me that more precisely)? Unnecessary to my understanding this flag is problematic as it slow down application and in my case (low latency applications) it is quite problematic.

To summary: 1. Why this crash occur? 2. Why -fPIC flag is enough to resolve this crash? 3. Why it is enough to set -fPIC flag to executable? 4. Is it possible to resolve my problem in other way so shared library and client application could use different versions of libraries (like boost, ace etc, compiler, linux version and std libraries are guarantee to be same)? 5. Removing flag RTLD_DEEPBIND will fix crash too but from gcc man it looks that I should use this flag as it will change symbol scope order for shared library - first it will search symbols in local scope then in global - looks as must have for me as shared library will use different external libraries than executable (and dynamic loading will protect executable with polluting its symbol scope). Why removing this flag fix crash in this simple case?

Shared library dynLib.cpp:

#include <string>
class Aclass
{
    std::string s;
    s = "123";
}

Aclass a;

Exacutable main.cpp:

#include <stdlib.h>
#include <dlfcn.h>
#include <string>
#include <unistd.h>
#include <iostream>

int main()
{
    std::string dummyCrasher;
    dlerror();
    void* handle = dlopen("./libdynLib.so", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);
    if(!handle)
    {
        std::cout << "handle is null" << dlerror();
    }

    usleep(1000 * 1000 * 10);
}  

Makefile: makefile

CXXFLAGS=-m32 -march=x86-64 -Wl,v -g -O3 -Wformat -Werror=format -c
CLINKFLAGS=-Wl,-Bstatic -Wl,Bdynamic -ldl -m32 -march=x86-64

all: dynLib.so dynamiclinking

dynLib.so: dynLib.o
    g++44 $(CLINKFLAGS) -shared -o libdynLib.so dynLib.o

dynLib.o: dynLib.cpp
    g++44 $(CXXFLAGS) dynLib.cpp

dynamiclinking: main.o
    g++44 $(CLINKFLAGS) -o dynamiclinking main.o -ldl

main.o: main.cpp
    g++44 (CXXFLAGS) main.cpp

.PHONY: clean
clean:
    rm dynLib.o main.o dynamiclinking libdynLib.so

PS. I write that code by hand (could did some spell errors) PS 2. with -fPIC flag it will work:

main.o: main.cpp
    g++44 (CXXFLAGS) main.cpp -fPIC

UPDATE It is possible to resolve this problem by static linkage of libstdc++. But still my questions are not answered :( Maybe someone have some time to look at it?

UPDATE2 Same problem occur on GCC 4.4.6 and 4.8.1.

1
  • I recently had a similar problem and found this bug report: gcc.gnu.org/bugzilla/show_bug.cgi?id=42679 It does not give a full solution, but hopefully some insight into the problem at least.
    – Filip
    Commented Nov 16, 2017 at 16:51

1 Answer 1

1

I think you are facing the same problem as in When we are supposed to use RTLD_DEEPBIND?, where the executable gets a copy of global variables:

well that's a wonderful feature of you building the main application without the -fPIC option. [...] This means that when the symbol is found in libdep.so, it gets copied into the initial data segment of the main executable at that address. Then the reference to duplicate in libdep.so is looked up and it points to the copy of the symbol that's in the main executable.

Due to RTLD_DEEPBIND, dynLib.so is seeing the wrong set of global variables from original libstdc++ when initializing std::string and thus crashes.

As for why the linker has such behavior, this article has a detailed explanation (emphasis mine):

Recall that the program/executable is not relocatable, and thus its data addresses have to bound at link time. Therefore, the linker has to create a copy of the variable in the program's address space, and the dynamic loader will use that as the relocation address. This is similar to the discussion in the previous section - in a sense, myglob in the main program overrides the one in the shared library, and according to the global symbol lookup rules, it's being used instead.

One final note: this behavior is platform-specific, at least on PowerPC there is no such additional copy of global variables in main executable.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.