6

If my app loads (using dlopen) a .so file, that is linking to the Boost Test Framework, I can't unload the so file. Without linking to boost it seems to be fine to unload it.

App file main.cpp:

#include <dlfcn.h>
#include <iostream>

int main()
{
   auto sFileName = "./libtest_library.so";
   auto handle = dlopen(sFileName, RTLD_LAZY | RTLD_LOCAL);

   if (!handle) 
      std::cerr << "Error: " << dlerror() << std::endl;

   auto closing = dlclose(handle);
   while(1);
   return 0;
}

Library .so file (libtest_library.so):

#include <iostream>
//#include "boost/test/unit_test.hpp"

static void con() __attribute__((constructor));
static void dcon() __attribute__((destructor));

void con()
{
   std::cout << "Constructing library..." << std::endl;
}

void dcon()
{
   std::cout << "Destructing library..." << std::endl;
}

Running this I get the output:

Constructing library...
Destructing library...

If I link to Boost's unit test framework in libtest_library.so I only get the Constructing library... output. dlclose(handle) returns 0 (which is success).

Currently linking against Boost v. 1.60.0, compiling with gcc 5_2_0 on Ubuntu 14.04. Is this a bug in Boost? Compiler? Any ideas?

I need to reload the .so files several times in a project, and it needs to be fully unloaded (not existing in the memory). How can I solve this? Thanks.


Update 1:
It seems like if I only link to boost the libtest_library destructor is actually called but the boost_test_framework library is not unloaded. However, if I include "boost/test/unit_test.hpp", the destructor won't be called (libtest_library.so refuse to unload).

Update 2:
Looking through boost's sources I’ve found that there is a c++ singleton in boost causing the problem.

I can replicate the problem in a simplified version. Basically if I add the following singleton to the libtest_library it doesn’t work (can’t unload the .so file):

alt 1

class Singleton
{
public:
   static Singleton & getInstance() { static Singleton instance; return instance; }    
private:
   Singleton() {}
   ~Singleton() {}
};

static Singleton & singleton = Singleton::getInstance();

But using this works:

alt 2

class Singleton
{
public:
    static Singleton & getInstance();
private:
   Singleton() {}
   ~Singleton() {}
};

Singleton & Singleton::getInstance() { static Singleton instance; return instance; }

static Singleton & singleton = Singleton::getInstance();

I’ve tried the different GCC compilers and it leads to the same results. For me this seems to be bug?

Also the symbols are a bit different: doing nm –C libtest_library.so | grep –i singleton I get

alt 1 (not working one):

0000000000201460 u guard variable for Singleton::getInstance()::instance
0000000000201458 b singleton
0000000000000e66 W Singleton::getInstance()
0000000000000f08 W Singleton::Singleton()
0000000000000f08 W Singleton::Singleton()
0000000000000f1c W Singleton::~Singleton()
0000000000000f1c W Singleton::~Singleton()
0000000000201468 u Singleton::getInstance()::instance

And alt 2:

00000000002012f8 b guard variable for Singleton::getInstance()::instance
0000000000201300 b singleton
0000000000000bb0 T Singleton::getInstance()
0000000000000cec W Singleton::Singleton()
0000000000000cec W Singleton::Singleton()
0000000000000d00 W Singleton::~Singleton()
0000000000000d00 W Singleton::~Singleton()
0000000000201308 b Singleton::getInstance()::instance

Any Ideas?

Update 3

I've extracted out the part in boost that seems to create the problem and created a minimal example that demonstrate the problem:

main_app.cpp - main app

#include <dlfcn.h>
#include <iostream>

int main()
{
   for(auto i = 0; i < 2; i++) {
      auto sFileName = "./libtest_library.so";
      auto handle = dlopen(sFileName, RTLD_LAZY | RTLD_LOCAL);

      if (!handle) {
         printf("Dlerror: %s\n", dlerror());
         continue;
      }

      auto closing = dlclose(handle);
      printf("Dlerror: %s\n", dlerror());
   }

   return 0;
}

main_lib.cpp - libtest_library.so

#include <iostream>

template<typename Derived>
class trivial_singleton_t {
public:
   static Derived& instance() { static Derived the_inst; return the_inst; }
protected:
   trivial_singleton_t() {}
   ~trivial_singleton_t() {}
};

class singleton_t : public trivial_singleton_t<singleton_t> {

private:
   friend class trivial_singleton_t<singleton_t>;
   singleton_t() {}
};

singleton_t & singleton = singleton_t::instance();

static void con() __attribute__((constructor));
static void dcon() __attribute__((destructor));

void con()
{
   std::cout << "Constructing library..." << std::endl;
}

void dcon()
{
   std::cout << "Destructing library..." << std::endl;
}

I get the following output:

Constructing library...
Dlerror: (null)
DLerror: (null)
Destructing library...

hence the library is only unloaded when main exists.

6
  • Is the library of interest pulled in somehow with the different linking combination? Can you use ldd to look at the executable and see if the dl library is on the list? Otherwise, have you tried the RTLD_NOLOAD option to see if the library is already loaded? Did dlerror give any information? The dlclose will not unload a library if other libraries are linking symbols against it. I suspect there is a link dependency somehow when you include the different header Commented Aug 10, 2016 at 18:50
  • Hi @MatthewFisher. The dl library is linked corrently. I've double checked with gdb to see if it picked up the correct dl library during runtime. The library is not loaded before (I doubled checked using gdb). dlerror() is not returning any errors... Please see update 2 for more information. Any more ideas?
    – A.Fagrell
    Commented Aug 11, 2016 at 11:07
  • I see three possibilities. It is an obscure bug in the dynamic linker. Or the name Singleton is somehow used in multiple place and the linker is being conservative, try changing the class name to nochance_of_collsion_Singleton or some such. The 'u' classification of the symbols is causing the linker to leave them in memory to be certain of their uniqueness. Maybe try to create a minimal example that the rest of us can compile and analyze, sounds like you are close to that. Commented Aug 11, 2016 at 12:10
  • @MatthewFisher hmm no success so far :/. I've added a "minimal example" you can change the auto keywords to its type if you'd like to build with a c++98 compiler. Do you get the same result?
    – A.Fagrell
    Commented Aug 11, 2016 at 16:00
  • A.Fargrell was the answer below helpful? If so, I'd appreciate it if you would accept the answer, I'm trying to build my reputation. I've never seen this issue before, interesting question! Commented Aug 15, 2016 at 13:47

2 Answers 2

8

As noted in the question there are STB_GNU_UNIQUE symbols in the compiled binary files.

The problem is that a library that is using those symbols and loaded with dlopen will be flagged as NODELETE, and therefore persist between the dlopen/dlclose calls. See line 445 here: http://osxr.org:8080/glibc/source/elf/dl-lookup.c

https://sourceware.org/binutils/docs/binutils/nm.html, STB_GNU_UNIQUE or u:

The symbol is a unique global symbol. This is a GNU extension to the standard set of ELF symbol bindings. For such a symbol the dynamic linker will make sure that in the entire process there is just one symbol with this name and type in use.

These are made when variables/methods are created within an anonymous namespace or as static global variables.

The quickest solution is to force the compiler to not build those symbols as STB_GNU_UNIQUE using a linker flag --no-gnu-unique.

Unfortunately this didn't work for me since I didn't have a sufficiently recent linker, luckily I could rebuild gcc with the following configuration option: --disable-gnu-unique-object. Remember to also rebuild the boost library using the linker flag or new gcc.

1

The circular dependency created by template trivial_singleton_t depending on singleton_t and vice versa is confusing the runtime linking. Here is a simpler example.

#include <iostream>

template<class T>
struct base
{
   static T& method() { static T t; return t;} 
};

struct derived:public base<derived>
{

};


static void con() __attribute__((constructor));
static void dcon() __attribute__((destructor));

void never_called()
{
   base<derived>::method();
}

void con()
{
   std::cout << "Constructing library..." << std::endl;
}

void dcon()
{
   std::cout << "Destructing library..." << std::endl;
}

It seems that the compile time creation of the template type will lead to the circular dependency. The example does not actually create an instance at runtime.

Here is a hack workaround that delegates the singleton to the derived class. Not too useful for the singleton example but shows a possible way to break the cycle.

template<class T>
struct base
{
   static T& method() { return T::delegate(); }
};


struct derived;

derived * d_ptr;

struct derived:public base<derived>
{
   friend class base<derived>;
   static derived& delegate() { if(d_ptr == nullptr) d_ptr = new derived; return * d_ptr; }
};


static void con() __attribute__((constructor));
static void dcon() __attribute__((destructor));

void never_called()
{
   derived & d = base<derived>::method();
}

I'm not really sure if this is a bug or not. Seems to be an edge case. Really cool question!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.