2

I realized a little C# porting of this application that allows to load libraries from memory/stream instead of using LoadLibrary API function that works through filesystem. After messing around a little bit with pointers and mimatching results... finally I have something working as intended. The only problem I have is that the call to DLLMain always fails (I tried it with Kernel32.dll and User32.dll). I can't understand why and I don't know how to debug the issue.

Here is the main function of my project (a simple 32bit console application) which reads a library, allocates it into memory and loads it manually:

public static UInt32 Load(String libraryName)
{
    if (libraries.ContainsKey(libraryName))
        return libraries[libraryName];

    String libraryPath = Path.Combine(Environment.SystemDirectory, libraryName);
    Byte[] libraryBytes = File.ReadAllBytes(libraryPath);

    fixed (Byte* libraryPointer = libraryBytes)
    {
        HeaderDOS* headerDOS = (HeaderDOS*)libraryPointer;

        if ((UInt16)((headerDOS->Magic << 8) | (headerDOS->Magic >> 8)) != IMAGE_DOS_SIGNATURE)
            return 0;

        HeadersNT* headerNT = (HeadersNT*)(libraryPointer + headerDOS->LFANEW);

        UInt32 addressLibrary = VirtualAlloc(headerNT->OptionalHeader.ImageBase, headerNT->OptionalHeader.SizeOfImage, AllocationType.RESERVE, MemoryProtection.READWRITE);

        if (addressLibrary == 0)
            addressLibrary = VirtualAlloc(0, headerNT->OptionalHeader.SizeOfImage, AllocationType.RESERVE, MemoryProtection.READWRITE);

        if (addressLibrary == 0)
            return 0;

        Library* library = (Library*)Marshal.AllocHGlobal(sizeof(Library));
        library->Address = (Byte*)addressLibrary;
        library->ModulesCount = 0;
        library->Modules = null;
        library->Initialized = false;

        VirtualAlloc(addressLibrary, headerNT->OptionalHeader.SizeOfImage, AllocationType.COMMIT, MemoryProtection.READWRITE);

        UInt32 addressHeaders = VirtualAlloc(addressLibrary, headerNT->OptionalHeader.SizeOfHeaders, AllocationType.COMMIT, MemoryProtection.READWRITE);

        MemoryCopy((Byte*)headerDOS, (Byte*)addressHeaders, (headerDOS->LFANEW + headerNT->OptionalHeader.SizeOfHeaders));

        library->Headers = (HeadersNT*)((Byte*)addressHeaders + headerDOS->LFANEW);
        library->Headers->OptionalHeader.ImageBase = addressLibrary;

        CopySections(library, headerNT, libraryPointer);

        UInt32 locationDelta = addressLibrary - headerNT->OptionalHeader.ImageBase;

        if (locationDelta != 0)
            PerformBaseRelocation(library, locationDelta);

        UInt32 libraryHandle = (UInt32)library;

        if (!BuildImportTable(library))
        {
            Free(libraryName);
            return 0;
        }

        FinalizeSections(library);

        if (library->Headers->OptionalHeader.AddressOfEntryPoint == 0)
        {
            Free(libraryName);
            return 0;
        }

        UInt32 libraryEntryPoint = addressLibrary + library->Headers->OptionalHeader.AddressOfEntryPoint;

        if (libraryEntryPoint == 0)
        {
            Free(libraryName);
            return 0;
        }

        LibraryMain main = (LibraryMain)Marshal.GetDelegateForFunctionPointer(new IntPtr(libraryEntryPoint), typeof(LibraryMain));
        UInt32 result = main(addressLibrary, DLL_PROCESS_ATTACH, 0);

        if (result == 0)
        {
            Free(libraryName);
            return 0;
        }

        library->Initialized = true;

        libraries[libraryName] = libraryHandle;

        return libraryHandle;
    }
}

And here is an example on how to use it:

private const Byte VK_Z_BREAK = 0x5A;

[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate void KeyboardEventDelegate(Byte key, Byte scan, KeyboardFlags flags, Int32 extra);

[Flags]
private enum KeyboardFlags : uint
{
    EXTENDEDKEY = 0x0001,
    KEYUP = 0x0002,
}

public static void Main()
{
    UInt32 libraryHandle = LibraryLoader.Load("User32.dll");

    if (libraryHandle != 0)
    {
        UInt32 functionHandle = LibraryLoader.GetFunctionAddress("User32.dll", "keybd_event");

        if (functionHandle != 0)
        {
            KeyboardEventDelegate s_KeyboardEvent = (KeyboardEventDelegate)Marshal.GetDelegateForFunctionPointer(new IntPtr(functionHandle), typeof(KeyboardEventDelegate));

            while (true)
            {
                s_KeyboardEvent(VK_Z_BREAK, VK_Z_SCANCODE, 0, 0);
                s_KeyboardEvent(VK_Z_BREAK, VK_Z_SCANCODE, KeyboardFlags.KEYUP, 0);
            }
        }
    }

    Console.ReadLine();
}

If you want to give it a fast try, you can download the project from this link.

[EDIT] After a few tries, using Marshal.GetLastWin32Error() just after calling DllMain, I discovered that an error code 14 is being produced, which corresponds to ERROR_OUTOFMEMORY. If I go on after the DllMain call failure, and I get the address of a library function, trying to call it using a delegate produces a PInvokeStackImbalance exception. Any clues about this? ^_^

7
  • 1
    wouldn't it be easier (also for maintainance) to not port it completely but provide a small C++/CLI layer on top of the original project?
    – stijn
    Jan 4, 2013 at 14:23
  • 3
    do not post all your code here.. only post the portion that's relevant if you have all your logic in a single method .. then I would suggest refactoring it.. I don't have time to look at any of your code if it's over 10-15 lines..
    – MethodMan
    Jan 4, 2013 at 14:23
  • What does your DLLMain do, and what does it return as the result? Does it get called at all (e.g. can you do an OutoutDebugString from within DLLMain and see the output)? Jan 4, 2013 at 22:32
  • As per my thread edit, DllMain is called with a delegate "main(addressLibrary, DLL_PROCESS_ATTACH, 0)", and the result is 0, NULL or whatever you want it to be. The call with DLL_PROCESS_ATTACH fails and the last Win32 error after that line of code is 14 (ERROR_OUTOFMEMORY). Jan 4, 2013 at 22:56
  • C++/CLI is what you want. Also, why are you trying to load kernel32 from memory? That's crazy. It's loaded into every single process. And I bet your process also has user32 loaded by the loader. Try using a trivial DLL of your own making. Jan 6, 2013 at 15:26

1 Answer 1

9
+100

This code is only a first-order approximation of what the Windows loader does to load a DLL. It can only work for the simplest of DLLs, the translation from C to C# code is also highly likely to cause trouble like the stack imbalance problem you are dealing with. The chief problems I see:

  • It doesn't do anything to ensure that the DLL wasn't already loaded before. This is pretty much guaranteed to be a source of trouble when you try to load kernel32.dll and user32.dll, those DLLs are already loaded before managed code starts executing. They will not take kindly to getting loaded again.

  • It doesn't do anything obvious to ensure that dependent DLLs get loaded as well and that their DllMain() entrypoints are called in the correct order and strictly serialized.

  • It doesn't do anything to properly deal with the managed code loader stub, MSCoree.dll, which makes it very unlikely you can ever properly load any DLLs that contain mixed mode code.

  • It doesn't do anything to ensure that the Windows loader is aware of these modules, which makes it very likely that any subsequent request for the DLL will fail. Such failure is quite undiagnosable.

The likelihood that you'll be able to address these problems correctly is quite low.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.