202

I've always wondered. I know that compilers convert the code you write into binaries but what do linkers do? They've always been a mystery to me.

I roughly understand what 'linking' is. It is when references to libraries and frameworks are added to the binary. I don't understand anything beyond that. For me it "just works". I also understand the basics of dynamic linking but nothing too deep.

Could someone explain the terms?

0

5 Answers 5

247

To understand linkers, it helps to first understand what happens "under the hood" when you convert a source file (such as a C or C++ file) into an executable file (an executable file is a file that can be executed on your machine or someone else's machine running the same machine architecture).

Under the hood, when a program is compiled, the compiler converts the source file into object byte code. This byte code (sometimes called object code) is mnemonic instructions that only your computer architecture understands. Traditionally, these files have an .OBJ extension.

After the object file is created, the linker comes into play. More often than not, a real program that does anything useful will need to reference other files. In C, for example, a simple program to print your name to the screen would consist of:

printf("Hello Kristina!\n");

When the compiler compiled your program into an obj file, it simply puts a reference to the printf function. The linker resolves this reference. Most programming languages have a standard library of routines to cover the basic stuff expected from that language. The linker links your OBJ file with this standard library. The linker can also link your OBJ file with other OBJ files. You can create other OBJ files that have functions that can be called by another OBJ file. The linker works almost like a word processor's copy and paste. It "copies" out all the necessary functions that your program references and creates a single executable. Sometimes other libraries that are copied out are dependent on yet other OBJ or library files. Sometimes a linker has to get pretty recursive to do its job.

Note that not all operating systems create a single executable. Windows, for example, uses DLLs that keep all these functions together in a single file. This reduces the size of your executable, but makes your executable dependent on these specific DLLs. DOS used to use things called Overlays (.OVL files). This had many purposes, but one was to keep commonly used functions together in 1 file (another purpose it served, in case you're wondering, was to be able to fit large programs into memory. DOS has a limitation in memory and overlays could be "unloaded" from memory and other overlays could be "loaded" on top of that memory, hence the name, "overlays"). Linux has shared libraries, which is basically the same idea as DLLs (hard core Linux guys I know would tell me there are MANY BIG differences).

4
  • 15
    Great answer. Additionally most modern linkers will remove redundant code like template instantiations. Jul 23, 2010 at 23:52
  • 1
    Is this an appropriate place to go over some of those differences?
    – John P
    Oct 1, 2015 at 19:31
  • 2
    Hi, Suppose my file does not reference any other file. Suppose I simply declare and initialize two variables. Will this source file also go to the linker? May 13, 2016 at 3:13
  • 4
    @MangeshKherdekar - Yes, it always goes through a linker. The linker might not link any external libraries, but the linking phase still has to happen to produce an executable.
    – Icemanind
    May 13, 2016 at 14:59
153

Address relocation minimal example

Address relocation is one of the crucial functions of linking.

So let's have a look on how it works with a minimal example.

0) Introduction

Summary: relocation edits the .text section of object files to translate:

  • object file address
  • into the final address of the executable

This must be done by the linker because the compiler only sees one input file at a time, but we must know about all object files at once to decide how to:

  • resolve undefined symbols like declared undefined functions
  • not clash multiple .text and .data sections of multiple object files

Prerequisites: minimal understanding of:

Linking has nothing to do with C or C++ specifically: compilers just generate the object files. The linker then takes them as input without ever knowing what language compiled them. It might as well be Fortran.

So to reduce the crust, let's study a NASM x86-64 ELF Linux hello world:

section .data
    hello_world db "Hello world!", 10
section .text
    global _start
    _start:

        ; sys_write
        mov rax, 1
        mov rdi, 1
        mov rsi, hello_world
        mov rdx, 13
        syscall

        ; sys_exit
        mov rax, 60
        mov rdi, 0
        syscall

compiled and assembled with:

nasm -o hello_world.o hello_world.asm
ld -o hello_world.out hello_world.o

with NASM 2.10.09.

1) .text of .o

First we decompile the .text section of the object file:

objdump -d hello_world.o

which gives:

0000000000000000 <_start>:
   0:   b8 01 00 00 00          mov    $0x1,%eax
   5:   bf 01 00 00 00          mov    $0x1,%edi
   a:   48 be 00 00 00 00 00    movabs $0x0,%rsi
  11:   00 00 00
  14:   ba 0d 00 00 00          mov    $0xd,%edx
  19:   0f 05                   syscall
  1b:   b8 3c 00 00 00          mov    $0x3c,%eax
  20:   bf 00 00 00 00          mov    $0x0,%edi
  25:   0f 05                   syscall

the crucial lines are:

   a:   48 be 00 00 00 00 00    movabs $0x0,%rsi
  11:   00 00 00

which should move the address of the hello world string into the rsi register, which is passed to the write system call.

But wait! How can the compiler possibly know where "Hello world!" will end up in memory when the program is loaded?

Well, it can't, specially after we link a bunch of .o files together with multiple .data sections.

Only the linker can do that since only he will have all those object files.

So the compiler just:

  • puts a placeholder value 0x0 on the compiled output
  • gives some extra information to the linker of how to modify the compiled code with the good addresses

This "extra information" is contained in the .rela.text section of the object file

2) .rela.text

.rela.text stands for "relocation of the .text section".

The word relocation is used because the linker will have to relocate the address from the object into the executable.

We can disassemble the .rela.text section with:

readelf -r hello_world.o

which contains;

Relocation section '.rela.text' at offset 0x340 contains 1 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
00000000000c  000200000001 R_X86_64_64       0000000000000000 .data + 0

The format of this section is fixed documented at: http://www.sco.com/developers/gabi/2003-12-17/ch4.reloc.html

Each entry tells the linker about one address which needs to be relocated, here we have only one for the string.

Simplifying a bit, for this particular line we have the following information:

  • Offset = C: what is the first byte of the .text that this entry changes.

    If we look back at the decompiled text, it is exactly inside the critical movabs $0x0,%rsi, and those that know x86-64 instruction encoding will notice that this encodes the 64-bit address part of the instruction.

  • Name = .data: the address points to the .data section

  • Type = R_X86_64_64, which specifies what exactly what calculation has to be done to translate the address.

    This field is actually processor dependent, and thus documented on the AMD64 System V ABI extension section 4.4 "Relocation".

    That document says that R_X86_64_64 does:

    • Field = word64: 8 bytes, thus the 00 00 00 00 00 00 00 00 at address 0xC

    • Calculation = S + A

      • S is value at the address being relocated, thus 00 00 00 00 00 00 00 00
      • A is the addend which is 0 here. This is a field of the relocation entry.

      So S + A == 0 and we will get relocated to the very first address of the .data section.

3) .text of .out

Now lets look at the text area of the executable ld generated for us:

objdump -d hello_world.out

gives:

00000000004000b0 <_start>:
  4000b0:   b8 01 00 00 00          mov    $0x1,%eax
  4000b5:   bf 01 00 00 00          mov    $0x1,%edi
  4000ba:   48 be d8 00 60 00 00    movabs $0x6000d8,%rsi
  4000c1:   00 00 00
  4000c4:   ba 0d 00 00 00          mov    $0xd,%edx
  4000c9:   0f 05                   syscall
  4000cb:   b8 3c 00 00 00          mov    $0x3c,%eax
  4000d0:   bf 00 00 00 00          mov    $0x0,%edi
  4000d5:   0f 05                   syscall

So the only thing that changed from the object file are the critical lines:

  4000ba:   48 be d8 00 60 00 00    movabs $0x6000d8,%rsi
  4000c1:   00 00 00

which now point to the address 0x6000d8 (d8 00 60 00 00 00 00 00 in little-endian) instead of 0x0.

Is this the right location for the hello_world string?

To decide we have to check the program headers, which tell Linux where to load each section.

We disassemble them with:

readelf -l hello_world.out

which gives:

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000400000 0x0000000000400000
                 0x00000000000000d7 0x00000000000000d7  R E    200000
  LOAD           0x00000000000000d8 0x00000000006000d8 0x00000000006000d8
                 0x000000000000000d 0x000000000000000d  RW     200000

 Section to Segment mapping:
  Segment Sections...
   00     .text
   01     .data

This tells us that the .data section, which is the second one, starts at VirtAddr = 0x06000d8.

And the only thing on the data section is our hello world string.

Bonus level

4
22

In languages like 'C', individual modules of code are traditionally compiled separately into blobs of object code, which is ready to execute in every respect other than that all the references that module makes outside itself (i.e. to libraries or to other modules) have not yet been resolved (i.e. they're blank, pending someone coming along and making all the connections).

What the linker does is to look at all the modules together, look at what each module needs to connect to outside itself, and look at all the things it is exporting. It then fixes that all up, and produces a final executable, which can then be run.

Where dynamic linking is also going on, the output of the linker is still not capable of being run - there are still some references to external libraries not yet resolved, and they get resolved by the OS at the time it loads the app (or possibly even later during the run).

3
  • It's worth noting that some assemblers or compilers can output an executable file directly if the compiler "sees" everything necessary (typically in a single source file plus anything it #includes). A few compilers, typically for small micros, have that as their only mode of operation.
    – supercat
    Jul 23, 2010 at 23:27
  • Yes, I tried to give a middle-of-the-road answer. Of course, as well as your case the opposite is true too, in that some kinds of object file don't even have the full code-generation done; that's done by the linker (that's how MSVC whole-program-optimisation works).
    – Will Dean
    Jul 25, 2010 at 19:41
  • @WillDean and GCC's Link-Time Optimisation, far as I can tell - it streams all the 'code' as GIMPLE intermediate language with required metadata, makes that available to the linker, and optimises in one go at the end. (Despite what outdated documentation implies, only GIMPLE is now streamed by default, rather than the old 'fat' mode with both representations of the object code.) Mar 14, 2016 at 21:17
16

When the compiler produces an object file, it includes entries for symbols that are defined in that object file, and references to symbols that aren't defined in that object file. The linker takes those and puts them together so (when everything works right) all the external references from each file are satisfied by symbols that are defined in other object files.

It then combines all those object files together and assigns addresses to each of the symbols, and where one object file has an external reference to another object file, it fills in the address of each symbol wherever it's used by another object. In a typical case, it'll also build a table of any absolute addresses used, so the loader can/will "fix up" the addresses when the file is loaded (i.e., it'll add the base load address to each of those addresses so they all refer to the correct memory address).

Quite a few modern linkers can also carry out some (in a few cases a lot) of other "stuff", such as optimizing the code in ways that are only possible once all of the modules are visible (e.g., removing functions that were included because it was possible that some other module might call them, but once all the modules are put together it's apparent that nothing ever calls them).

-3

The linker links your OBJ file with this standard library.

1
  • Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – Community Bot
    Sep 15, 2022 at 3:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.