34

I have a C library, which I build as a shared object for Linux and a DLL for Windows with MinGW32. The API depends on a couple of data files (statistical models) which I'd really like to roll in with the SO/DLL so that deployment is just one file.

It looks like I can achieve this for Windows with a "resource file" compiled with windres, but then I've got to write a bunch of resource-handling code for Windows, and I'm still stuck with the files on Linux.

Is there a way to achieve the same functionality on Linux?

Even better, is there a portable solution?

5
  • 1
    Similar (possible dup): stackoverflow.com/questions/1656968/… and possibly other, but it is a hard topic to search... Jan 4, 2010 at 3:09
  • Not voting for close, though, the answer to the existing question are only marginally useful. Jan 4, 2010 at 3:18
  • It would help enormously if you would tell us what (portable) code you want to write to get access to the data files. fopen and fread simply won't work on blocks of memory, at least not on any system I'm aware of. Jan 5, 2010 at 5:56
  • @Norman: I'm using C++ to implement the library (although it only exports C functions, for ABI compatibility) so I can just construct a stringstream from the binary data and then it acts like an file anyway. It's also worth noting that with avr-libc (and probably other libcs targeted at microcontrollers) you actually can make fread() work on arbitary blocks of memory (or anything else) by supplying your own getc() and putc() via function pointers to the fdevopen() function (fopen() doesn't make much sense without an OS, much less a file system).
    – cibyr
    Jan 5, 2010 at 7:58

2 Answers 2

35

It's actually quite simple on Linux and other ELF systems: http://www.linuxjournal.com/content/embedding-file-executable-aka-hello-world-version-5967

OS X has bundles, so you just build your library as a framework and put the file in the bundle.

8
  • Does this approach work for building Windows DLLs as well? It seems like a good, simple option.
    – cibyr
    Jan 4, 2010 at 4:19
  • I have no idea, I've never built anything for Windows and know nothing about how the linker works. Jan 4, 2010 at 4:41
  • You're still left with the problem that the API for access to data in a file is different from the API for access to a named block in memory. But if you can live with that, you don't need objcopy; you can easily write a short script that generates a .c file from any binary file. E.g., cs.tufts.edu/~nr/drop/lua/lua2c; run the script with the -s option. Jan 4, 2010 at 6:42
  • @Norman: The downside to that is the extra binary→escaped C→binary steps take time and memory; this becomes noticeable with big binary blobs. (Not that you should be embedding them; my preference in all cases would be to stick stuff in $PREFIX/share but apparently OP doesn't like that too much.)
    – ephemient
    Jan 4, 2010 at 22:22
  • The $PREFIX/share solution is fine for UNIX where everybody is used to installing dependencies, but on Windows people really just want a DLL they can redistribute with their app (and there isn't a good equivalent of $PREFIX/share anyway).
    – cibyr
    Jan 5, 2010 at 3:31
-2

Two potential solutions:

  • Phong Vo's sfio library, which is part of the AT&T Advanced Software Technology toolset, is a wonderful replacement for C stdio.h, and it will allow you to open either files or memory blocks using a single API. So you can easily convert your existing files to C initialized data to include in your DLL or SO file.

    This is a good cross-platform solution, but the penalty is that the learning curve to get started is pretty high. They don't make it easy to figure out how stuff works or to take one part of their toolset and split it out for use independent of the other parts. But the good news is that if you want to adopt their U/Win system for running Unix codes on windows (all part of the same toolset), you can create DLLs and SOs using the same system.

  • For this kind of problem I often fall back on Lua; I can stored Lua data either in external files or within C as initialized data. This is great for distributing everything in one .so file; I do this for my students.

    Again the downside is that you have to master and incorporate a new technology.

In my own work I use Lua over the AT&T stuff for these reasons:

  • Lua has a much smaller footprint and is designed to play well with others; with AST you really have do adopt their way of doing things.

  • The learning curve with Lua is much less steep; you can be productive very quickly.

  • Lua is dead easy to install and it's easy to get information about it. AST has its own quirky installation process shared by nobody else in the world; it's often hard to make the installation work; and it's harder to get information about it.

  • Using Lua has a lot of other payoffs, so the effort spent learning Lua and learning how to incorporate Lua into C codes is easy to amortize over multiple projects.

3
  • I don't see how embedding a scripting language solves the problem of embedding arbitrary file data, and if I was going to embed a scripting language I'd probably go with Python (since I've been using Boost.Python a lot recently anyway). sfio looks handy, but looks like it probably wouldn't be worth the effort refactoring everything use it.
    – cibyr
    Jan 5, 2010 at 3:38
  • @cibyr: My asnwer bears on what to embed, not how to embed it. How to embed a file as C code is very, very easy; all the thought goes into the question of what API you want for the bytes. If you look in cs.tufts.edu/~nr/drop/lua you will see a script called lua2c that may give you some ideas. Run with the -s option and it will embed any binary file, although it's not super useful since the API I generated tries to compile the file as Lua code. But once you decide what API you want I'm sure you can do something in Python. If not, post another question. Jan 5, 2010 at 5:54
  • 7
    I'm not sure how this remotely comes close to answer the OP question which "is there a way to add resource files with my executable" (my interpretation). So far the objcopy route seems far more reasonable than embedding lua. Sep 11, 2013 at 17:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.