3

I am developing a grader for programming contests. Basically, the grader must run the solution program in an 'isolated' process. So, I would like the solution not to call any harming system calls (such as system(), fork(), etc.). Can I use ptrace() to achieve that?

2 Answers 2

1

I think there are 2 possible solutions:

  1. Using the LD_PRELOAD mechanism to create 'shim' to replace the system calls you want to stop.
  2. Use setrlimit() to limit what the calling process can do. Unfortunately these limits seem to be a per-user, not per-process, basis, which makes calculating the correct value to set very difficult.

EDIT: I have the first option working, and have included the necessary code below. Build binaries using make all and then test with make runtests:

$ make all
gcc -fPIC -shared -Wl,-soname,libmy.so.1 -o libmy.so.1.0 lib.c
ln -sf libmy.so.1.0 libmy.so.1
ln -sf libmy.so.1 libmy.so
gcc -o test test.c

$ make runtests
Without LD_PRELOAD:
./test
in child: retval=9273
in parent: retval=0
With LD_PRELOAD:
LD_PRELOAD=./libmy.so ./test
libmy.so fork!
fork error: error=Operation not permitted (1)

Makefile:

all: libs test

runtests:
    @echo Without LD_PRELOAD:
    ./test
    @echo With LD_PRELOAD:
    LD_PRELOAD=./libmy.so ./test


libs: lib.c
    gcc -fPIC -shared -Wl,-soname,libmy.so.1 -o libmy.so.1.0 lib.c
    ln -sf libmy.so.1.0 libmy.so.1
    ln -sf libmy.so.1 libmy.so

test: test.c
    gcc -o test test.c

clean:
    rm -f test libmy.so.1.0 libmy.so.1 libmy.so lib.o

lib.c:

#include <stdio.h>
#include <unistd.h>
#include <errno.h>

pid_t fork()
{
    printf("libmy.so fork!\n");
    errno = EPERM;
    return (pid_t)-1;
}

test.c:

#include <stdio.h>
#include <string.h>
#include <errno.h>

int main(int argc, char **argv)
{
    int retval = fork();
    if (retval == 0)
        printf("in parent: retval=%d\n", retval);
    else if (retval > 0)
        printf("in child: retval=%d\n", retval);
    else
        printf("fork error: error=%s (%d)\n", strerror(errno), errno);
    return 0;
}
8
  • Thank you, but can you explain a little bit more about 'LD_PRELOAD mechanism' ?
    – fushar
    Jan 16, 2011 at 12:04
  • If you set the $LD_PRELOAD environment variable to point to a shared library then that library will be used in preference to those set within the executable. I don't know if it works for system-level functions or not though.
    – trojanfoe
    Jan 16, 2011 at 12:54
  • Okay, then the second option: what resource in setrlimit() do I need to set to prevent the system calls?
    – fushar
    Jan 16, 2011 at 13:07
  • RLIMIT_NPROC. I am currently implementing a test version of the first option, but it doesn't look good so far. I will update my answer with the results.
    – trojanfoe
    Jan 16, 2011 at 13:19
  • 4
    Be aware that LD_PRELOAD does not stop the program from making direct system calls such as via int 0x80. It is not designed as a security measure.
    – jilles
    Jan 16, 2011 at 20:14
0

Yes you can use ptrace() to block certain syscalls using the PTRACE_SYSCALL option. Here is a project using this feature:

https://github.com/t00sh/p-sandbox/blob/master/p-sandbox.c

If you only target Linux, I would suggest seccomp instead, which is a faster technique, to white list/blacklist certain syscalls or restrict their arguments.

An alternative would be Google's Native Client. This projects provides a cross-platform implementation of an application sandbox.

You could also run the applications as an unprivileged user within a container such as docker or LXC to limit the damage.

Using LD_PRELOAD alone is insecure as executable may come with their own syscall implementation and bypass the underlying libc.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.