I have a bunch of different R processes (independently launched from the command line) that all need to load different big files. To avoid clogging the network, I want to add a lock / semaphore, e.g. via a lock file, so that they get their file one after the other. Only one process should be able to acquire the lock, on a standard Linux system.
-
What benefits do you expect from implementing this? How many processes are there, and how big are the files? Shouldn't this rather be implemented by the file server?– krlmlrMar 26, 2013 at 13:33
-
1I run 15 processes on the same server, file sizes are up to 3 GB. I observed waves of processes that wait for their files (CPU wasted), followed by times where all of them are computing (bandwidth wasted). Staggering the load times leads to more efficient CPU and network use. The fileserver tries to fulfill all requests at once.– Michael KuhnMar 26, 2013 at 14:03
2 Answers
While I couldn't find an R package, there is the Linux command lockfile that can be used:
write("Attempting to get lock", stderr())
system("lockfile /tmp/my_simple_lock")
# Do stuff
write("Releasing lock", stderr())
system("rm -f /tmp/my_simple_lock")
-
2instead of
system("rm -f /tmp/my_simple_lock")you can also usefile.remove("/tmp/my_simple_lock")– blueblobNov 22, 2013 at 15:21
A maybe more explicit version of the previous answer. If the file you are accessing is called filename, and you are operating in a single directory... (if you use different directories, then obvious replace filename with the entire path)
write("Attempting to get lock", stderr())
system("lockfile filename.lock")
\# do whatever you want to the file
write("Releasing lock", stderr())
remove.file("filename.lock")