What's a quick-and-dirty way to make sure that only one instance of a shell script is running at a given time?
We started with Q&A. Technical documentation is next, and we need your help.
Whether you're a beginner or an experienced developer, you can contribute.
|
|
Here's an implementation that uses a lockfile and echoes a PID into it. This serves as a protection if the process is killed before removing the pidfile:
The trick here is the |
|||||||||||||||||||||
|
|
All approaches that test the existence of "lock files" are flawed. Why? Because there is no way to check whether a file exists and create it in a single atomic action. Because of this; there is a race condition that WILL make your attempts at mutual exclusion break. Instead, you need to use
For all details, see the excellent BashFAQ: http://mywiki.wooledge.org/BashFAQ/045 If you want to take care of stale locks, fuser(1) comes in handy. The only downside here is that the operation takes about a second, so it isn't instant. Here's a function I wrote once that solves the problem using fuser:
You can use it in a script like so:
If you don't care about portability (these solutions should work on pretty much any UNIX box), Linux' fuser(1) offers some additional options and there is also flock(1). |
|||||||||||||||||||||
|
|
Use
This ensures that code between Caveat: this particular command is a part of |
|||||||||||||||||||||
|
|
There's a wrapper around the flock(2) system call called, unimaginatively, flock(1). This makes it relatively easy to reliably obtain exclusive locks without worrying about cleanup etc. There are examples on the man page as to how to use it in a shell script. |
|||||||||
|
|
You need an atomic operation, like flock, else this will eventually fail. But what to do if flock is not available. Well there is mkdir. That's an atomic operation too. Only one process will result in a successful mkdir, all others will fail. So the code is:
You need to take care of stale locks else aftr a crash your script will never run again. |
|||||||||||||||||
|
|
Another option is to use shell's In brief:
This causes the shell to call:
which atomically creates the file or fails if the file already exists. According to a comment on BashFAQ 045, this may fail in
Interesting that How you do the Delete on clean exit New runs fail until the issue that caused the unclean exit to be resolved and the lockfile is manually removed.
Delete on any exit New runs succeed provided the script is not already running.
|
|||||
|
|
To make locking reliable you need an atomic operation. Many of the above proposals are not atomic. The proposed lockfile(1) utility looks promising as the man-page mentioned, that its "NFS-resistant". If your OS does not support lockfile(1) and your solution has to work on NFS, you have not many options.... NFSv2 has two atomic operations:
With NFSv3 the create call is also atomic. Directory operations are NOT atomic under NFSv2 and NFSv3 (please refer to the book 'NFS Illustrated' by Brent Callaghan, ISBN 0-201-32570-5; Brent is a NFS-veteran at Sun). Knowing this, you can implement spin-locks for files and directories (in shell, not PHP): lock current dir:
lock a file:
unlock current dir (assumption, the running process really acquired the lock):
unlock a file (assumption, the running process really acquired the lock):
Remove is also not atomic, therefore first the rename (which is atomic) and then the remove. For the symlink and rename calls, both filenames have to reside on the same filesystem. My proposal: use only simple filenames (no paths) and put file and lock into the same directory. |
|||||||||||||||||||||
|
|
For shell scripts, I tend to go with the Either way, using For proper lock cleanup, you really should set your traps to something like this psuedo code (lifted, simplified and untested but from actively used scripts) :
Here's what will happen. All traps will produce an exit so the function Note: my exit values are not low values. Why? Various batch processing systems make or have expectations of the numbers 0 through 31. Setting them to something else, I can have my scripts and batch streams react accordingly to the previous batch job or script. |
|||||||||||||||||
|
|
Really quick and really dirty? This one-liner on the top of your script will work:
Of course, just make sure that your script name is unique. :) |
|||||
|
|
Create a lock file in a known location and check for existence on script start? Putting the PID in the file might be helpful if someone's attempting to track down an errant instance that's preventing execution of the script. |
|||
|
|
|
This example is explained in the man flock, but it needs some impovements, because we should manage bugs and exit codes:
You can use another method, list processes that I used in the past. But this is more complicated that method above. You should list processes by ps, filter by its name, additional filter grep -v grep for remove parasite nad finally count it by grep -c . and compare with number. Its complicated and uncertain |
|||||
|
|
PID and lockfiles are definitely the most reliable. When you attempt to run the program, it can check for the lockfile which and if it exists, it can use |
|||
|
|
|
Some unixes have From the manpage:
|
|||||||||
|
|
If flock's limitations, which have already been described elsewhere on this thread, aren't an issue for you, then this should work:
|
|||||||||
|
|
Actually although the answer of bmdhacks is almost good, there is a slight chance the second script to run after first checked the lockfile and before it wrote it. So they both will write the lock file and they will both be running. Here is how to make it work for sure:
The P.S. I didn't see that Mikel already answered the question correctly, although he didn't include the trap command to reduce the chance the lock file will be left over after stopping the script with Ctrl-C for example. So this is the complete solution |
|||
|
|
|
Here's an approach that combines atomic directory locking with a check for stale lock via PID and restart if stale. Also, this does not rely on any bashisms.
|
|||
|
|
|
When targeting a Debian machine I find the Here's my solution which uses To use it simply call
|
||||
|
|
|
I find that bmdhack's solution is the most practical, at least for my use case. Using flock and lockfile rely on removing the lockfile using rm when the script terminates, which can't always be guaranteed (e.g., kill -9). I would change one minor thing about bmdhack's solution: It makes a point of removing the lock file, without stating that this is unnecessary for the safe working of this semaphore. His use of kill -0 ensures that an old lockfile for a dead process will simply be ignored/over-written. My simplified solution is therefore to simply add the following to the top of your singleton:
Of course, this script still has the flaw that processes that are likely to start at the same time have a race hazard, as the lock test and set operations are not a single atomic action. But the proposed solution for this by lhunath to use mkdir has the flaw that a killed script may leave behind the directory, thus preventing other instances from running. |
|||
|
|
|
Quick and dirty?
|
|||||||||||||||||
|
|
I use a simple approach that handles stale lock files. Note that some of the above solutions that store the pid, ignore the fact that the pid can wrap around. So - just checking if there is a valid process with the stored pid is not enough, especially for long running scripts. I use noclobber to make sure only one script can open and write to the lock file at one time. Further, I store enough information to uniquely identify a process in the lockfile. I define the set of data to uniquely identify a process to be pid,ppid,lstart. When a new script starts up, if it fails to create the lock file, it then verifies that the process that created the lock file is still around. If not, we assume the original process died an ungraceful death, and left a stale lock file. The new script then takes ownership of the lock file, and all is well the world, again. Should work with multiple shells across multiple platforms. Fast, portable and simple.
|
|||||
|
|
Take a look to FLOM (Free LOck Manager) http://sourceforge.net/projects/flom/: you can synchronize commands and/or scripts using abstract resources that does not need lock files in a filesystem. You can synchronize commands running in different systems without a NAS (Network Attached Storage) like an NFS (Network File System) server. Using the simplest use case, serializing "command1" and "command2" may be as easy as executing:
and
from two different shell scripts. |
||||
|
|
|
why dont we use something like
|
|||
|
|
|
The flock path is the way to go. Think about what happens when the script suddenly dies. In the flock-case you just loose the flock, but that is not a problem. Also, note that an evil trick is to take a flock on the script itself .. but that of course lets you run full-steam-ahead into permission problems. |
|||
|
|
|
Simply add
I have not fully realized how it works, but it seems it runs itself again using itself as a lockfile. It seems to not work on Debian 7, but seems to work back again with experimental util-linux 2.25 package. It writes "flock: ... Text file busy". It could be overridden by disabling write permission on your script. |
||||
|
|
|
Try something like the below,
Then match the variable with 1 using an if loop. |
|||||
|
|
Here is a more elegant, fail-safe, quick Usage
Script Filesh_lock_functions.sh
Usage examplesh_lock_usage_example.sh
Features
|
||||
|
|
|
The semaphoric utility uses It's like sem but much lighter-weight. (Full disclosure: I wrote it after finding the sem was way too heavy for our needs and there wasn't a simple counting semaphore utility available.) |
|||
|
|
|
|||||||||
|
|
I have a simple solution based on the file name
|
||||
|
|
|
You can use
If you want a timeout too, use:
Note that you should give it a name (with
|
|||
|
|