I have checked for a solution here but cannot seem to find one. I am dealing with a very slow wan connection about 300kb/sec. For my downloads I am using a remote box, and then I am downloading them to my house. I am trying to run a cronjob that will rsync two directories on my remote and local server every hour. I got everything working but if there is a lot of data to transfer the rsyncs overlap and end up creating two instances of the same file thus duplicate data sent.

I want to instead call a script that would run my rsync command but only if rsync isn't running?

share|improve this question
    
Here's a similar answer about a single shell script instance, but it takes the answer you selected and makes it more robust. stackoverflow.com/questions/185451/… – physicsmichael Feb 27 '13 at 2:07
up vote 7 down vote accepted

Via the script you can create a "lock" file. If the file exists, the cronjob should skip the run ; else it should proceed. Once the script completes, it should delete the lock file.

if [ -e /home/myhomedir/rsyncjob.lock ]
then
  echo "Rsync job already running...exiting"
  exit
fi

touch /home/myhomedir/rsyncjob.lock

#your code in here

#delete lock file at end of your job

rm /home/myhomedir/rsyncjob.lock
share|improve this answer
    
ok, im really new to the whole scripting thing, found this:codeif ( set -o noclobber; echo "locked" > "$lockfile") 2> /dev/null; then trap 'rm -f "$lockfile"; exit $?' INT TERM EXIT echo "Locking succeeded" >&2 rm -f "$lockfile" else echo "Lock failed - exit" >&2 exit 1 fi \code – mfpockets Feb 22 '12 at 6:55
    
It does not need to be that complex. Simply create a lock file. Use if -e to check existence of the file and if it exists exit ; else proceed. – user Feb 22 '12 at 6:59
    
Thanks, this will help once I can figure out how to do lock files properly. :( not helpful for a newbie. – mfpockets Feb 22 '12 at 15:03
    
Added logic above. – user Feb 22 '12 at 16:39
    
That looks simpler than I thought. I will try this tonight. – mfpockets Feb 22 '12 at 17:46

The problem with creating a "lock" file as suggested in a previous solution, is that the lock file might already exist if the script responsible to removing it terminates abnormally. This could for example happen if the user terminates the rsync process, or due to a power outage. Instead one should use flock, which does not suffer from this problem.

As it happens flock is also easy to use, so the solution would simply look like this:

flock -n lock_file -c "rsync ..."

The command after the -c option is only executed if there is no other process locking on the lock_file. If the locking process for any reason terminates, the lock will be released on the lock_file. The -n options says that flock should be non-blocking, so if there is another processes locking the file nothing will happen.

share|improve this answer
1  
This is absolutely correct. Another solution is to use leases, instead of locks, so that the lock times out after a period of time unless the lease is renewed. A lease is basically a lockfile with a timestamp in it. – cha0site Apr 2 '12 at 19:17
6  
Worked perfectly. This should be the accepted answer. – chovy Jul 9 '13 at 1:17
1  
Should be noted that this will not work on NFS – WarmWaffles Oct 19 '13 at 5:52
    
I noticed this same circumstance while reading the other proposal on this thread; thanks for explaining how flock addresses this possibility. I've read other people suggesting flock on other threads, but yours is the first I've come across where you both recommend it and address my exact concern. – Lonnie Best Nov 14 '13 at 21:42

A simple solution without using a lock file is to just do this:

pgrep rsync > /dev/null || rsync -avz ...

This will work as long as it is the only rsync job you run on the server, and you can then run this directly in cron, but you will need to redirect the output to a log file.

If you do run multiple rsync jobs, you can get pgrep to match against the full command line with a pattern like this:

pgrep -f rsync.*/data > /dev/null || rsync -avz --delete /data/ otherhost:/data/
pgrep -f rsync.*/www  > /dev/null || rsync -avz --delete /var/www/ otherhost:/var/www/
share|improve this answer

To use the lock file example given by @User above, a trap should be used to verify that the lock file is removed when the script is exited for any reason.

if [ -e /home/myhomedir/rsyncjob.lock ]
then
  echo "Rsync job already running...exiting"
  exit
fi

touch /home/myhomedir/rsyncjob.lock

#delete lock file at end of your job

trap 'rm /home/myhomedir/rsyncjob.lock' EXIT

#your code in here

This way the lock file will be removed even if the script exits before the end of the script.

share|improve this answer
    
This is actually the best solution – berezovskyi Apr 30 '17 at 10:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.