15

I want to know how I can send the command(s) spawned by xargs to background. For example, consider

find . -type f  -mtime +7 | tee compressedP.list | xargs compress

I tried

find . -type f  -mtime +7 | tee compressedP.list | xargs -i{} compress {} &

.. and as unexpected, it seems to send xargs to the background instead?

How do I make each instance of the compress command go to the background?

5 Answers 5

20

Use the --max-procs / -P option to run xargs targets in the background. From the man page of GNU xargs version 4.2.27:

--max-procs=max-procs, -P max-procs

Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a time. Use the -n option with -P; otherwise chances are that only one exec will be done.

(You may want to combine this with -n 1 to makes sure that there is a new process for each file you want to compress)

1
  • (Hmm, just realised my answer is a duplicate of the second-half of gnarf's answer, which I didn't read after the opening comment about writing a shell script. I'll leave my answer here in case others also skim over gnarf's answer as I did)
    – RobM
    May 26, 2011 at 11:44
8

You could probably make a very quick shellscript to call compress.

#!/bin/sh 
# call it 'compbg' and chmod a+x
compress $* &

then

find . -type f  -mtime +7 | tee compressedP.list | xargs -I{} compbg {}

Although I think you might be happier using this xargs argument:

 -P maxprocs
         Parallel mode: run at most maxprocs invocations of utility at once.

This command should find / tee / compress 10 files at a time until its done, as well as returning control immediately to the calling script/shell.

find . -type f  -mtime +7 | tee compressedP.list | xargs -I{} -P10 compress {} &
2
  • The UNIX I have ( $ uname -a SunOS xxx 5.10 Generic_xx7xx1-06 sun4u sparc SUNW,Sun-Fire ) does not have -P ( $ xargs -P xargs: illegal option -- P xargs: Usage: xargs: [-t] [-p] [-e[eofstr]] [-E eofstr] [-I replstr] [-i[replstr]] [-L #] [-l[#]] [-n # [-x]] [-s size] [cmd [args ...]] ) Any other alternatives than the shell script wrapper?
    – PoorLuzer
    Dec 23, 2009 at 12:19
  • 1
    Sorry, I can't find much for SunOS - although I did find this package "xjobs" which you might be able to compile... maier-komor.de/xjobs.html
    – gnarf
    Dec 24, 2009 at 1:13
5

For SunOS you may have a look at GNU Parallel http://www.gnu.org/software/parallel/

find . -type f  -mtime +7 | tee compressedP.list | parallel compress

It has the added benefit of not terminating incorrectly if the filename contains ' " or space. Adding -j+0 will make it run one compress per CPU core.

1

This works for me when I want to edit all files that contain foo

grep -rl foo ./ | `xargs emacs` &

1
  • 1
    This starts an extra unnecessary subshell. It is better to use -P argument to xargs. Mar 27, 2020 at 23:01
0

On SunOS/Solaris, wrap the command in a call to sh (or other shell):

find . -type f  -mtime +7 | tee compressedP.list | xargs -Ifname sh -c 'compress fname &'

This should run the compresses in parallel without additional tools.

1
  • On Linux anyway, this backgrounding will cause xargs to immediately spawn the next process called for by the list being piped in. If there's a lot in compressedP.list, you'll run out of memory. Apr 12, 2021 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.