7

I want to write a bash script that (recursively) processes all files of a certain type.

I know I can get the matching file list by using find thusly:

find . -name "*.ext"

I want to use this in a script:

  1. recursively obatin list of files with a given extension
  2. obtain the full file pathname
  3. pass the full pathname to another script
  4. Check the return code from the script. If non zero, log the name of the file that could not be processed.

My first attempt looks (pseudocode) like this:

ROOT_DIR = ~/work/projects
cd $ROOT_DIR
for f in `find . -name "*.ext"`
do
    #need to lop off leading './' from filename, but I havent worked out how to use
    #cut yet
    newname = `echo $f | cut -c 3
    filename = "$ROOT_DIR/$newname"

    retcode = ./some_other_script $filename

    if $retcode ne 0
       logError("Failed to process file: $filename")
done

This is my first attempt at writing a bash script, so the snippet above is not likely to run. Hopefully though, the logic of what I'm trying to do is clear enough, and someone can show how to join the dots and convert the pseudocode above into a working script.

I am running on Ubuntu

1
  • 1
    If you use 'find $ROOT_DIR -name "*.exe"' you won't need to fiddle with the leading './'. Dec 10, 2010 at 15:39

2 Answers 2

17
find . -name '*.ext' \( -exec ./some_other_script "$PWD"/{} \; -o -print \)
3
  • +1 Good solution, I didn't know you could use -o this way. The only problem is that sometimes (not here) you need to call a function instead of an external executable.
    – tokland
    Dec 10, 2010 at 15:52
  • Hmm quite cryptic this .... (reminds me of Perl!). I dont understand this. How is the filename being passed to the other script, and how do I know if the script returned non zero, so I know to do something?
    – skyeagle
    Dec 10, 2010 at 17:42
  • 1
    @skyeagle: The {} in the -exec predicate indicates where to inject the current match. The -exec predicate itself tests the result code of the command, yielding true if 0, false otherwise. Dec 10, 2010 at 17:44
2

Using | while read to iterate over file names is fine as long as there are no files with carrier return to be processed:

find . -name '*.ext' | while IFS=$'\n' read -r FILE; do
  process "$(readlink -f "$FILE")" || echo "error processing: $FILE"
done
7
  • 1
    I wouldn't say it's officially discouraged at all. It's much better than for f in $(find). There can be issues if there are spaces, tabs or newlines in filenames. It can be improved like this: while IFS=$'\n' read -r FILE Dec 11, 2010 at 2:06
  • @Dennis, I cannot find the page right now (may be not that official), but it discouraged using "while | read" because carrier returns are valid characters for files (fortunately nobody uses them). Added your suggestion.
    – tokland
    Dec 11, 2010 at 13:07
  • Such assumptions are always problematic. Even if "normal" users often avoid newlines in file names, it doesn't mean nobody does it—and an attacker will certainly do it. Any program should try hard to accept all allowed file names; fortunately that is quite easy in every real programming language—another reason to avoid shell scripts.
    – Philipp
    Dec 11, 2010 at 15:01
  • @Philipp: Well, this is a more general discussion. I wouldn't call shell languages "non-real programming languages", but indeed they have their drawbacks and historical luggage. However, IMHO they deserve high respect for devising one of the most powerful programming concepts ever: piping.
    – tokland
    Dec 11, 2010 at 15:14
  • 1
    Perhaps you were thinking of this page. It's a good discussion of the challenges presented by filenames. Dec 11, 2010 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.