7

I have a tar archive which is very big ~ 5GB.

I want to grep for a pattern on all files (and also print the name of the file that has the pattern ) in the archive but do not want to fill up my disk space by extracting the archive.

Anyway I can do that?

I tried these, but this does not give me the file names that contain the pattern, just the matching lines:

tar -O -xf test.tar.gz | grep 'this'
tar -xf test.tar.gz --to-command='grep awesome'

Also where is this feature of tar documented? tar xf test.tar $FILE

4

7 Answers 7

14

Seems like nobody posted this simple solution that processes the archive only once:

tar xzf archive.tgz --to-command \
    'grep --label="$TAR_FILENAME" -H PATTERN ; true'

Here tar passes the name of each file in a variable (see the docs) and it is used by grep to print it with each match. Also true is added so that tar doesn't complain about failing to extract files that don't match.

5
  • 1
    Best answer, works perfectly and as intended by tar.
    – Matt
    Jan 10, 2016 at 16:02
  • The ; true isn’t working for me; those are being passed as arguments to grep, which complains about ; and true not being files or directories.
    – Daniel H
    Oct 6, 2016 at 14:02
  • @DanielH Which shell are you using? This can affect how the command is parsed and run.
    – Petr
    Oct 6, 2016 at 17:52
  • zsh, but it seems more like the kind of thing which would change between versions of tar than between shells. Both zsh and bash would interpret the single-quoted grep command the same way if it were typed in directly.
    – Daniel H
    Oct 6, 2016 at 20:12
  • Thanks, just what I was looking for. The other answers here and in related (dup?) question stackoverflow.com/q/2407111/446767 read the archive multiple times or seemed extra complicated.
    – Ted
    May 18, 2018 at 17:28
7

Here's my take on this:

while read filename; do tar -xOf file.tar "$filename" | grep 'pattern' | sed "s|^|$filename:|"; done < <(tar -tf file.tar | grep -v '/$')

Broken out for explanation:

  • while read filename; do -- it's a loop...
  • tar -xOf file.tar "$filename" -- this extracts each file...
  • | grep 'pattern' -- here's where you put your pattern...
  • | sed "s|^|$filename:|"; - prepend the filename, so this looks like grep. Salt to taste.
  • done < <(tar -tf file.tar | grep -v '/$') -- end the loop, get the list of files as to fead to your while read.

One proviso: this breaks if you have OR bars (|) in your filenames.

Hmm. In fact, this makes a nice little bash function, which you can append to your .bashrc file:

targrep() {

  local taropt=""

  if [[ ! -f "$2" ]]; then
    echo "Usage: targrep pattern file ..."
  fi

  while [[ -n "$2" ]]; do    

    if [[ ! -f "$2" ]]; then
      echo "targrep: $2: No such file" >&2
    fi

    case "$2" in
      *.tar.gz) taropt="-z" ;;
      *) taropt="" ;;
    esac

    while read filename; do
      tar $taropt -xOf "$2" \
       | grep "$1" \
       | sed "s|^|$filename:|";
    done < <(tar $taropt -tf $2 | grep -v '/$')

  shift

  done
}
1
  • I hate thanks on SO, but this one really helped me out - thx (:
    – drevicko
    Mar 13, 2014 at 5:57
3

Here's a bash function that may work for you. Add the following to your ~/.bashrc

targrep () {
    for i in $(tar -tzf "$1"); do
        results=$(tar -Oxzf "$1" "$i" | grep --label="$i" -H "$2")
        echo "$results"
    done
}

Usage:

targrep archive.tar.gz "pattern"
2
  • This does not work. It prints (standard input) as the file name. I tried with -l and -H. Oct 24, 2012 at 0:00
  • 1
    Doh, I didn't see your bash function before I wrote mine. And yours is nicer than mine. :-) +1. (Now that I check, it seems that FreeBSD's tar automatically recognizes gzipped files, so my filename recognition may be redundant.)
    – ghoti
    Oct 24, 2012 at 3:32
1

It's incredibly hacky, but you could abuse tar's -v option to process and delete each file as it is extracted.

grep_and_delete() {
  if [ -n "$1" -a -f "$1" ]; then
    grep -H 'this' -- "$1" </dev/null
    rm -f -- "$1" </dev/null
  fi
}
mkdir tmp; cd tmp
tar -xvzf test.tar.gz | (
  prev=''
  while read pathname; do
    grep_and_delete "$prev"
    prev="$pathname"
  done
  grep_and_delete "$prev"
)
2
1
tar -tf test.tar.gz | grep -v '/$'| \
xargs -n 1 -I _ \
sh -c 'tar -xOf test.tar.gz _|grep -q <YOUR SEARCH PATTERN>  && echo _'
2
  • Can you please answer my question in the comment of original post ? Oct 24, 2012 at 0:34
  • @abc, if it's part of your question, why don't you add it to your question?
    – Graham
    Oct 24, 2012 at 3:46
0

Try:

    tar tvf name_of_file |grep --regex="pattern"

The t option will test the tar file without extracting the files. The v is verbose and the f prints he filenames. This should save you considerable hard disk space.

3
  • This is not right either. Have you tried this before posting? Oct 24, 2012 at 0:10
  • 1
    What's wrong with it, @abc? It doesn't extract the files; it gets the file names on standard output, and greps for them. What result did you get? Or is the trouble that you want to get the names of the files that contain the pattern in the body of the file without extracting the files...that's harder, but not clearly described in the question. Oct 24, 2012 at 0:15
  • This is great for extracting and searching the file name patterns. I found it helpful, since I was looking for just the file names. However, it is correct to say this will not search the contents of the files.
    – zerocog
    Dec 2, 2019 at 21:29
0

may help

zcat log.tar.gz | grep -a -i "string"

zgrep -i "string" log.tar.gz

http://www.commandlinefu.com/commands/view/9261/grep-compressed-log-files-without-extracting

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.