62

How can we GZip every file separately?

I don't want to have all of the files in a big tar.

8 Answers 8

99

You can use gzip *


Note:

  • This will zip each file individually and DELETE the original.
  • Use -k (--keep) option to keep the original files.
  • This may not work if you have a huge number of files due to limits of the shell
  • To run gzip in parallel see @MarkSetchell's answer below.
1
  • If I want to run it under a crontab, which command line I'll put ? For example /tmp/app/gzip * ?
    – MCunha98
    Feb 28, 2020 at 13:21
54

Easy and very fast answer that will use all your CPU cores in parallel:

parallel gzip ::: *

GNU Parallel is a fantastic tool that should be used far more in this world where CPUs are only getting more cores rather than more speed. There are loads of examples that we would all do well to take 10 minutes to read... here

3
  • Knew about parallel, but keep forgetting to use it! Ran the accepted answer then scrolled down to see your answer... saved lots of hours! Maybe a good idea to make your comment an edit to your answer?
    – zx8754
    Jan 3, 2018 at 12:42
  • Is there a way to use parallel if the argument list is too long? Have about 60k files that I need individually compressed. May 24, 2019 at 17:56
  • 2
    @Californian Sure find . -name XYZ -print0 | parallel -0 gzip May 24, 2019 at 18:34
43

After seven years, this highly upvoted comment still doesn't have its own full-fledged answer, so I'm promoting it now:

gzip -r .

This has two advantages over the currently accepted answer: it works recursively if there are any subdirectories, and it won't fail from Argument list too long if the number of files is very large.

2
  • Does this keep or delete the files? Do you still need to add the -k option mentioned in the other answer. Sep 29, 2019 at 1:10
  • 2
    Add -k if you want to keep the original files. Sep 29, 2019 at 6:30
14

If you want to gzip every file recursively, you could use find piped to xargs:

$ find . -type f -print0 | xargs -0r gzip
4
  • 11
    No need for find+xargs. Gzip can handle recursion itself: gzip -9r .
    – Idelic
    Nov 25, 2009 at 6:19
  • 2
    As always, find . -type f -print0 | xargs -0r gzip is better.
    – musiphil
    Mar 19, 2013 at 5:42
  • 1
    For the equivalent of gzip *, you may also need -maxdepth 1 in find.
    – musiphil
    Mar 19, 2013 at 5:43
  • @musiphil: good point about protecting for spaces in file names! I just edited the answer to integrate your comment (waiting for peer review). Dec 5, 2015 at 21:36
8

Try a loop

$ for file in *; do gzip "$file"; done
0
8

Or, if you have pigz (gzip utility that parallelizes compression over multiple processors and cores)

pigz *
2

The following command can run multiple times inside a directory (without "already has .gz suffix" warnings) to gzip whatever is not already gzipped.

find . -maxdepth 1 -type f ! -name '*.gz' -exec gzip "{}" \;

A more useful example of utilizing find is when you want to gzip rolling logs. E.g. you want every day or every month to gzip rolled logs but not current logs.

# Considering that current logs end in .log and 
# rolled logs end in .log.[yyyy-mm-dd] or .log.[number]
find . -maxdepth 1 -type f ! -name '*.gz' ! -name '*.log' -exec gzip "{}" \;
0

fyi, this will help to overwrite if an existing gz file along with creating few gz file if it is not present:

find . -type f | grep "in case any specific" | grep -E -v "*.gz$" | xargs -n1 -P8 sh -c 'yes | gzip --force --best -f $0'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.