150

Is it possible to compress (create a compressed archive) data while reading from stdin on Linux?

1
  • 1
    I'm voting to close this question as off-topic because it belongs to Unix & Linux Jun 30, 2019 at 10:53

3 Answers 3

220

Yes, use gzip for this. The best way is to read data as input and redirect the compressed to output file i.e.

cat test.csv | gzip > test.csv.gz

cat test.csv will send the data as stdout and using pipe-sign gzip will read that data as stdin. Make sure to redirect the gzip output to some file as compressed data will not be written to the terminal.

5
  • 5
    Note that the test.csv.gz files remains of 0 size until the data flux is closed. Apr 26, 2013 at 10:00
  • ... and how does one do that? Is it really a separate question? Mar 8, 2015 at 14:26
  • 13
    This also works with xz: mysqldump mydb | xz > dbdump.sql.xz
    – jeroen
    Jul 27, 2015 at 18:32
  • 3
    Could use -9, default compression level is -6. Oct 5, 2017 at 13:48
  • 1
    @MUY Belgium: not exactly, it starts writing only after 16KB of compressed blocks. See here
    – Francois
    Oct 5, 2021 at 15:37
114

Yes, gzip will let you do this. If you simply run gzip > foo.gz, it will compress STDIN to the file foo.gz. You can also pipe data into it, like some_command | gzip > foo.gz.

1

gzip > stdin.gz perhaps? Otherwise, you need to flesh out your question.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.