139

I have found the question How to determine if data is valid tar file without a file?, but I was wondering: is there a ready made command line solution?

8 Answers 8

158

What about just getting a listing of the tarball and throw away the output, rather than decompressing the file?

tar -tzf my_tar.tar.gz >/dev/null

Edited as per comment. Thanks zrajm!

Edit as per comment. Thanks Frozen Flame! This test in no way implies integrity of the data. Because it was designed as a tape archival utility most implementations of tar will allow multiple copies of the same file!

10
  • 18
    Why use -v if you're just piping the output to /dev/null?
    – zrajm
    Dec 29, 2013 at 13:38
  • 6
    The -z option is also unneeded. It does nothing in extract or list mode.
    – asmeurer
    Jan 14, 2014 at 22:07
  • 1
    @asmeurer Re: -z That's definitely the case with GNU tar -- do you know if this is this true elsewhere (BSD, etc.)?
    – belacqua
    May 13, 2014 at 17:59
  • 2
    @belacqua For others' information, as in the manpage of BSD TAR, -z: "... In extract or list modes, this option is ignored." Jul 9, 2015 at 7:48
  • 2
    @bobwells But does successful uncompressing or content files listing imply data integrity of the tar.gz? Any support information? Jul 9, 2015 at 7:53
123

you could probably use the gzip -t option to test the files integrity

http://linux.about.com/od/commands/l/blcmdl1_gzip.htm

from: http://unix.ittoolbox.com/groups/technical-functional/shellscript-l/how-to-test-file-integrity-of-targz-1138880 (archived version here)

To test the gzip file is not corrupt:

gunzip -t file.tar.gz

To test the tar file inside is not corrupt:

gunzip -c file.tar.gz | tar -t > /dev/null

As part of the backup you could probably just run the latter command and check the value of $? afterwards for a 0 (success) value. If either the tar or the gzip has an issue, $? will have a non zero value.

5
  • 3
    And bzip2 -t file.bz2 for bz2 files.
    – asmeurer
    Jan 14, 2014 at 22:08
  • 6
    Isn't this just using two commands to do what "tar -tzf my_tar.tar.gz >/dev/null" does?
    – Rob Wells
    Apr 16, 2018 at 18:11
  • Isn't it supposed to be tar -t > /dev/null (note: t vs -t)? May 30, 2020 at 3:29
  • 1
    @IntrastellarExplorer that is a typo on my part, although it should still work without the hyphen. I'm just used to the old style options on tar. unix.stackexchange.com/questions/394060/…
    – John Boker
    Jun 1, 2020 at 14:25
  • At first glance I feel this is the most correct answer here, as at least the integrity of the zipped 'file,tar' is checked against the checksum contained in .gz file. However for individual files tared in 'files.tar' relying on tar -t is riskier because random byte errors may very likely not be detected. Anyhow I suppose there is nothing better than 'tar -t" you can get here.
    – grenix
    Feb 14, 2022 at 12:43
39

If you want to do a real test extract of a tar file without extracting to disk, use the -O option. This spews the extract to standard output instead of the filesystem. If the tar file is corrupt, the process will abort with an error.

Example of failed tar ball test...

$ echo "this will not pass the test" > hello.tgz
$ tar -xvzf hello.tgz -O > /dev/null
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error exit delayed from previous errors
$ rm hello.*

Working Example...

$ ls hello*
ls: hello*: No such file or directory
$ echo "hello1" > hello1.txt
$ echo "hello2" > hello2.txt
$ tar -cvzf hello.tgz hello[12].txt
hello1.txt
hello2.txt
$ rm hello[12].txt
$ ls hello*
hello.tgz
$ tar -xvzf hello.tgz -O
hello1.txt
hello1
hello2.txt
hello2
$ ls hello*
hello.tgz
$ tar -xvzf hello.tgz
hello1.txt
hello2.txt
$ ls hello*
hello1.txt  hello2.txt  hello.tgz
$ rm hello*
5
  • 2
    It seems to me that the best test is this one. It truly extracts each file and makes sure there are no errors.
    – sleeves
    Jan 23, 2012 at 20:36
  • Really useful. I've made a shell script, add an argument hook to pass the path of the file and put it in my path :) [ tar -xvzf $1 -O > /dev/null ]
    – smonff
    Aug 22, 2012 at 19:19
  • @sleeves Why do you think that it is better than the accepted answer? tar -tvzf hello.tgz > /dev/null also gives the same error.
    – dash17291
    Mar 4, 2013 at 15:48
  • 7
    @dash17291 I say this because I expect it to be a tough problem to prove that for all cases, a -tvf will catch all errors or corruptions that a -xvf. In other words, -xvf will catch all that -tvf, but I cannot say the converse is true.
    – sleeves
    May 25, 2013 at 2:20
  • Maybe you should use > /dev/null for the working example, too.
    – moi
    Jul 23, 2016 at 15:34
12

You can also check contents of *.tag.gz file using pigz (parallel gzip) to speedup the archive check:

pigz -cvdp number_of_threads /[...]path[...]/archive_name.tar.gz | tar -tv > /dev/null
2
  • 1
    Mini benchmark: running it with pigz -cvd: 80s, while running with accepted answer tar -tzv: 143s on a 22G archive.
    – Wadih M.
    Aug 30, 2018 at 13:18
  • how do u remove the notification from cronjob about "tar: Removing leading `/' from member names" when using this method ?
    – MaXi32
    Jun 1, 2020 at 14:42
5

A nice option is to use tar -tvvf <filePath> which adds a line that reports the kind of file.

Example in a valid .tar file:

> tar -tvvf filename.tar 
drwxr-xr-x  0 diegoreymendez staff       0 Jul 31 12:46 ./testfolder2/
-rw-r--r--  0 diegoreymendez staff      82 Jul 31 12:46 ./testfolder2/._.DS_Store
-rw-r--r--  0 diegoreymendez staff    6148 Jul 31 12:46 ./testfolder2/.DS_Store
drwxr-xr-x  0 diegoreymendez staff       0 Jul 31 12:42 ./testfolder2/testfolder/
-rw-r--r--  0 diegoreymendez staff      82 Jul 31 12:42 ./testfolder2/testfolder/._.DS_Store
-rw-r--r--  0 diegoreymendez staff    6148 Jul 31 12:42 ./testfolder2/testfolder/.DS_Store
-rw-r--r--  0 diegoreymendez staff  325377 Jul  5 09:50 ./testfolder2/testfolder/Scala.pages
Archive Format: POSIX ustar format,  Compression: none

Corrupted .tar file:

> tar -tvvf corrupted.tar 
tar: Unrecognized archive format
Archive Format: (null),  Compression: none
tar: Error exit delayed from previous errors.
1
  • This occurs on BSD tar but not on GNU tar.
    – mcallister
    May 20, 2014 at 16:48
3

I have tried the following command and they work well.

bzip2 -t file.bz2
gunzip -t file.gz

However, we can found these two command are time-consuming. Maybe we need some more quick way to determine the intact of the compress files.

1

These are all very sub-optimal solutions. From the GZIP spec

ID2 (IDentification 2)
These have the fixed values ID1 = 31 (0x1f, \037), ID2 = 139 (0x8b, \213), to identify the file as being in gzip format.

Has to be coded into whatever language you're using.

1
  • if the file has been truncated, that would seem to require full decompression to detect
    – philwalk
    Feb 19, 2020 at 16:06
0

> use the -O option. [...] If the tar file is corrupt, the process will abort with an error.

Sometimes yes, but sometimes not. Let's see an example of a corrupted file:

echo Pete > my_name
tar -cf my_data.tar my_name 

# // Simulate a corruption
sed < my_data.tar 's/Pete/Fool/' > my_data_now.tar
# // "my_data_now.tar" is the corrupted file

tar -xvf my_data_now.tar -O

It shows:

my_name
Fool  

Even if you execute

echo $?

tar said that there was no error:

0

but the file was corrupted, it has now "Fool" instead of "Pete".

2
  • People rarely use tar files without any compression. I guess your remark regards only uncompressed files.
    – Jarekczek
    Nov 18, 2012 at 12:47
  • 8
    You are confusing integrity with corruption. Your file has lost it's integrity but it is still an acceptable archive format.
    – Phil
    Oct 19, 2015 at 8:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.