12

An installation process is downloading a .tar.gz archive, then extract the files to a destination directory. However, not all the files in the archive are required, and I'd like to specify which files should be extracted. The naïve way would be to delete the unnecessary files after extraction, but I'd like a "cleaner" way and filter out instead.

Is this possible?

The (relevant) code I have so far is (stripped for readability)

var fs = require('fs');
var tar = require('tar');
var zlib = require('zlib');

var log = console.log;

var tarball = 'path/to/downloaded/archive.tar.gz';
var dest = 'path/to/destination';

fs.createReadStream(tarball)
  .on("error", log)
  .pipe(zlib.Unzip())
  .pipe(tar.Extract({ path: dest }))
  .on("end", log);

Thank you.

2 Answers 2

14

It works similar to the unzip module:

var fs = require('fs');
var tar = require('tar');
var zlib = require('zlib');
var path = require('path');
var mkdirp = require('mkdirp'); // used to create directory tree

var log = console.log;

var tarball = 'path/to/downloaded/archive.tar.gz';
var dest    = 'path/to/destination';

fs.createReadStream(tarball)
  .on('error', log)
  .pipe(zlib.Unzip())
  .pipe(tar.Parse())
  .on('entry', function(entry) {
    if (/\.js$/.test(entry.path)) { // only extract JS files, for instance
      var isDir     = 'Directory' === entry.type;
      var fullpath  = path.join(dest, entry.path);
      var directory = isDir ? fullpath : path.dirname(fullpath);

      mkdirp(directory, function(err) {
        if (err) throw err;
        if (! isDir) { // should really make this an `if (isFile)` check...
          entry.pipe(fs.createWriteStream(fullpath));
        }
      });
    }
  });
6
  • @rynop good catch, although I would perform that check before calling mkdirp().
    – robertklep
    Apr 1, 2015 at 18:52
  • lol ya duh. fixed. Also I'm seeing the files within the .tar.gz get corrupted on disk after extraction. You ever seen this before? Files get onto disk with the correct name and structure, but jar's inside the file are corrupted. Omitting the extract only JS files line and using tar.Extract() this does not happen
    – rynop
    Apr 1, 2015 at 19:20
  • @rynop I've used similar code to extract tar files, and it has always worked for me. If you have a sample tar file for me, I'd be happy to take a look.
    – robertklep
    Apr 1, 2015 at 20:26
  • @robertklep I appreciate it: dynamodb-local.s3-website-us-west-2.amazonaws.com/… You can see my code that is working at github.com/doapp-ryanp/dynamodb-local/blob/master/index.js#L105 - if i change to tar.Parse() and use your logic the binary files get corrupted (different size)
    – rynop
    Apr 2, 2015 at 16:16
  • @rynop this seems to work just fine for me: gist.github.com/robertklep/01dd2c64a0fe9f5483d5 (feel free to leave comments there instead of here :-)
    – robertklep
    Apr 2, 2015 at 17:22
0

You can take a look at this post to find a good solution.

By the way, in the zlib-documentation you'll see that you can specify a "buffer" calling .unzip().

3
  • No, the archive can be quite large and I don't want to allocate that much RAM. The memory footprint must be at it's minimum. Besides, the archives contains a directory structure and what you are proposing does not apply. Feb 24, 2014 at 13:55
  • I have to read an extracted file from the tar. When should I call my read function? I tried onClose but my file is not fully written till then. You can see my code. Feb 6, 2018 at 10:43
  • fs.createReadStream(filename) .pipe(zlib.Unzip()) .pipe(new tar.Parse()) .on('entry', function(entry) { { var isDir = 'Directory' === entry.type; var fullpath = path.join(dest, entry.path); var directory = isDir ? fullpath : path.dirname(fullpath); mkdirp(directory, function(err) { if (err) throw err; if (! isDir) { entry.pipe(fs.createWriteStream(fullpath) .on('error', function(e){alert('Error');}) ); }});} }).on('close', function(){setTimeout(readXMLFile(sysObject.path + '\\layout_new.xml'),0);}) Feb 6, 2018 at 10:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.