632

The project that I am working on (Node.js) implies lots of operations with the file system (copying, reading, writing, etc.).

Which methods are the fastest?

3
  • 50
    It's a good question, though it is interesting that it gets 25 upvotes when other similar format questions will get 3 or 4 downvotes right away for not meeting the SO "standards" (maybe the javascript tag is crawled by kinder people :)
    – Ben
    Commented Sep 11, 2013 at 11:47
  • 25
    Mostly we're just fresh new and excited about this whole "files" business after years of normalizing browsers. Commented Oct 30, 2014 at 17:32
  • 6
    The only correct answer on the page is this one. None of the other answers actually copy files. Files on MacOS and Windows have other metadata that is lost by just copying bytes. Examples of data not copied by any other answer on this page, windows and macos. Even on Unix the other answers don't copy the creation date, something that's often important when copying a file.
    – gman
    Commented Oct 23, 2018 at 3:58

17 Answers 17

944

Use the standard built-in way fs.copyFile:

const fs = require('fs');

// File destination.txt will be created or overwritten by default.
fs.copyFile('source.txt', 'destination.txt', (err) => {
  if (err) throw err;
  console.log('source.txt was copied to destination.txt');
});

If you have to support old end-of-life versions of Node.js - here is how you do it in versions that do not support fs.copyFile:

const fs = require('fs');
fs.createReadStream('test.log').pipe(fs.createWriteStream('newLog.log'));
18
  • 71
    Just remember that in real life, you'd want to check both the createReadStream and createWriteStream for errors, so you wouldn't get a one-liner (though it would still be just as fast).
    – ebohlman
    Commented Jul 4, 2012 at 0:37
  • 22
    How much faster/slower is this than executing the raw cp test.log newLog.log via require('child_process').exec?
    – HareSurf
    Commented Jan 30, 2013 at 20:03
  • 44
    Well copy is not portable on Window, contrary to a full Node.js solution.
    – Jean
    Commented Jul 3, 2013 at 18:51
  • 12
    Unfortunately on my system using streams is extremely slow compared to child_process.execFile('/bin/cp', ['--no-target-directory', source, target]).
    – Robert
    Commented Sep 25, 2013 at 22:27
  • 13
    I used this method and all I got was a blank file on write. any ideas why? fs.createReadStream('./init/xxx.json').pipe(fs.createWriteStream('xxx.json'));
    – Timmerz
    Commented Aug 20, 2014 at 15:23
299

Same mechanism, but this adds error handling:

function copyFile(source, target, cb) {
  var cbCalled = false;

  var rd = fs.createReadStream(source);
  rd.on("error", function(err) {
    done(err);
  });
  var wr = fs.createWriteStream(target);
  wr.on("error", function(err) {
    done(err);
  });
  wr.on("close", function(ex) {
    done();
  });
  rd.pipe(wr);

  function done(err) {
    if (!cbCalled) {
      cb(err);
      cbCalled = true;
    }
  }
}
10
  • 5
    It is worth noting that cbCalled flag is needed because pipe errors trigger an error on both streams. Source and destination streams. Commented Mar 26, 2014 at 21:33
  • 4
    How do you handle the error if the source file doesn't exist? Destination file still gets created in that case.
    – Michel Hua
    Commented May 18, 2014 at 18:04
  • 1
    I think an error in the WriteStream will only unpipe it. You would have to call rd.destroy() yourself. At least that's what happened to me. Sadly there's not much documentation except from the source code.
    – Robert
    Commented Aug 6, 2014 at 5:46
  • what does the cb stand for? what should we pass in as the third argument?
    – SaiyanGirl
    Commented Feb 26, 2015 at 19:29
  • 4
    @SaiyanGirl 'cb' stands for "callback". You should pass in a function. Commented Feb 26, 2015 at 22:08
159

Since Node.js 8.5.0 we have the new fs.copyFile and fs.copyFileSync methods.

Usage example:

var fs = require('fs');

// File "destination.txt" will be created or overwritten by default.
fs.copyFile('source.txt', 'destination.txt', (err) => {
    if (err) 
        throw err;
    console.log('source.txt was copied to destination.txt');
});
5
  • 5
    This is the only correct answer on the page. None of the other answers actually copy files. Files on MacOS and Windows have other metadata that is lost by just copying bytes. Examples of data not copied by any other answer on this page, windows and macos. Even on Unix the other answer don't copy the creation date, something that's often important when copying a file.
    – gman
    Commented Oct 23, 2018 at 3:56
  • well sadly this fails to copy everything on mac. Hopefully they'll fix it: github.com/nodejs/node/issues/30575
    – gman
    Commented Nov 22, 2019 at 2:43
  • BTW keep in mind that copyFile() is bugged while overwriting longer files. Courtesy of uv_fs_copyfile() till Node v8.7.0 (libuv 1.15.0). see github.com/libuv/libuv/pull/1552 Commented Mar 3, 2020 at 11:02
  • Does this work if a file is on a server? I want to copy from "http: //www.example.com/file.pdf" to my local node js server, "/pdf" Commented Mar 24, 2023 at 22:54
  • This gives me the error UnhandledPromiseRejectionWarning: TypeError [ERR_INVALID_ARG_TYPE]: The "mode" argument must be integer. Received type function
    – mrid
    Commented Mar 25 at 13:11
149

I was not able to get the createReadStream/createWriteStream method working for some reason, but using the fs-extra npm module it worked right away. I am not sure of the performance difference though.

npm install --save fs-extra

var fs = require('fs-extra');

fs.copySync(path.resolve(__dirname, './init/xxx.json'), 'xxx.json');
9
  • 3
    This is the best option now
    – Zain Rizvi
    Commented Jan 5, 2015 at 22:23
  • 13
    Using syncronous code in node kills your application performance.
    – mvillar
    Commented May 21, 2015 at 6:44
  • 5
    Oh please... The question is about fastest method to copy a file. While fastest is always subjective, I don't think a synchronous piece of code has any business here. Commented Sep 3, 2015 at 18:25
  • 27
    Fastest to implement or fastest to execute? Differing priorities mean this is a valid answer. Commented Sep 23, 2015 at 20:35
  • 17
    fs-extra also has asynchronous methods, i.e. fs.copy(src, dst, callback);, and these should resolve @mvillar's concern. Commented Nov 9, 2015 at 10:49
76

Fast to write and convenient to use, with promise and error management:

function copyFile(source, target) {
  var rd = fs.createReadStream(source);
  var wr = fs.createWriteStream(target);
  return new Promise(function(resolve, reject) {
    rd.on('error', reject);
    wr.on('error', reject);
    wr.on('finish', resolve);
    rd.pipe(wr);
  }).catch(function(error) {
    rd.destroy();
    wr.end();
    throw error;
  });
}

The same with async/await syntax:

async function copyFile(source, target) {
  var rd = fs.createReadStream(source);
  var wr = fs.createWriteStream(target);
  try {
    return await new Promise(function(resolve, reject) {
      rd.on('error', reject);
      wr.on('error', reject);
      wr.on('finish', resolve);
      rd.pipe(wr);
    });
  } catch (error) {
    rd.destroy();
    wr.end();
    throw error;
  }
}
9
  • 1
    What happens when no more input exists (broken network share), but the write still succeeds? Will both reject (from read) and resolve (from write) be called? What if both read/write fails (bad disk sectors during read, full disk during write)? Then reject will be called twice. A Promise solution based on Mike's answer with a flag (unfortunately) seems to be the only viable solution that properly considers error handling.
    – Lekensteyn
    Commented May 24, 2015 at 13:15
  • The promise is resolved once the copy succeed. If it's rejected, its state is settled and calling reject multiple times won't make no difference.
    – benweet
    Commented May 24, 2015 at 17:08
  • 2
    I just tested new Promise(function(resolve, reject) { resolve(1); resolve(2); reject(3); reject(4); console.log("DONE"); }).then(console.log.bind(console), function(e){console.log("E", e);}); and looked up the spec on this and you are right: Attempting to resolve or reject a resolved promise has no effect. Perhaps you could extend your answer and explain why you have written the function in this way? Thanks :-)
    – Lekensteyn
    Commented May 24, 2015 at 22:42
  • 2
    By the way, close should be finish for Writable streams.
    – Lekensteyn
    Commented May 25, 2015 at 9:32
  • And if you wonder why your application never closes after pipe errors on /dev/stdin, that is a bug github.com/joyent/node/issues/25375
    – Lekensteyn
    Commented May 27, 2015 at 9:26
46

Well, usually it is good to avoid asynchronous file operations. Here is the short (i.e. no error handling) sync example:

var fs = require('fs');
fs.writeFileSync(targetFile, fs.readFileSync(sourceFile));
15
  • 10
    To say that in general is extremely false, particularly since it leads to people re-slurping files for every request made to their server. This can get expensive.
    – Catalyst
    Commented Jun 19, 2014 at 16:14
  • 9
    using the *Sync methods are totally against nodejs' philosphy! I also think they are slowly being deprecated. The whole idea of nodejs is that it's single threaded and event-driven.
    – gillyb
    Commented Oct 14, 2014 at 21:15
  • 11
    @gillyb The only reason I can think of for using them is for simplicity - if you are writing a quick script that you will only use once, you probably aren't going to be all that bothered about blocking the process. Commented Oct 24, 2014 at 7:23
  • 13
    I'm not aware of them being deprecated. Sync methods are almost always a terrible idea on a web server but sometimes ideal in something like node-webkit where it only locks up action in the window while files are copying. Throw up a loading gif and maybe a load bar that updates at certain points and let sync methods block all action until the copying is done. It's not really a best practice thing so much as a when and where they have their place thing. Commented Oct 30, 2014 at 17:42
  • 6
    Sync methods are fine when you are interacting with another sync operation or what you want is to perform sequential operation (ie. you would be emulating sync anyway). If the operations are sequential just avoid the callback hell (and/or promise soup) and use the sync method. In general they should be used with caution on servers but are fine for most cases that involve CLI scripts.
    – srcspider
    Commented Jul 31, 2015 at 9:55
18

If you don't care about it being async, and aren't copying gigabyte-sized files, and would rather not add another dependency just for a single function:

function copySync(src, dest) {
  var data = fs.readFileSync(src);
  fs.writeFileSync(dest, data);
}
5
  • 8
    @RobGleeson, and requires as much memory as the file content... I am amazed by the count of upvotes there.
    – Konstantin
    Commented Jun 19, 2017 at 18:25
  • I've added a "and aren't copying gigabyte-sized files" caveat. Commented Nov 22, 2017 at 17:16
  • The fs.existsSync call should be omitted. The file could disappear in the time between the fs.existsSync call and the fs.readFileSync call, which means the fs.existsSync call doesn't protect us from anything.
    – qntm
    Commented Aug 8, 2019 at 13:53
  • Additionally, returning false if fs.existsSync fails is likely poor ergonomics because few consumers of copySync will think to manually inspect the return value every time it's called, any more than we do for fs.writeFileSync et al.. Throwing an exception is actually preferable.
    – qntm
    Commented Jan 18, 2020 at 15:02
  • The OP does not specifically mention that their files are UTF-8 text, so I'm removing the 'utf-8' encoding from the snippet too, which means this will now work on any file. data is now a Buffer, not a String.
    – qntm
    Commented Mar 13, 2020 at 18:06
18

Mike Schilling's solution with error handling with a shortcut for the error event handler.

function copyFile(source, target, cb) {
  var cbCalled = false;

  var rd = fs.createReadStream(source);
  rd.on("error", done);

  var wr = fs.createWriteStream(target);
  wr.on("error", done);
  wr.on("close", function(ex) {
    done();
  });
  rd.pipe(wr);

  function done(err) {
    if (!cbCalled) {
      cb(err);
      cbCalled = true;
    }
  }
}
13

You may want to use async/await, since node v10.0.0 it's possible with the built-in fs Promises API.

Example:

const fs = require('fs')

const copyFile = async (src, dest) => {
  await fs.promises.copyFile(src, dest)
}

Note:

As of node v11.14.0, v10.17.0 the API is no longer experimental.

More information:

Promises API

Promises copyFile

7
   const fs = require("fs");
   fs.copyFileSync("filepath1", "filepath2"); //fs.copyFileSync("file1.txt", "file2.txt");

This is what I personally use to copy a file and replace another file using Node.js :)

3
  • 2
    This does not answer the question, which is about how to efficiently copy files in an IO-heavy application. Commented Aug 10, 2019 at 13:26
  • 1
    @JaredSmith True, but my google search lead me here and this is what I wanted.
    – codepleb
    Commented Nov 28, 2019 at 14:22
  • I wonder why copyFileSync in an async function wouldn't perform well. I would think it would be optimized to match copyFile or stream copying. Commented Feb 10, 2021 at 0:06
2

For fast copies you should use the fs.constants.COPYFILE_FICLONE flag. It allows (for filesystems that support this) to not actually copy the content of the file. Just a new file entry is created, but it points to a Copy-on-Write "clone" of the source file.

To do nothing/less is the fastest way of doing something ;)

https://nodejs.org/api/fs.html#fs_fs_copyfile_src_dest_flags_callback

let fs = require("fs");

fs.copyFile(
  "source.txt",
  "destination.txt",
  fs.constants.COPYFILE_FICLONE,
  (err) => {
    if (err) {
      // TODO: handle error
      console.log("error");
    }
    console.log("success");
  }
);

Using promises instead:

let fs = require("fs");
let util = require("util");
let copyFile = util.promisify(fs.copyFile);


copyFile(
  "source.txt",
  "destination.txt",
  fs.constants.COPYFILE_FICLONE
)
  .catch(() => console.log("error"))
  .then(() => console.log("success"));
2
  • 1
    fs.promises.copyFile
    – gman
    Commented Nov 24, 2019 at 8:55
  • 1
    Re "To do nothing/less is the fastest way of doing something": Yes, indeed. That is the first rule of optimisation - eliminate unnecessary operations. That is in contrast to make the existing ones go faster, e.g. by fiddling with compiler flags. Commented Oct 27, 2020 at 22:57
1

Use Node.js's built-in copy function

It provides both async and sync version:

const fs = require('fs');

// File "destination.txt" will be created or overwritten by default.
fs.copyFile('source.txt', 'destination.txt', (err) => {
  if (err) 
      throw err;
  console.log('source.txt was copied to destination.txt');
});

fs.copyFileSync(src, dest[, mode])

1
  • 6
    Not upvoting because this answer is a duplicate.
    – Qwertie
    Commented Jul 3, 2018 at 0:23
0

You can do it using the fs-extra module very easily:

const fse = require('fs-extra');

let srcDir = 'path/to/file';
let destDir = 'pat/to/destination/directory';

fse.moveSync(srcDir, destDir, function (err) {

    // To move a file permanently from a directory
    if (err) {
        console.error(err);
    } else {
        console.log("success!");
    }
});

Or

fse.copySync(srcDir, destDir, function (err) {

     // To copy a file from a directory
     if (err) {
         console.error(err);
     } else {
         console.log("success!");
     }
});
0

I wrote a little utility to test the different methods:

https://www.npmjs.com/package/copy-speed-test

run it with

npx copy-speed-test --source someFile.zip --destination someNonExistentFolder

It does a native copy using child_process.exec(), a copy file using fs.copyFile and it uses createReadStream with a variety of different buffer sizes (you can change buffer sizes by passing them on the command line. run npx copy-speed-test -h for more info).

0

Asynchronously, the following code is currently the best:

const fs = require("fs").promises;
let src = "sourcePath";
let dest = "destinationPath";
try {
    await fs.copyFile(src, dest).then(console.log(`Copy success. ${src} >> ${dest}`));
} catch (err) {
    console.log("Error: ", err);
}
-1

Mike's solution, but with promises:

const FileSystem = require('fs');

exports.copyFile = function copyFile(source, target) {
    return new Promise((resolve,reject) => {
        const rd = FileSystem.createReadStream(source);
        rd.on('error', err => reject(err));
        const wr = FileSystem.createWriteStream(target);
        wr.on('error', err => reject(err));
        wr.on('close', () => resolve());
        rd.pipe(wr);
    });
};
0
-1

Improvement of one other answer.

Features:

  • If the dst folders do not exist, it will automatically create it. The other answer will only throw errors.
  • It returns a promise, which makes it easier to use in a larger project.
  • It allows you to copy multiple files, and the promise will be done when all of them are copied.

Usage:

var onePromise = copyFilePromise("src.txt", "dst.txt");
var anotherPromise = copyMultiFilePromise(new Array(new Array("src1.txt", "dst1.txt"), new Array("src2.txt", "dst2.txt")));

Code:

function copyFile(source, target, cb) {
    console.log("CopyFile", source, target);

    var ensureDirectoryExistence = function (filePath) {
        var dirname = path.dirname(filePath);
        if (fs.existsSync(dirname)) {
            return true;
        }
        ensureDirectoryExistence(dirname);
        fs.mkdirSync(dirname);
    }
    ensureDirectoryExistence(target);

    var cbCalled = false;
    var rd = fs.createReadStream(source);
    rd.on("error", function (err) {
        done(err);
    });
    var wr = fs.createWriteStream(target);
    wr.on("error", function (err) {
        done(err);
    });
    wr.on("close", function (ex) {
        done();
    });
    rd.pipe(wr);
    function done(err) {
        if (!cbCalled) {
            cb(err);
            cbCalled = true;
        }
    }
}

function copyFilePromise(source, target) {
    return new Promise(function (accept, reject) {
        copyFile(source, target, function (data) {
            if (data === undefined) {
                accept();
            } else {
                reject(data);
            }
        });
    });
}

function copyMultiFilePromise(srcTgtPairArr) {
    var copyFilePromiseArr = new Array();
    srcTgtPairArr.forEach(function (srcTgtPair) {
        copyFilePromiseArr.push(copyFilePromise(srcTgtPair[0], srcTgtPair[1]));
    });
    return Promise.all(copyFilePromiseArr);
}
2
  • What other answer? Commented Oct 27, 2020 at 22:45
  • @PeterMortensen Mike Schilling's.
    – ch271828n
    Commented Oct 28, 2020 at 2:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.