127

I need some help with my code. I'm new at Node.js and have a lot of trouble with it.

What I'm trying to do:

  1. Fetch a .txt with Amazon products (ASINs) ;

  2. Fetch all products using the amazon-product-api package;

  3. Save each product in a .json file.

My code is not working. I think I messed up with this asynchronous-synchronous stuff - help me!

var amazon = require('amazon-product-api');
var fs = require('fs');

var client = amazon.createClient({
    awsId: "XXX",
    awsSecret: "XXX",
    awsTag: "888"
});

var array = fs.readFileSync('./test.txt').toString().split('\n');
for (var i = 1; i < array.length; i++) {
     var ASIN = array[i];

    client.itemLookup({
            domain: 'webservices.amazon.de',
            responseGroup: 'Large',
            idType: 'ASIN',
            itemId: ASIN
        })
        .then(function(results) {
            fs.writeFile(ASIN + '.json', JSON.stringify(results), function(err) {
                if (err) {
                    console.log(err);
                } else {
                    console.log("JSON saved");
                }
            })

            return results;

        }).catch(function(err) {
            console.log(err);
        });
};
2
  • 1
    After 6 years, 10 answers and hundreds of votes, still no one has noticed that the for-loop ALWAYS exits in the first iteration because of the return statement! I'm going to edit the question and remove return, but as a sanity check it would be great if one of the highly voter answerers confirmed my edit. It would also be great if one of you improved the title of the quesion, making it more specific.
    – Inigo
    Apr 4, 2022 at 19:41
  • Even better if the answers had more of the question's context, e.g. rewriting the full for-loop, at least at the end of their answers.
    – Inigo
    Apr 4, 2022 at 19:59

12 Answers 12

273

As of 2019...

...the correct answer is to use async/await with the native fs promises module included in node. Upgrade to Node.js 10 or 11 (already supported by major cloud providers) and do this:

const fs = require('fs').promises;

// This must run inside a function marked `async`:
const file = await fs.readFile('filename.txt', 'utf8');
await fs.writeFile('filename.txt', 'test');

Do not use third-party packages and do not write your own wrappers, that's not necessary anymore.

No longer experimental

Before Node 11.14.0, you would still get a warning that this feature is experimental, but it works just fine and it's the way to go in the future. Since 11.14.0, the feature is no longer experimental and is production-ready.

What if I prefer import instead of require?

It works, too - but only in Node.js versions where this feature is not marked as experimental.

import { promises as fs } from 'fs';

(async () => {
    await fs.writeFile('./test.txt', 'test', 'utf8');
})();
10
  • 9
    Heads up this is marked as experimental as of 10.9.0
    – rynop
    Sep 6, 2018 at 18:06
  • 1
    To avoid the message: ExperimentalWarning: The fs.promises API is experimental, just use node --no-warnings app.js
    – robe007
    Nov 5, 2018 at 15:56
  • 2
    there is no warning in node 11.x+ May 9, 2019 at 4:53
  • 1
    This should be the correct answer as long as you're not stuck on an old version of Node. async and await is the (beautiful) future of handling asynchronous stuff. May 31, 2019 at 19:10
  • 4
    If you use import you can also use import fs from 'fs/promises'; Dec 19, 2020 at 14:01
65

Because fs.writefile is a traditional asynchronous callback - you need to follow the promise spec and return a new promise wrapping it with a resolve and rejection handler like so:

return new Promise(function(resolve, reject) {
    fs.writeFile("<filename.type>", data, '<file-encoding>', function(err) {
        if (err) reject(err);
        else resolve(data);
    });
});

So in your code you would use it like so right after your call to .then():

 .then(function(results) {
    return new Promise(function(resolve, reject) {
            fs.writeFile(ASIN + '.json', JSON.stringify(results), function(err) {
               if (err) reject(err);
               else resolve(data);
            });
    });
  }).then(function(results) {
       console.log("results here: " + results)
  }).catch(function(err) {
       console.log("error here: " + err);
  });
2
  • 4
    Starting in Node v8 you can use util.promisify() to convert the fs I/O functions into promises, e.g. const util = require('util'); const writeFile = util.promisify(fs.writeFile); ... return writeFile(ASIN + '.json', JSON.stringify(results)); Jul 31, 2018 at 23:10
  • 1
    Starting in Node v10 you can use await fs. directly, check out my answer below
    – user6269864
    Aug 30, 2018 at 9:57
63

say

const util = require('util')
const fs_writeFile = util.promisify(fs.writeFile)

https://nodejs.org/api/util.html#util_util_promisify_original

this is less prone to bugs than the top-voted answer

1
  • 14
    util.promisify was added in node.js version 8. Thought this info might be helpful for people thinking of using it.
    – Tim
    Nov 28, 2017 at 16:43
25

Finally, the latest node.js release v10.3.0 has natively supported fs promises.

const fsPromises = require('fs').promises; // or require('fs/promises') in v10.0.0
fsPromises.writeFile(ASIN + '.json', JSON.stringify(results))
  .then(() => {
    console.log('JSON saved');
  })
  .catch(er => {
    console.log(er);
  });

You can check the official documentation for more details. https://nodejs.org/api/fs.html#fs_fs_promises_api

6
  • 3
    const fs = require('fs').promises, not /promises May 31, 2018 at 14:37
  • @LeonardPauli Updated my answer.
    – Lewis
    Jun 3, 2018 at 4:51
  • I'm getting a TypeError: Cannot read property 'writeFile' of undefined. Possibly, because this method does not have a constructor for handling binary Strings?: fsPromises.writeFile(tempFile, response.auioContent, 'binary').then(() => { console.log('Audio content written to file: ' + tempFile); return { filePath: "filePath" } }) Dec 9, 2018 at 2:14
  • It appears fs.writeFile(...) returns a promise now... May someone please confirm? The then(...) portion of my code is being reached. Dec 9, 2018 at 3:10
  • @Lewis see this comment. Could you also edit your answer to be more complete, e.g. to show it in the full context of the question's code... e.g. within the for-loop?
    – Inigo
    Apr 4, 2022 at 19:53
18

If you want to import the promise based version of fs as an ES module you can do:

import { promises as fs } from 'fs'

await fs.writeFile(...)

As soon as node v14 is released (see this PR), you can also use

import { writeFile } from 'fs/promises'
1
  • see this comment. Could you also edit your answer to be more complete, e.g. to show it in the full context of the question's code... e.g. within the for-loop?
    – Inigo
    Apr 4, 2022 at 19:54
14

What worked for me was fs.promises.

Example One:

const fs = require("fs")

fs.promises
  .writeFile(__dirname + '/test.json', "data", { encoding: 'utf8' })
  .then(() => {
    // Do whatever you want to do.
    console.log('Done');
  });

Example Two. Using Async-Await:

const fs = require("fs")

async function writeToFile() {
  await fs.promises.writeFile(__dirname + '/test-22.json', "data", {
    encoding: 'utf8'
  });

  console.log("done")
}

writeToFile()
5

2023, Node v16+, ESM, import, using promises

(I've also refactored the code a bit to avoid nested try..catch blocks, YMMV.

import * as fs from 'node:fs/promises';

async function doFile(filepath) { // choose a better name than `doFile` for your use case!!
    try {
        const fileData = await fs.readFile(filepath, {encoding: 'utf8'});
        var array = fileData.toString().split('\n');
        for (var i = 1; i < array.length; i++) {
            var ASIN = array[i];
            let results = await doClientStuff(args); // e.g. client.itemLookup
            await saveResults(ASIN, results);
        };
    } catch (error) {
        console.error(error?.message ?? error?.toString()); // or however you extract your error msg
    }
}
async function saveResults(ASIN, results) {
    try {
        await fs.writeFile(ASIN + '.json', JSON.stringify(results));
        console.log("JSON saved");
    } catch (error) {
        console.error(error?.message ?? error?.toString()); // or however you extract your error msg
    }
}
await doFile('./test.txt');

This uses the OP's desire to us a fs default import. I personally would change it to import only your individual functions and remove the fs. references to enable esm tree-shaking:

import { readFile, writeFile } from 'node:fs/promises';

const fileData = await readFile(filepath, {encoding: 'utf8'}); // fs. removed

await writeFile(ASIN + '.json', JSON.stringify(results)); // fs. removed

further info

If no encoding is specified (using options.encoding), the data is returned as a object. Otherwise, the data will be a string.

Refer to Node documentation for additional info, including alternative usage with sync/callbacks. Be sure to select your correct version in the docs for your Node environment.

4

Update Sept 2017: fs-promise has been deprecated in favour of fs-extra.


I haven't used it, but you could look into fs-promise. It's a node module that:

Proxies all async fs methods exposing them as Promises/A+ compatible promises (when, Q, etc). Passes all sync methods through as values.

3

Use require('fs/promises')

var fs = require('fs/promises'); // Since 11.14.0
var path = require('path'); // to help us to join better the paths

var content = JSON.stringify(["this is your content"]); // Must be a string to be written.

fs
  .writeFile(path.join(__dirname, 'test.json'), content, { encoding: 'utf8' })
  .then(() => {
    console.log('Write is done!');
  });

Example using async/await

var fs = require('fs/promises'); // Since 11.14.0
var path = require('path'); // to help us to join better the paths

var content = JSON.stringify(["this is your content"]); // Must be a string to be written.

(async function autorun(){
  
  await fs.writeFile(path.join(__dirname, 'test.json'), content, { encoding: 'utf8' })  
  console.log('Write is done!');

})() // This is called a IIFE: Immediately invoked function expression

2
const util = require('util')
const fs = require('fs');

const fs_writeFile = util.promisify(fs.writeFile)

fs_writeFile('message.txt', 'Hello Node.js')
    .catch((error) => {
        console.log(error)
    });
-2

For easy to use asynchronous convert all callback to promise use some library like "bluebird" .

      .then(function(results) {
                fs.writeFile(ASIN + '.json', JSON.stringify(results), function(err) {
                    if (err) {
                        console.log(err);
                    } else {
                        console.log("JSON saved");
                        return results;
                    }
                })


            }).catch(function(err) {
                console.log(err);
            });

Try solution with promise (bluebird)

var amazon = require('amazon-product-api');
var fs = require('fs');
var Promise = require('bluebird');

var client = amazon.createClient({
    awsId: "XXX",
    awsSecret: "XXX",
    awsTag: "888"
});


var array = fs.readFileSync('./test.txt').toString().split('\n');
Promise.map(array, function (ASIN) {
    client.itemLookup({
        domain: 'webservices.amazon.de',
        responseGroup: 'Large',
        idType: 'ASIN',
        itemId: ASIN
    }).then(function(results) {
        fs.writeFile(ASIN + '.json', JSON.stringify(results), function(err) {
            if (err) {
                console.log(err);
            } else {
                console.log("JSON saved");
                return results;
            }
        })
    }).catch(function(err) {
        console.log(err);
    });
});
-3

Use fs.writeFileSync inside the try/catch block as below.

var fs = require('fs');
try {
    const file = fs.writeFileSync(ASIN + '.json', JSON.stringify(results))
    console.log("JSON saved");
    return results;
} catch (error) {
    console.log(err);
}
1
  • 2
    never do this! writeFileSync is a synchronous function that halts the execution of the script and makes the app slow and defeats the entire purpose of using JS on server side. im downvoting, sorry. Dec 26, 2019 at 12:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.