288

I would like to clarify this point, as the documentation is not too clear about it;

Q1: Is Promise.all(iterable) processing all promises sequentially or in parallel? Or, more specifically, is it the equivalent of running chained promises like

p1.then(p2).then(p3).then(p4).then(p5)....

or is it some other kind of algorithm where all p1, p2, p3, p4, p5, etc. are being called at the same time (in parallel) and results are returned as soon as all resolve (or one rejects)?

Q2: If Promise.all runs in parallel, is there a convenient way to run an iterable sequencially?

Note: I don't want to use Q, or Bluebird, but all native ES6 specs.

7
  • 1
    Are you asking about node (V8) implementation, or about the spec?
    – Amit
    Jun 13, 2015 at 21:22
  • 1
    I'm pretty sure Promise.all executes them in parallel.
    – royhowie
    Jun 13, 2015 at 21:23
  • @Amit I flagged node.js and io.js as this is where I'm using it. So, yes, the V8 implementation if you will. Jun 13, 2015 at 21:24
  • 13
    Promises cannot "be executed". They start their task when they are being created - they represent the results only - and you are executing everything in parallel even before passing them to Promise.all.
    – Bergi
    Jun 13, 2015 at 21:27
  • 1
    Promises are executed at the moment of creation. (can be confirmed by running a bit of code). In new Promise(a).then(b); c(); a is executed first, then c, then b. It isn't Promise.all that runs these promises, it just handles when they resolve.
    – Mateon1
    Jun 13, 2015 at 21:31

14 Answers 14

368

Is Promise.all(iterable) executing all promises?

No, promises cannot "be executed". They start their task when they are being created - they represent the results only - and you are executing everything in parallel even before passing them to Promise.all.

Promise.all does only await multiple promises. It doesn't care in what order they resolve, or whether the computations are running in parallel.

is there a convenient way to run an iterable sequencially?

If you already have your promises, you can't do much but Promise.all([p1, p2, p3, …]) (which does not have a notion of sequence). But if you do have an iterable of asynchronous functions, you can indeed run them sequentially. Basically you need to get from

[fn1, fn2, fn3, …]

to

fn1().then(fn2).then(fn3).then(…)

and the solution to do that is using Array::reduce:

iterable.reduce((p, fn) => p.then(fn), Promise.resolve())
16
  • 3
    In this example, is iterable an array of the functions that return a promise that you want to call? Jan 15, 2016 at 23:40
  • 2
    @SSHThis: It's exactly as the then sequence - the return value is the promise for the last fn result, and you can chain other callbacks to that.
    – Bergi
    May 31, 2016 at 21:58
  • 1
    @wojjas That's exactly equivalent to fn1().then(p2).then(fn3).catch(…? No need to use a function expression.
    – Bergi
    Dec 1, 2016 at 0:07
  • 1
    @wojjas Of course the retValFromF1 is passed into p2, that's exactly what p2 does. Sure, if you want to do more (pass additional variables, call multiple functions, etc) you need to use a function expression, though changing p2 in the array would be easier
    – Bergi
    Dec 1, 2016 at 11:22
  • 1
    @robe007 Yes, I meant that iterable is the [fn1, fn2, fn3, …] array
    – Bergi
    Aug 7, 2018 at 16:21
162

In parallel

await Promise.all(items.map(async (item) => { 
  await fetchItem(item) 
}))

Advantages: Faster. All iterations will be started even if one fails later on. However, it will "fail fast". Use Promise.allSettled, to complete all iterations in parallel even if some throw. Technically, these are concurrent invocations not in parallel.

In sequence

for (const item of items) {
  await fetchItem(item)
}

Advantages: Variables in the loop can be shared by each iteration. Behaves like normal imperative synchronous code.

16
  • 21
    Or: for (const item of items) await fetchItem(item); Feb 24, 2018 at 21:33
  • 2
    @david_adler In parallel example advantages you said All iterations will be executed even if one fails. If I'm not wrong this would still fail fast. To change this behaviour one can do something like: await Promise.all(items.map(async item => { return await fetchItem(item).catch(e => e) }))
    – Taimoor
    Nov 7, 2018 at 9:33
  • @Taimoor yes it does "fail fast" and continue executing code after the Promise.all but all iterations are still executed codepen.io/mfbx9da4/pen/BbaaXr Feb 25, 2019 at 17:12
  • 1
    Note that javascript isn't actually executing the asynchronous requests in "parallel" using threads since javascript is single threaded. developer.mozilla.org/en-US/docs/Web/JavaScript/EventLoop Jun 18, 2020 at 16:18
  • 1
    In the parallel example, is the await in await fetchItem(item) necessary? Why not await Promise.all(items.map(item => fetchItem(item)))
    – kehers
    Apr 6, 2022 at 6:21
46

NodeJS does not run promises in parallel, it runs them concurrently since it’s a single-threaded event loop architecture. There is a possibility to run things in parallel by creating a new child process to take advantage of the multiple core CPU.

Parallel Vs Concurent

In fact, what Promise.all does is, stacking the promises function in the appropriate queue (see event loop architecture) running them concurrently (call P1, P2,...) then waiting for each result, then resolving the Promise.all with all the promises results. Promise.all will fail at the first promise which fails unless you have to manage the rejection yourself.

There is a major difference between parallel and concurrent, the first one will run a different computation in a separate process at exactly the same time and they will progress at their rhythm, while the other one will execute the different computation one after another without waiting for the previous computation to finish and progress at the same time without depending on each other.

Finally, to answer your question, Promise.all will execute neither in parallel nor sequentially but concurrently.

3
  • 7
    This is not right. NodeJS can run things in parallel. NodeJS has a concept of worker thread. By default the number of worker thread is 4. For example, if you use crypto library to hash two values then you can execute them in parallel. Two worker threads will handle the task. Of course, your CPU has to be multi-core to support parallelism.
    – Shihab
    May 11, 2020 at 17:51
  • 3
    Yeah you right, it’s what i said at the end of the first paragraph, but i talked about child process, of course they can run workers. May 11, 2020 at 17:54
  • 4
    Best answer so far. I was so confused that how a single-threaded architecture like Node.js could run multiple promises in parallel. Thanks alot sir. P.S. I know how worker threads are and how they work but promises are resolved by Node.js event-loop itself and not by using libuv. So the best Node.js could do is to execute them (promises) concurrently. Jan 19, 2021 at 6:25
14

Bergi's answer got me on the right track using Array.reduce.

However, to actually get the functions returning my promises to execute one after another I had to add some more nesting.

My real use case is an array of files that I need to transfer in order one after another due to limits downstream...

Here is what I ended up with:

getAllFiles().then( (files) => {
    return files.reduce((p, theFile) => {
        return p.then(() => {
            return transferFile(theFile); //function returns a promise
        });
    }, Promise.resolve()).then(()=>{
        console.log("All files transferred");
    });
}).catch((error)=>{
    console.log(error);
});

As previous answers suggest, using:

getAllFiles().then( (files) => {
    return files.reduce((p, theFile) => {
        return p.then(transferFile(theFile));
    }, Promise.resolve()).then(()=>{
        console.log("All files transferred");
    });
}).catch((error)=>{
    console.log(error);
});

didn't wait for the transfer to complete before starting another and also the "All files transferred" text came before even the first file transfer was started.

Not sure what I did wrong, but wanted to share what worked for me.

Edit: Since I wrote this post I now understand why the first version didn't work. then() expects a function returning a promise. So, you should pass in the function name without parentheses! Now, my function wants an argument so then I need to wrap in in a anonymous function taking no argument!

8

You can also process an iterable sequentially with an async function using a recursive function. For example, given an array a to process with asynchronous function someAsyncFunction():

var a = [1, 2, 3, 4, 5, 6]

function someAsyncFunction(n) {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      console.log("someAsyncFunction: ", n)
      resolve(n)
    }, Math.random() * 1500)
  })
}

//You can run each array sequentially with: 

function sequential(arr, index = 0) {
  if (index >= arr.length) return Promise.resolve()
  return someAsyncFunction(arr[index])
    .then(r => {
      console.log("got value: ", r)
      return sequential(arr, index + 1)
    })
}

sequential(a).then(() => console.log("done"))

9
  • using array.prototype.reduce is much better in terms of performance than a recursive function Jul 18, 2018 at 15:32
  • @MateuszSowiński, there is a 1500ms timeout between each call. Considering that this is doing async calls sequentially, it’s hard to see how that’s relevant even for a very quick async turnaround.
    – Mark
    Jul 18, 2018 at 15:44
  • Let's say you have to execute 40 of really quick async functions after each other - using recursive functions would clog your memory pretty fast Jul 18, 2018 at 15:46
  • @MateuszSowiński, that the stack doesn't wind up here...we're returning after each call. Compare that with reduce where you have to build the entire then() chain in one step and then execute.
    – Mark
    Jul 18, 2018 at 15:56
  • In the 40th call of the sequential function the first call of the function is still in memory waiting for the chain of sequential functions to return Jul 18, 2018 at 16:00
5

Just to elaborate on @Bergi's answer (which is very succinct, but tricky to understand ;)

This code will run each item in the array and add the next 'then chain' to the end:

function eachorder(prev,order) {
        return prev.then(function() {
          return get_order(order)
            .then(check_order)
            .then(update_order);
        });
    }
orderArray.reduce(eachorder,Promise.resolve());
4

Using async await an array of promises can easily be executed sequentially:

let a = [promise1, promise2, promise3];

async function func() {
  for(let i=0; i<a.length; i++){
    await a[i]();
  }  
}

func();

Note: In above implementation, if a promise is rejected, the rest wouldn't be executed.If you want all your promises to be executed, then wrap your await a[i](); inside try catch

3

parallel

see this example

const resolveAfterTimeout = async i => {
  return new Promise(resolve => {
    console.log("CALLED");
    setTimeout(() => {
      resolve("RESOLVED", i);
    }, 5000);
  });
};

const call = async () => {
  const res = await Promise.all([
    resolveAfterTimeout(1),
    resolveAfterTimeout(2),
    resolveAfterTimeout(3),
    resolveAfterTimeout(4),
    resolveAfterTimeout(5),
    resolveAfterTimeout(6)
  ]);
  console.log({ res });
};

call();

by running the code it'll console "CALLED" for all six promises and when they are resolved it will console every 6 responses after timeout at the same time

2

I stumbled across this page while trying to solve a problem in NodeJS: reassembly of file chunks. Basically: I have an array of filenames. I need to append all those files, in the correct order, to create one large file. I must do this asynchronously.

Node's 'fs' module does provide appendFileSync but I didn't want to block the server during this operation. I wanted to use the fs.promises module and find a way to chain this stuff together. The examples on this page didn't quite work for me because I actually needed two operations: fsPromises.read() to read in the file chunk, and fsPromises.appendFile() to concat to the destination file. Maybe if I was better with JavaScript I could have made the previous answers work for me. ;-)

I stumbled across this and I was able to hack together a working solution:

/**
 * sequentially append a list of files into a specified destination file
 */
exports.append_files = function (destinationFile, arrayOfFilenames) {
    return arrayOfFilenames.reduce((previousPromise, currentFile) => {
        return previousPromise.then(() => {
            return fsPromises.readFile(currentFile).then(fileContents => {
                return fsPromises.appendFile(destinationFile, fileContents);
            });
        });
    }, Promise.resolve());
};

And here's a jasmine unit test for it:

const fsPromises = require('fs').promises;
const fsUtils = require( ... );
const TEMPDIR = 'temp';

describe("test append_files", function() {
    it('append_files should work', async function(done) {
        try {
            // setup: create some files
            await fsPromises.mkdir(TEMPDIR);
            await fsPromises.writeFile(path.join(TEMPDIR, '1'), 'one');
            await fsPromises.writeFile(path.join(TEMPDIR, '2'), 'two');
            await fsPromises.writeFile(path.join(TEMPDIR, '3'), 'three');
            await fsPromises.writeFile(path.join(TEMPDIR, '4'), 'four');
            await fsPromises.writeFile(path.join(TEMPDIR, '5'), 'five');

            const filenameArray = [];
            for (var i=1; i < 6; i++) {
                filenameArray.push(path.join(TEMPDIR, i.toString()));
            }

            const DESTFILE = path.join(TEMPDIR, 'final');
            await fsUtils.append_files(DESTFILE, filenameArray);

            // confirm "final" file exists    
            const fsStat = await fsPromises.stat(DESTFILE);
            expect(fsStat.isFile()).toBeTruthy();

            // confirm content of the "final" file
            const expectedContent = new Buffer('onetwothreefourfive', 'utf8');
            var fileContents = await fsPromises.readFile(DESTFILE);
            expect(fileContents).toEqual(expectedContent);

            done();
        }
        catch (err) {
            fail(err);
        }
        finally {
        }
    });
});
2

You can do it by for loop.

async function return promise:

async function createClient(client) {
    return await Client.create(client);
}

let clients = [client1, client2, client3];

if you write following code then client are created parallelly:

const createdClientsArray = yield Promise.all(clients.map((client) =>
    createClient(client);
));

But if you want to create client sequentially then you should use for loop:

const createdClientsArray = [];
for(let i = 0; i < clients.length; i++) {
    const createdClient = yield createClient(clients[i]);
    createdClientsArray.push(createdClient);
}
1
  • 9
    At this time, async/await is only available with a transpiler, or using other engines than Node. Also, you really should not mix async with yield. Whle they act the same with a transpiler and co, they really are quite different and should not ordinarily substitude each other. Also, you should mention these restrictions as your answer is confusing to novice programmers. Feb 25, 2016 at 13:48
2

Bergi's answer helped me to make the call synchronous. I have added an example below where we call each function after the previous function is called:

function func1 (param1) {
    console.log("function1 : " + param1);
}
function func2 () {
    console.log("function2");
}
function func3 (param2, param3) {
    console.log("function3 : " + param2 + ", " + param3);
}

function func4 (param4) {
    console.log("function4 : " + param4);
}
param4 = "Kate";

//adding 3 functions to array

a=[
    ()=>func1("Hi"),
    ()=>func2(),
    ()=>func3("Lindsay",param4)
  ];

//adding 4th function

a.push(()=>func4("dad"));

//below does func1().then(func2).then(func3).then(func4)

a.reduce((p, fn) => p.then(fn), Promise.resolve());
1
  • Is this an answer to the original question? May 10, 2019 at 20:10
1

I've been using for of in order to solve sequential promises. I'm not sure if it helps here but this is what I've been doing.

async function run() {
    for (let val of arr) {
        const res = await someQuery(val)
        console.log(val)
    }
}

run().then().catch()
0

Yes, you can chain an array of promise returning functions as follows (this passes the result of each function to the next). You could of course edit it to pass the same argument (or no arguments) to each function.

function tester1(a) {
  return new Promise(function(done) {
    setTimeout(function() {
      done(a + 1);
    }, 1000);
  })
}

function tester2(a) {
  return new Promise(function(done) {
    setTimeout(function() {
      done(a * 5);
    }, 1000);
  })
}

function promise_chain(args, list, results) {

  return new Promise(function(done, errs) {
    var fn = list.shift();
    if (results === undefined) results = [];
    if (typeof fn === 'function') {
      fn(args).then(function(result) {
        results.push(result);
        console.log(result);
        promise_chain(result, list, results).then(done);
      }, errs);
    } else {
      done(results);
    }

  });
}

promise_chain(0, [tester1, tester2, tester1, tester2, tester2]).then(console.log.bind(console), console.error.bind(console));

-2

see this sample

Promise.all working parallel

const { range, random, forEach, delay} = require("lodash");  
const run = id => {
    console.log(`Start Task ${id}`);
    let prom = new Promise((resolve, reject) => {
        delay(() => {
            console.log(`Finish Task ${id}`);
            resolve(id);
        }, random(2000, 15000));
    });
    return prom;
}


const exec = () => {
    let proms = []; 
    forEach(range(1,10), (id,index) => {
        proms.push(run(id));
    });
    let allPromis = Promise.all(proms); 
    allPromis.then(
        res => { 
            forEach(res, v => console.log(v));
        }
    );
}

exec();

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.