323

I have a simplified function that looks like this:

function(query) {
  myApi.exec('SomeCommand', function(response) {
    return response;
  });
}

Basically i want it to call myApi.exec, and return the response that is given in the callback lambda. However, the above code doesn't work and simply returns immediately.

Just for a very hackish attempt, i tried the below which didn't work, but at least you get the idea what i'm trying to achieve:

function(query) {
  var r;
  myApi.exec('SomeCommand', function(response) {
    r = response;
  });
  while (!r) {}
  return r;
}

Basically, what's a good 'node.js/event driven' way of going about this? I want my function to wait until the callback gets called, then return the value that was passed to it.

3
  • 4
    Or am i going about it entirely the wrong way here, and should i be calling another callback, rather than returning a response?
    – Chris
    Feb 15, 2011 at 22:33
  • This is in my opinion the best SO explanation why the busy loop doesn't work.
    – bluenote10
    May 20, 2017 at 16:11
  • Don't try to wait. Just call next function(callback-dependent) within at the end of callback itself
    – Atul
    Aug 5, 2019 at 7:08

10 Answers 10

307

The "good node.js /event driven" way of doing this is to not wait.

Like almost everything else when working with event driven systems like node, your function should accept a callback parameter that will be invoked when then computation is complete. The caller should not wait for the value to be "returned" in the normal sense, but rather send the routine that will handle the resulting value:

function(query, callback) {
  myApi.exec('SomeCommand', function(response) {
    // other stuff here...
    // bla bla..
    callback(response); // this will "return" your value to the original caller
  });
}

So you dont use it like this:

var returnValue = myFunction(query);

But like this:

myFunction(query, function(returnValue) {
  // use the return value here instead of like a regular (non-evented) return value
});
13
  • 5
    Ok great. What about if myApi.exec never called the callback? How would i make it so that the callback gets called after say 10 seconds with an error value saying it timed our or something?
    – Chris
    Feb 15, 2011 at 22:42
  • 5
    Or better yet (added a check so the callback cant be invoked twice): jsfiddle.net/LdaFw/1
    – Jakob
    Feb 15, 2011 at 23:04
  • 180
    It is clear non-blocking is the standard in node/js, however there are certainly times when blocking is desired (e.g. blocking on stdin). Even node has "blocking" methods (see all the fs sync* methods). As such, I think this is still a valid question. Is there a nice way to achieve blocking in node aside from busy waiting?
    – nategood
    May 12, 2012 at 16:02
  • 7
    A late answer to the comment by @nategood: I can think of a couple of ways; too much to explain in this comment, but google them. Remember that Node is not made to be blocked, so these are not perfect. Think of them as suggestions. Anyway, here goes: (1) Use C to implement your function and publish it to NPM in order to use it. That's what the sync methods do. (2) Use fibers, github.com/laverdet/node-fibers, (3) Use promises, for example the Q-library, (4) Use a thin layer on top of javascript, that looks blocking, but compiles to async, like maxtaco.github.com/coffee-script
    – Jakob
    Mar 21, 2013 at 10:52
  • 164
    It is so frustrating when people answer a question with "you shouldn't do that." If one wants to be helpful and answer a question, that is a stand up thing to do. But telling me unequivocally that I shouldn't do something is just unfriendly. There are a million different reasons why someone would want to call a routine synchronously or asynchronously. This was a question about how to do it. If you provide helpful advice about the nature of the api while providing the answer, that is helpful, but if you don't provide an answer, why bother replying. (I guess I should really head my own advice.) Feb 4, 2016 at 19:30
129

One way to achieve this is to wrap the API call into a promise and then use await to wait for the result.

// Let's say this is the API function with two callbacks,
// one for success and the other for error.
function apiFunction(query, successCallback, errorCallback) {
    if (query == "bad query") {
        errorCallback("problem with the query");
    }
    successCallback("Your query was <" + query + ">");
}

// Next function wraps the above API call into a Promise
// and handles the callbacks with resolve and reject.
function apiFunctionWrapper(query) {
    return new Promise((resolve, reject) => {
        apiFunction(query,(successResponse) => {
            resolve(successResponse);
        }, (errorResponse) => {
            reject(errorResponse);
        });
    });
}

// Now you can use await to get the result from the wrapped api function
// and you can use standard try-catch to handle the errors.
async function businessLogic() {
    try {
        const result = await apiFunctionWrapper("query all users");
        console.log(result);
        
        // the next line will fail
        const result2 = await apiFunctionWrapper("bad query");
    } catch(error) {
        console.error("ERROR:" + error);
    }
}

// Call the main function.
businessLogic();

Output:

Your query was <query all users>
ERROR:problem with the query
4
  • 9
    This is a very well done example of wrapping a function with a callback so you can use it with async/await I dont often need this, so have trouble remembering how to handle this situation, I'm copying this for my personal notes/references. Apr 19, 2020 at 23:31
  • Very well written example. easy to understand for beginners like me. happy to recover from async/await callback hell
    – Ravi Yadav
    Sep 27, 2021 at 6:39
  • Good job. This is exactly what I needed, as I got an API function call that uses callbacks and I had no idea how to "await" its result.
    – Jax Teller
    Oct 12, 2021 at 8:12
  • This is the exact use case when one wants to wait till a callback is called: create an async API around a callback mechanism. Great spot, and great description. Feb 15 at 13:07
25

check this: https://github.com/luciotato/waitfor-ES6

your code with wait.for: (requires generators, --harmony flag)

function* (query) {
  var r = yield wait.for( myApi.exec, 'SomeCommand');
  return r;
}
0
12

If you don't want to use call back then you can Use "Q" module.

For example:

function getdb() {
    var deferred = Q.defer();
    MongoClient.connect(databaseUrl, function(err, db) {
        if (err) {
            console.log("Problem connecting database");
            deferred.reject(new Error(err));
        } else {
            var collection = db.collection("url");
            deferred.resolve(collection);
        }
    });
    return deferred.promise;
}


getdb().then(function(collection) {
   // This function will be called afte getdb() will be executed. 

}).fail(function(err){
    // If Error accrued. 

});

For more information refer this: https://github.com/kriskowal/q

7

It's 2020 and chances are the API has already a promise-based version that works with await. However, some interfaces, especially event emitters will require this workaround:

// doesn't wait
let value;
someEventEmitter.once((e) => { value = e.value; });
// waits
let value = await new Promise((resolve) => {
  someEventEmitter.once('event', (e) => { resolve(e.value); });
});

In this particular case it would be:

let response = await new Promise((resolve) => {
  myAPI.exec('SomeCommand', (response) => { resolve(response); });
});

Await has been in new Node.js releases for the past 3 years (since v7.6).

1
  • 1
    Exactly what I needed - for some reason it's easy to get lost in thinking about how to do this with callbacks when something as simple as wrapping an event emitter in a Promise is all that's required
    – lys
    Nov 8, 2022 at 8:13
5

Note: This answer should probably not be used in production code. It's a hack and you should know about the implications.

There is the uvrun module (updated for newer Nodejs versions here) where you can execute a single loop round of the libuv main event loop (which is the Nodejs main loop).

Your code would look like this:

function(query) {
  var r;
  myApi.exec('SomeCommand', function(response) {
    r = response;
  });
  var uvrun = require("uvrun");
  while (!r)
    uvrun.runOnce();
  return r;
}

(You might alternative use uvrun.runNoWait(). That could avoid some problems with blocking, but takes 100% CPU.)

Note that this approach kind of invalidates the whole purpose of Nodejs, i.e. to have everything async and non-blocking. Also, it could increase your callstack depth a lot, so you might end up with stack overflows. If you run such function recursively, you definitely will run into troubles.

See the other answers about how to redesign your code to do it "right".

This solution here is probably only useful when you do testing and esp. want to have synced and serial code.

5

Since node 4.8.0 you are able to use the feature of ES6 called generator. You may follow this article for deeper concepts. But basically you can use generators and promises to get this job done. I'm using bluebird to promisify and manage the generator.

Your code should be fine like the example below.

const Promise = require('bluebird');

function* getResponse(query) {
  const r = yield new Promise(resolve => myApi.exec('SomeCommand', resolve);
  return r;
}

Promise.coroutine(getResponse)()
  .then(response => console.log(response));
1

supposing you have a function:

var fetchPage(page, callback) {
   ....
   request(uri, function (error, response, body) {
        ....
        if (something_good) {
          callback(true, page+1);
        } else {
          callback(false);
        }
        .....
   });
};

you can make use of callbacks like this:

fetchPage(1, x = function(next, page) {
if (next) {
    console.log("^^^ CALLBACK -->  fetchPage: " + page);
    fetchPage(page, x);
}
});
1
  • I thought I understood recursion until I've read this code
    – Raul Luna
    Jul 1, 2022 at 10:41
0

Using async and await it is lot more easy.

router.post('/login',async (req, res, next) => {
i = await queries.checkUser(req.body);
console.log('i: '+JSON.stringify(i));
});

//User Available Check
async function checkUser(request) {
try {
    let response = await sql.query('select * from login where email = ?', 
    [request.email]);
    return response[0];

    } catch (err) {
    console.log(err);

  }

}
1
  • 2
    The API used in the question doesn't return a promise, so you would need to wrap it in one first … like this answer did two years ago.
    – Quentin
    May 2, 2020 at 9:10
-1

That defeats the purpose of non-blocking IO -- you're blocking it when it doesn't need blocking :)

You should nest your callbacks instead of forcing node.js to wait, or call another callback inside the callback where you need the result of r.

Chances are, if you need to force blocking, you're thinking about your architecture wrong.

5
  • I had a suspicion i had this around backwards.
    – Chris
    Feb 15, 2011 at 22:39
  • 33
    Chances are, I just want to write a quick script to http.get() some URL and console.log() its contents. Why do I have to jump over backwards to do that in Node? Dec 21, 2013 at 11:42
  • 6
    @DanDascalescu: And why do I have to declare type signatures to do it in static languages? And why do I have to put it in a main-method in C-like languages? And why do I have to compile it in a compiled language? What you're questioning is a fundamental design decision in Node.js. That decision has pros and cons. If you don't like it, you can use another language that fits your style better. That's why we have more than one.
    – Jakob
    Dec 17, 2014 at 14:46
  • @Jakob: the solutions you've listed are suboptimal indeed. That doesn't mean there aren't good ones, such as Meteor's server-side use of Node in fibers, which eliminates the callback hell problem. Dec 17, 2014 at 14:54
  • 18
    @Jakob: If the best answer to "why does ecosystem X make common task Y needlessly difficult?" is "if you don't like it, don't use ecosystem X," then that is a strong sign that the designers and maintainers of ecosystem X are prioritizing their own egos above the actual usability of their ecosystem. It's been my experience that the Node community (in contrast to the Ruby, Elixir, and even PHP communities) goes out of its way to make common tasks difficult. Thank you SO MUCH for offering yourself as a living example of that antipattern.
    – Jazz
    May 24, 2016 at 21:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.