121

So I have an understanding of how Node.js works: it has a single listener thread that receives an event and then delegates it to a worker pool. The worker thread notifies the listener once it completes the work, and the listener then returns the response to the caller.

My question is this: if I stand up an HTTP server in Node.js and call sleep on one of my routed path events (such as "/test/sleep"), the whole system comes to a halt. Even the single listener thread. But my understanding was that this code is happening on the worker pool.

Now, by contrast, when I use Mongoose to talk to MongoDB, DB reads are an expensive I/O operation. Node seems to be able to delegate the work to a thread and receive the callback when it completes; the time taken to load from the DB does not seem to block the system.

How does Node.js decide to use a thread pool thread vs the listener thread? Why can't I write event code that sleeps and only blocks a thread pool thread?

2
  • @Tobi - I have seen that. It still doesn't answer my question. If the work was on another thread, the sleep would only affect that thread and not the listener as well.
    – Haney
    Commented Mar 25, 2014 at 19:28
  • 11
    A genuine question, where you try to understand something by yourself, and when you can't find an exit to the maze, you ask for help. Commented May 27, 2015 at 23:29

4 Answers 4

281

Your understanding of how node works isn't correct... but it's a common misconception, because the reality of the situation is actually fairly complex, and typically boiled down to pithy little phrases like "node is single threaded" that over-simplify things.

For the moment, we'll ignore explicit multi-processing/multi-threading through cluster and webworker-threads, and just talk about typical non-threaded node.

Node runs in a single event loop. It's single threaded, and you only ever get that one thread. All of the javascript you write executes in this loop, and if a blocking operation happens in that code, then it will block the entire loop and nothing else will happen until it finishes. This is the typically single threaded nature of node that you hear so much about. But, it's not the whole picture.

Certain functions and modules, usually written in C/C++, support asynchronous I/O. When you call these functions and methods, they internally manage passing the call on to a worker thread. For instance, when you use the fs module to request a file, the fs module passes that call on to a worker thread, and that worker waits for its response, which it then presents back to the event loop that has been churning on without it in the meantime. All of this is abstracted away from you, the node developer, and some of it is abstracted away from the module developers through the use of libuv.

As pointed out by Denis Dollfus in the comments (from this answer to a similar question), the strategy used by libuv to achieve asynchronous I/O is not always a thread pool, specifically in the case of the http module a different strategy appears to be used at this time. For our purposes here it's mainly important to note how the asynchronous context is achieved (by using libuv) and that the thread pool maintained by libuv is one of multiple strategies offered by that library to achieve asynchronicity.


On a mostly related tangent, there is a much deeper analysis of how node achieves asynchronicity, and some related potential problems and how to deal with them, in this excellent article. Most of it expands on what I've written above, but additionally it points out:

  • Any external module that you include in your project that makes use of native C++ and libuv is likely to use the thread pool (think: database access)
  • libuv has a default thread pool size of 4, and uses a queue to manage access to the thread pool - the upshot is that if you have 5 long-running DB queries all going at the same time, one of them (and any other asynchronous action that relies on the thread pool) will be waiting for those queries to finish before they even get started
  • You can mitigate this by increasing the size of the thread pool through the UV_THREADPOOL_SIZE environment variable, so long as you do it before the thread pool is required and created: process.env.UV_THREADPOOL_SIZE = 10;

If you want traditional multi-processing or multi-threading in node, you can get it through the built in cluster module or various other modules such as the aforementioned webworker-threads, or you can fake it by implementing some way of chunking up your work and manually using setTimeout or setImmediate or process.nextTick to pause your work and continue it in a later loop to let other processes complete (but that's not recommended).

Please note, if you're writing long running/blocking code in javascript, you're probably making a mistake. Other languages will perform much more efficiently.

24
  • 1
    Holy crap, this completely clears it up for me. Thank you so much @Jason!
    – Haney
    Commented Mar 25, 2014 at 19:55
  • 7
    No problem :) I found myself where you are not too long ago, and it was tough to come to a well defined answer because on one side you have C/C++ devs for whom the answer is obvious, and on the other you have typical web devs who haven't delved too deeply into these sorts of questions before. I'm not even sure my answer is 100% technically correct when you get down to the C level, but it's right in the broad strokes.
    – Jason
    Commented Mar 25, 2014 at 20:00
  • 4
    Using the thread pool for network requests would be a huge resource waste. According to this question "It does the async network I/O based on the async I/O interfaces in different platforms, such as epoll, kqueue and IOCP, without a thread pool" -- which makes sense. Commented Dec 16, 2014 at 14:52
  • 1
    How is nodejs able to handle concurrent connections, provided it runs on a single thread? If I get 1000 concurrent requests, will each request be handled one after the another? If yes, will not there be a unwanted lag in the response sent to each request? What will happen if it receives 1000 * 1000 concurrent requests? Commented May 4, 2016 at 7:01
  • 1
    @Jason What my perception (without the use of any cluster), is that for 1000 simultaneous HTTP requests each request will be handled one after the another so the last request (1000th request) will be served late in comparison to the request served first. (A lag seems unavoidable here). Commented May 4, 2016 at 12:17
23

So I have an understanding of how Node.js works: it has a single listener thread that receives an event and then delegates it to a worker pool. The worker thread notifies the listener once it completes the work, and the listener then returns the response to the caller.

This is not really accurate. Node.js has only a single "worker" thread that does javascript execution. There are threads within node that handle IO processing, but to think of them as "workers" is a misconception. There are really just IO handling and a few other details of node's internal implementation, but as a programmer you cannot influence their behavior other than a few misc parameters such as MAX_LISTENERS.

My question is this: if I stand up an HTTP server in Node.js and call sleep on one of my routed path events (such as "/test/sleep"), the whole system comes to a halt. Even the single listener thread. But my understanding was that this code is happening on the worker pool.

There is no sleep mechanism in JavaScript. We could discuss this more concretely if you posted a code snippet of what you think "sleep" means. There's no such function to call to simulate something like time.sleep(30) in python, for example. There's setTimeout but that is fundamentally NOT sleep. setTimeout and setInterval explicitly release, not block, the event loop so other bits of code can execute on the main execution thread. The only thing you can do is busy loop the CPU with in-memory computation, which will indeed starve the main execution thread and render your program unresponsive.

How does Node.js decide to use a thread pool thread vs the listener thread? Why can't I write event code that sleeps and only blocks a thread pool thread?

Network IO is always asynchronous. End of story. Disk IO has both synchronous and asynchronous APIs, so there is no "decision". node.js will behave according to the API core functions you call sync vs normal async. For example: fs.readFile vs fs.readFileSync. For child processes, there are also separate child_process.exec and child_process.execSync APIs.

Rule of thumb is always use the asynchronous APIs. The valid reasons to use the sync APIs are for initialization code in a network service before it is listening for connections or in simple scripts that do not accept network requests for build tools and that kind of thing.

18
  • 1
    Where are these asynchronous APIs coming from? I get what you're saying, but whoever wrote this APIs opted into IOCP/async. How did they choose to do this?
    – Haney
    Commented Mar 25, 2014 at 19:41
  • 3
    His question is how he would write his own time intensive code and not block.
    – Jason
    Commented Mar 25, 2014 at 19:46
  • 1
    Yes. Node provides basic UDP, TCP, and HTTP networking. It provides ONLY asynchronous "pool-based" APIs. All node.js code in the world without exception uses these pool-based asynchronous APIs as there are simply all that is available. Filesystem and child processes are a different story, but networking is consistently asynchronous. Commented Mar 25, 2014 at 19:56
  • 5
    Careful, Peter, lest you be the proverbial pot to his kettle. He wants to know how the writers of the network API did it, not how people who use the network API do it. I eventually gained an understanding of how node behaves re: non-blocking events because I wanted to write my own non-blocking code that has nothing to do with networking or any of the other built in asynchronous APIs. It's pretty clear David wants to do the same.
    – Jason
    Commented Mar 25, 2014 at 19:56
  • 2
    Node doesn't use thread pools for IO, it uses native non-blocking IO, the only exception is fs, as far as I know Commented Mar 25, 2014 at 20:20
16

Thread pool how when and who used:

First off when we use/install Node on a computer, it starts a process among other processes which is called node process in the computer, and it keeps running until you kill it. And this running process is our so-called single thread.

enter image description here

So the mechanism of single thread it makes easy to block a node application but this is one of the unique features that Node.js brings to the table. So, again if you run your node application, it will run in just a single thread. No matter if you have 1 or million users accessing your application at the same time.

So let's understand exactly what happens in the single thread of nodejs when you start your node application. At first the program is initialized, then all the top-level code is executed, which means all the codes that are not inside any callback function (remember all codes inside all callback functions will be executed under event loop).

After that, all the modules code executed then register all the callback, finally, event loop started for your application.

enter image description here

So as we discuss before all the callback functions and codes inside those functions will execute under event loop. In the event loop, loads are distributed in different phases. Anyway, I'm not going to discuss about event loop here.

Well for the sack of better understanding of Thread pool I a requesting you to imagine that in the event loop, codes inside of one callback function execute after completing execution of codes inside another callback function, now if there are some tasks are actually too heavy. They would then block our nodejs single thread. And so, that's where the thread pool comes in, which is just like the event loop, is provided to Node.js by the libuv library.

So the thread pool is not a part of nodejs itself, it's provided by libuv to offload heavy duties to libuv, and libuv will execute those codes in its own threads and after execution libuv will return the results to the event in the event loop.

enter image description here

Thread pool gives us four additional threads, those are completely separate from the main single thread. And we can actually configure it up to 128 threads.

So all these threads together formed a thread pool. and the event loop can then automatically offload heavy tasks to the thread pool.

The fun part is all this happens automatically behind the scenes. It's not us developers who decide what goes to the thread pool and what doesn't.

There are many tasks goes to the thread pool, such as

-> All operations dealing with files
->Everyting is related to cryptography, like caching passwords.
->All compression stuff
->DNS lookups
1
  • Do you mind clarifying one thing: so when we create a worker thread are these threads being utilized from thread pool or being spawned a new?
    – MiKr13
    Commented Feb 1, 2023 at 14:04
0

This misunderstanding is merely the difference between pre-emptive multi-tasking and cooperative multitasking...

The sleep turns off the entire carnival because there is really one line to all the rides, and you closed the gate. Think of it as "a JS interpreter and some other things" and ignore the threads...for you, there is only one thread, ...

...so don't block it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.