15

I have read a lot of article about how NodeJs works. But I still can not figure out exactly how the internal threads of Nodejs proceed IO operations.

In this answer https://stackoverflow.com/a/20346545/1813428 , he said there are 4 internal threads in the thread pool of NodeJs to process I/O operations . So what if I have 1000 request coming at the same time , every request want to do I/O operations like retrieve an enormous data from the database . NodeJs will deliver these request to those 4 worker threads respectively without blocking the main thread . So the maximum number of I/O operations that NodeJs can handle at the same time is 4 operations. Am I wrong?.

If I am right , where will the remaining requests will handle?. The main single thread is non blocking and keep driving the request to corresponding operators , so where will these requests go while all the workers thread is full of task? .

In the image below , all of the internal worker threads are full of task , assume all of them need to retrieve a lot of data from the database and the main single thread keep driving new requests to these workers, where will these requests go? Does it have a internal task queuse to store these requests?

enter image description here

3
  • How comfortable are you with C or C++?
    – slebetman
    Apr 2, 2015 at 8:13
  • 2
    Those 1000 requests comes through a single ethernet/wifi card. The hardware itself can only process bytestreams. There is no such thing as 1000 parallel network requests unless you have multiple ethernet cards trunked on your machine connected to multiple modems to your ISP (usually this means multiple internet accounts). What actually exists in the real world is 1000 network request taking turns sending bits down a single connection - do this fast enough and you have the illusion of 1000 parallel requests..
    – slebetman
    Jul 25, 2021 at 13:00
  • 1
    ... since this is the case (that in the real world there is normally no such thing as parallel requests) node.js avoids the memory and context-switch overhead of multiple threads that most people use (most people, not all - other languages have also started to become smart for example Java's Netty does all network processing on a single thread). Node.js waits in parallel instead of trying to execute code in parallel. Waiting requires exactly 0% CPU time.
    – slebetman
    Jul 25, 2021 at 13:02

3 Answers 3

13

The single, per-process thread pool provided by libuv creates 4 threads by default. The UV_THREADPOOL_SIZE environment variable can be used to alter the number of threads created when the node process starts, up to a maximum value of 1024 (as of libuv version 1.30.0).

When all of these threads are blocked, further requests to use them are queued. The API method to request a thread is called uv_queue_work.

This thread pool is used for any system calls that will result in blocking IO, which includes local file system operations. It can also be used to reduce the effect of CPU intensive operations, as @Andrey mentions.

Non-blocking IO, as supported by most networking operations, don't need to use the thread pool.

If the source code for the database driver you're using is available and you're able to find reference to uv_queue_work then it is probably using the thread pool.

The libuv thread pool documentation provides more technical details, if required.

0
4

In the image below , all of the internal worker threads are full of task , assume all of them need to retrieve a lot of data from the database and the main single thread keep driving new requests to these workers

This is not how node.js use those threads.

As per Node.js documentation, the threads are used like this:

enter image description here

All requests and responses are "handled" in the main thread. Your callbacks (and code after await) simply take turns to execute. The "loop" between the javascript interpreter and the "event loop" is usually just a while loop.

Apart from worker_threads that you yourself start there are only 4 things node.js use threads for: waiting for DNS response, disk I/O, the built-in crypto library and the built-in zip library. Worker_threads are the only places where node.js execute javascript outside the main thread. All other use of threads execute C/C++ code.

If you are want to know more then I've written several answers to related questions:

Node js architecture and performance

how node.js server is better than thread based server

node js - what happens to incoming events during callback excution

Does javascript process using an elastic racetrack algorithm

Is there any other way to implement a "listening" function without an infinite while loop?

1
  • Have you tired using multiple worker_thread under forked cluster instance? If crypto use more worker threads, then I suppose it'd work just fine? Sep 1, 2021 at 21:22
1

no, main use case for thread pool is offloading CPU intensive operations. IO is performed in one thread - you don't need multiple threads if you are waiting external data in parallel, and event loop is exactly a technique to organise execution flow so that you wait for as much as possible in parallel

Example: You need to send 100 emails with a question (y/n) and another one with number of answered "y". It takes about 30 second to write email and two hours on average for reply + 10 seconds to read response. You start by writing all 100 emails ( 50 minutes of time ), then you wait alert sound which wakes you up every time reply arrives, and as you receive answers you increase number of "y". in ~2 hours and 50 minutes you're done. This is example of async IO and event loop ( no thread pools )

Blocking example: send email, wait for answer, repeat. Takes 4 days (two if you can clone another you )

Async thread pool example: each response is in the language you don't know. You have 4 translator friends. You email text to them, and they email translated text back to you (Or, more accurate: you print text and put it into "needs translation" folder. Whenever translator available, text is pulled from folder)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.