17

After reading about the Tomcat NIO connector I still don't get one thing: is the nio connector beneficial if the application code is blocking, i.e. it blocks on reading from the database, on reading the file system, on calling external web services?

So, for example, you have a REST-like API that receives a request, reads something from the database, and returns a response. It doesn't use servlet 3 async, it just writes to the response.

I didn't find a full description of the thread pools used by the NIO connector, but I imagine it has a thread pool for handling the requests, so each request ends up in its own thread, which it can block.

If that's the case, are the benefits of NIO still there, or the blocking code diminishes the benefits of NIO (in terms of resource utilization)?

1

1 Answer 1

12

Is the nio connector beneficial if the application code is blocking...?

Yes, the NIO connector is built with the assumption that your app will block somewhere. The NIO connector basically has several socket placeholders and responds to new incoming requests until information starts getting written back.

I didn't find a full description of the thread pools used by the NIO connector

I think this is the start of your confusion. Tomcat NIO has a selector pool, not a thread pool (reference). The connector code polls each selector to see if it has incoming or outgoing bytes to send. In this sense, the selector for a given request will continue to receive information until there is enough to process the request with a Request/Response object that bridges the gap between synchronous I/O and asynchronous I/O (reference).

The polling code never blocks longer than the time it takes to serialize a packet of information, so it's free to handle new requests. The only real limitation is the amount of memory available to Tomcat. While there is a thread pool, the number of actual threads used are much lower than the number of connections the application can handle (reference).

While there are performance differences between Tomcat Connectors (reference), the difference in raw request/response time is pretty small when the servlet itself blocks. However, the difference in the number of simultaneous requests that Tomcat can handle is vastly different when you use non-blocking I/O.

8
  • 3
    So, what happens in this case with ThreadLocal? If socket is simulated and the same thread is shared among several request it can create mismatch when using ThreadLocal. But in fact this does not happen. How does Tomcat (and probably other containers) solve this problem?
    – AlexR
    Mar 2, 2015 at 18:14
  • 3
    @BerinLoritsch, although I agree with you concerning to using of ThreadLocal in web apps, but even some popular frameworks do this. For example Spring stores authentication and DB connection relevant information into thread local that causes issues when trying to perform some operations asynchronously. However when no async ops are done IMHO threadlocal works well, so it is very interesting how do they achieve this?
    – AlexR
    Mar 3, 2015 at 7:24
  • 5
    @AlexR, they have a thread per currently processed request. Tomcat can be listening to 1000 incoming requests, but only working on 100 at a time. The excess incoming requests are queued up and processed as soon as they can. The only difference is that all the request data is fully received before Tomcat uses a thread for the Servlet. Mar 3, 2015 at 13:42
  • 2
    It means that if (as Bozho said) the application performs REST request or turns to DB in the middle of its flow the thread is being blocked and is not reused for processing of other requests. It wold be cool if it is possible to return the thread to pool while such requests are being performed.
    – AlexR
    Mar 3, 2015 at 14:34
  • 3
    For that, you need the async servlet approach--which requires you to rethink a lot of things (i.e. you can't use ThreadLocal reliably any longer) Mar 3, 2015 at 15:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.