57

I would like to know the internals of the tomcat NIO connector. How exactly are threads used when we create a servlet that implements CometProcessor?Is it still one thread per connection?

From what I read, the conversation goes like this

  1. Client connects to a servlet

  2. Servlet hangs on to the connection till there is any data available to the connected client

  3. When data is ready , the server writes to the httpResponse and flushes it. This actually disconnects the connection?

  4. Client sends another request which the server again hangs onto..

How many thread are used when this keeps happening?

1
  • To which version of Tomcat do you refer? This kind of behavior is changing in later versions of Tomcat 7 and in Tomcat 8. Apr 19, 2014 at 11:15

4 Answers 4

60

NIO and Comet are completely unrelated: you can mix-and-match them.

Using the NIO (or APR for that matter) connector allows you to handle more requests with fewer threads due to the threading model. See http://tomcat.apache.org/tomcat-7.0-doc/config/http.html#Connector_Comparison for a comparison between the Connectors.

Comet (and Websocket) have a completely different dispatch model which requires a different application architecture and achieves higher throughput in a different way.

The scenario you pose in your question is the typical blocking one-thread-per-request model. In step 4, the Java BIO connector (which is the default up through Tomcat 7) will continue to wait for additional requests on the existing connector -- for keepalive HTTP requests. If the client does not set Connection:close on the previous request and does not close the connection, the thread will hang until the keepalive timeout is reached. If you use the NIO connector, the thread will go back into the thread pool immediately after the response is sent and you won't "waste" a thread on keepalive requests that may never arrive.

Comet/Websocket works entirely differently by delivering a message to a specially-written servlet (and optional filters) and the threads are only used when there are messages to send or data to be written.

UPDATE 2016-08-19

Tomcat 8.5 and 9.0 have completely dropped the BIO connector. This is because many of the new APIs and technologies (e.g. Websocket) require non-blocking semantics, and building a non-blocking service on top of a blocking API is very very difficult. The code required to get the job done was making the rest of the Tomcat code very ugly, etc. and so the decision was made to drop the BIO connector completely. So for Tomcat 8.5 and onward, only NIO, NIO2, and the APR-based connectors are available.

Note that, also with Tomcat 8.5 and 9.0, support for Comet has been dropped. Uses of Comet should all be replaced with Websocket which is a more standard protocol.

6
  • 4
    It sounds like NIO is a win-win, so why is BIO the default when APR is not available?
    – Jayen
    Sep 18, 2014 at 1:17
  • 4
    @Jayen BIO is the default connector for Tomcat up through 7.0.x. In Tomcat 8, NIO is the default connector. Sep 18, 2014 at 21:17
  • Any reason it wasn't the default on tomcat 6 or 7 (with java 7)?
    – Jayen
    Sep 19, 2014 at 19:22
  • 3
    @Jayen Yes, it was still experimental in Tomcat 6 and the decision to make NIO the default was made after Tomcat 7 was released. So, rather than changing the default connector in a point release and potentially causing all kinds of problems, the switch to a NIO-default was made in Tomcat 8. Anyone can explicitly change their connector to NIO at will... it's just the default that has changed and the NIO connector is quite stable at this point. Sep 22, 2014 at 17:26
  • I have a question about 'If you use the NIO connector, the thread will go back into the thread pool immediately after the response is sent and you won't "waste" a thread on keepalive requests that may never arrive.', it sound that in NIO, the keepalive is meaningless?
    – liam xu
    Feb 28, 2019 at 9:22
4

NIO use fewer thread, it means that the tcp/ip port use is fewer.

You know the port is 1 to 65534, so we can say that NIO can reach a higher TPS (Transactions Per Second) than BIO

I tested both protocol :HTTP/1.1 and org.apache.coyote.http11.Http11NioProtocol with same web-project, same host, and same server.xml but the protocol.

Use jmeter for test.

I set 1000 thread to run request, when HTTP/1.1 in a few minutes, the host use port is more than 30000 and the TPS is 300 only!

When org.apache.coyote.http11.Http11NioProtocol, the max count of use port is never overstep 3000 and the tps is more than 1200!

8
  • 2
    Please reread your post, it is very hard to understand. I made a few edits to help clear it up a bit. If you want to start a new paragraph you need 2 carriage returns. Dec 3, 2013 at 7:08
  • 1
    Transactions Per Second
    – user3122271
    May 9, 2014 at 10:59
  • 3
    TPS measurements are highly usage-dependent. NIO also requires more resources (CPU, specifically) than BIO. The best way to decide which connector to use is to benchmark your own application and see how it performs under different conditions and configurations. Sep 18, 2014 at 21:20
  • 5
    Also note that port use is irrelevant, here: each connector, regardless of the type (BIO, NIO, NIO2, APR) uses a single TCP/IP port. Feb 3, 2015 at 20:51
  • 1
    Are your benchmarks reproducible, do you mind sharing? I am assuming you enabled keep-alive on Jmeter client and not doing much CPU work in the servlet that gives NIO a better TPS.
    – kisna
    Dec 14, 2016 at 8:08
4

Adding late to this discussion - In this context of performance comparisons between blocking IO and Asynchronous NIO - an excellent read is "Old way of writing servers is new". In summary below thread per connection model was found to be better performing and easy to write as compared to the NIO version - contrary to the popular belief.

1

Here are two good articles on the NIO connector in case this helps someone considering the differences between BIO (request processing is bound to accept thread) & NIO (request processing is passed onto another worker thread) in Tomcat.

1
  • 1
    The tomcat-expert article is referring to the NIO AJP connector, not the http connector
    – avmohan
    Nov 22, 2016 at 18:46

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.