6

I run a java program which is a thread executor program that inserts thousands of documents to a table in mongodb. I get the following error

Exception in thread "pool-1-thread-301" com.mongodb.MongoWaitQueueFullException: Too many threads are already waiting for a connection. Max number of threads (maxWaitQueueSize) of 500 has been exceeded.
    at com.mongodb.PooledConnectionProvider.get(PooledConnectionProvider.java:70)
    at com.mongodb.DefaultServer.getConnection(DefaultServer.java:73)
    at com.mongodb.BaseCluster$WrappedServer.getConnection(BaseCluster.java:221)
    at com.mongodb.DBTCPConnector$MyPort.getConnection(DBTCPConnector.java:508)
    at com.mongodb.DBTCPConnector$MyPort.get(DBTCPConnector.java:456)
    at com.mongodb.DBTCPConnector.getPrimaryPort(DBTCPConnector.java:414)
    at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:176)
    at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:159)
    at com.mongodb.DBCollection.insert(DBCollection.java:93)
    at com.mongodb.DBCollection.insert(DBCollection.java:78)
    at com.mongodb.DBCollection.insert(DBCollection.java:120)
    at ScrapResults103$MyRunnable.run(MyProgram.java:368)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:695)

How can I resolve this? Please help me.

4
  • How much should I use? Is the 500 threads is from my Java program or mongodb limit?
    – sofs1
    Aug 17, 2014 at 7:08
  • 1
    Your program is running 500 threads. You probably are using a type of executor that creates a separate thread for every task. (A typical number would be 5-20ish depending on the task)
    – user253751
    Aug 17, 2014 at 22:36
  • @user3705478 Have you ever managed to solve this ? I have a customer having the same issue, and it would be great to have some feedback on this. Aug 30, 2016 at 11:46
  • Please don't solve the problem by raising the number of threads. There's a better solution now (2018): reactivemongo.org
    – alextsil
    Jan 27, 2018 at 7:22

3 Answers 3

7

You need to check what is the connections per host value which you have given while setting up connection (looking at the exception I think you would have set it to 500).

MongoClientOptions.Builder builder = new MongoClientOptions.Builder();
builder.connectionsPerHost(200);
MongoClientOptions options = builder.build();
mongoClient = new MongoClient(URI, connectionOptions);

An ideal way of setting the connections per host would be by trial and error but you need to make sure that the value which you set should not exceed the number of connections you can have by opening the mongo shell and executing:

db.serverStatus().connections.available

2
  • How to do this in xml based configration?
    – Shashank
    Sep 5, 2017 at 9:28
  • I'd suggest as @povis said to investigate more the reasons why it's overflowing the connections as 500 is already a big number.
    – Bill'o
    Aug 28, 2020 at 9:55
4

you are in maxWaitQueueSize limit , so increase multiplier ;)

 MongoClientOptions options = MongoClientOptions.builder()
                .threadsAllowedToBlockForConnectionMultiplier(10)
                .build();

 MongoClient mongo = new MongoClient("127.0.0.1:27017", options);
 //run 2000 threads and use database ;)
1
  • How to do this in xml based configration?
    – Shashank
    Sep 5, 2017 at 9:28
4

waitQueueMultiple is the product of maxConnectionPoolSize and threadsAllowedToBlockForConnectionMultiplier hence you can modify one of these three options to tune your app in MongoClientOptions with corresponding values and consume it to your MongoClient as an argument how it was done above here (marked as an answer) https://stackoverflow.com/a/25347310/2852528

BUT

I strongly recommend analysing your code first (where it communicates to the DB), and if no optimization is available (e.g. caching, using aggregation, paging etc.) then go ahead and change the options

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.