1

The objective of this question is to find the right choice of ThreadPool configuration to be used with Java7s CompletableFuture inside a Singleton pattern (i.e. Service layer) to make x number of concurrent calls.

A little of overview of the java application structure

  1. Java-based WebServices with a bunch of APIs powered by Jersey using Spring Framework's dependency injection, C3P0 connection pooling
  2. Controllers are all prototype/request scoped due to state, Transaction-enabled Service layer, Hibernate DAO layer (we have used Criteria's massively)

There are over a hundred (100) apis and then there is this 1 read-only aggregator api that combines the result of 10 other read-only api. The api has the following structure

@Controller
public class AggregatorController {
    @Autowired
    private ApplicationContext ctx;

    public AggregatorController() {}

    @GET
    public Response getAggregator() {
        AggregatedObject result = new AggregatedObject();

        // Instantiate ResourceA and hit it’s getById endpoint. These resources are request-scoped. 
        ResourceAResponse resA = (ResourceAResponse) ctx.getBean(ResourceA.class).getById(someId).getEntity();
        result.setResourceA(resA); 

        // Instantiate ResourceB and hit it’s getAll endpoint. 
        ResourceBResponse resB = (ResourceBResponse) ctx.getBean(ResourceB.class).getAll(someParentId).getEntity();
        result.setResourceB(resB); 

        //Repeat above 10x times for different getById and getAll endpoints. 

        return Response.ok(result).build();
    }
}

The above 10 endpoints are executed in synchronous fashion. I would like convert that to asynchronous and in order to do that I make use of Java 7s CompletableFuture.

  1. Create a Service layer and call it AsyncAggregatorServcie
  2. Wrap each endpoint call into a CompletableFuture (independent futures)
  3. Await for all CompletableFutures to complete and then return the response something as follow

A new service layer that wraps what was there in the above controller but, inside Java CompletableFuture.

@Service
public class AsyncAggregatorServcieImpl implements AsyncAggregatorServcie {
    @Autowired
    private ApplicationContext ctx;

    @Override
    public AggregatedObject getAggregatorAsynchronously() {
        AggregatedObject result = new AggregatedObject();

        // Instantiate ResourceA Future and hit it’s getById endpoint in a CompletableFuture
        CompletableFuture<Void> getResourceAFuture =  CompletableFuture.supplyAsync(() -> 
            (ResourceAResponse) ctx.getBean(ResourceA.class).getById(someId).getEntity()
        ).thenAcceptAsync(result -> result.setResourceA(result));

        // Instantiate ResourceB and hit it’s getAll endpoint. 
        CompletableFuture<Void> getResourceBFuture =  CompletableFuture.supplyAsync(() -> 
            (ResourceBResponse) ctx.getBean(ResourceB.class).getAll(someParentId).getEntity()
        ).thenAcceptAsync(result -> result.setResourceB(result));

        //Repeat above 10x times for different getById and getAll endpoints. There will be around 10 independent CompletableFuture

        // All the futures
        CompletableFuture<Void>[] futures = new CompletableFuture[] { getResourceAFuture, getResourceBFuture, … };
        CompletableFuture<Void> allFutures = CompletableFuture.allOf(futures);

        try {
            allFutures.get();
        } catch (CancellationException | ExecutionException e) {
            e.printStackTrace();
        } 

        return result;
    }
} 

Here are the different ways that getAggregatorAsynchronously can be configured

  1. As is above will make use of default ForkJoinPool
  2. Define a class-level ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(30);
  3. Define method-level ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(10); and then shutdown() or shutdownNow()
  4. Make use of CachedThreadPoolExecutor instead of FixedThreadPoolExecutor

#1 Stick Default ForkJoinPool

ForkJoinPool creates worker threads per CPU core and it's recommended that they should be used for CPU-intensive work rathe than I/O. The above endpoints make database calls so, using with ForkJoinPool option would mean using precious worker threads that specialize in CPU for I/O. Any suggestion?

#2 Class-level FixedThreadPool executor

A FixedThreadPool of size X threads once in a Singleton class. These X threads will be reserved and the rest of the application cannot use them when needed. This approach feels like wasting resources and also difficult to pick what's the right value for X.

#3 Method-level FixedThreadPool executor

Per getAggregatorAsynchronously invocation a FixedThreadPool of size X (i.e. assume X=5 or 10) is created. These threads are used to make 10 API. This seems to be ideal but, there are the following concerns

  • As the number of concurrent requests increase, the application might run out of threads, The C3P0 pool run out of connections
  • After completion of the method the shutdown() or shutdownNow() might fail to shutdown the pool of threads properly (might cause memory leak).

This approach seems ideal if and only if we can create a thread pool of dynamic size depending on availability of threads and database connections in the pool. Also gotta guarantee that thread pool is shutdown.

Any advise with this approach?

#4 Make use of CachedThreadPool

The CachedThreadPool is not recommended by online tutorials due to its performance. Unlike FixedThreadPool that queues requests when Threads are busy, I guess this thread pool keeps creating new threads for requests.

Taking into account the above scenarios, which approach is more performant as well as least likely to cause memory/resource leak to use?

I am new to Multithreading and CompletableFuture and I want to make sure that the right choice of ThreadPool executor and configuration is picked.

Thanks.

5
  • This is an awfully long question - and I'm sure the answer is going to be "it depends..." - is there any way you can boil it down?
    – theMayer
    Dec 2, 2017 at 3:10
  • Sorry about that. I wanted to share the details of how I am going to be using the thread pool. I could use any approach and it will work but, the question is what's the best option to go with taking into account how I will be using it. I guess those with experience of multithreading should be able to point the flaws with proposed approaches and recommend the best option.
    – Raf
    Dec 2, 2017 at 3:20
  • I have more experience than most with multi threading and programming in parallel, and I have no idea what your question is about. It's way too platform-specific, and the way you've written it, nobody but you can answer. Now to be fair, I don't work much in Java, but the design principles are all the same.
    – theMayer
    Dec 2, 2017 at 3:23
  • I don't know if I can make this question platform or technology agnostic. It is definitely java-specific question and depends on how the different thread pools work in java.
    – Raf
    Dec 2, 2017 at 3:34
  • @Raf - There is only one way to get a meaningful / reliable answer. Benchmark it.
    – Stephen C
    Dec 2, 2017 at 4:05

1 Answer 1

0

First of all, CompletableFuture was introduced in Java8 and not in Java7.

But coming back to your specific question, I will recommend to go with

2 Class-level FixedThreadPool executor.

You can start with 5 threads and test and optimize based on test results. This is because:

4 can cause OutOfMemory and other performance issues. For example, in case of heavy load too many threads will be opened waiting for db to respond.

There should definitely be a cap on number of threads that can be spawned for data retrieval from Database.

3 is ruled out for the same reasons.

1 is ruled out due to the reasons you mentioned. In addition the ForkJoin Pool is a common pool that would be shared at JVM level.

8
  • The point that ForkJoinPool uses a common pool doesn't really matter; it doesn't help overall throughput to create more threads; quite the opposite, only using the resources available avoids management overhead.
    – daniu
    Dec 2, 2017 at 7:50
  • Yes I agree with you but in this specific case where we are talking about blocking I/O it is best to keep a separate thread pool to do this I/O and let the main thread continue with CPU based tasks. Do you agree? Dec 2, 2017 at 7:54
  • Sure, the point in not using a common pool for I/O is entirely valid (but that was one of his initial reasons, I was just disputing your "in addition" part ;) ).
    – daniu
    Dec 2, 2017 at 7:59
  • I understand :) Dec 2, 2017 at 8:01
  • I also wanted to imply that there are other APIs in JDK that by default will use the common pool. For instance, if one use CompletableFuture API in some other class and doesn't provide an executor then by default it will also fall back on this common pool. There could be other scenarios as well. So, there is no guarantee that he will always get X threads for this specific work. And it is better to have separate pools tuned for specific work. Agree? Dec 2, 2017 at 8:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.