2

I'm working on a standard server application. The processing of each request is broken down into several stages (the second stage requires the first to be finished, and so on). Now, one of these stages takes a rather long time but can itself be broken down into several dozen or so tasks, which do not depend on each other, and hence are parallelizable. I would like to add a thread pool to SlowStageService, and am wondering how best to size it. It would be best for the thread pool to have at least one vacancy most of the time, which implies that it must process tasks as fast or faster than the server is receiving requests; this yields a reasonable lower bound on the size of the thread pool. However, I would like to be generous with the sizing, since many of the numbers involved in the calculation are likely to change.

So my question is: What is the downside of making my pool too large (say by a factor of 3 or 4), and having many idle threads? I understand it wastes some resources, but it doesn't actually reserve any CPU and make it thereby unavailable to other requests, right? How much slack in practice one can usually have? Say I've calculated 6 threads is the minimum I need; is it safe to go to say 12 on a 24-core box under medium load?

As I'm writing this, it sounds like maybe what I need is a ThreadPoolExecutor with a fairly narrow (3 or 4x) range?

1 Answer 1

6

You should refer to Section 8.2 from the book Java Concurrency In Practice by Brian Goetz.

If your stages are compute intensive stages and have very little I/O then an optimal size of the threadpool = Number of CPU cores + 1 (so in your case it would be 25). If however there are I/O bound tasks within your stages then the best applicable size for a threadpool to achieve optimum response times depends on various factors :

  1. Number of CPUs - N
  2. Target utilization of the CPU - UC
  3. The ratio of the wait time (blocking state) for I/O operations (W) to the compute time (C).

NUM_THREADS = N * UC * (1+(W/C))

In addition to the above metrics other factors that affect the calculation of the pool size are memory requirements, size of connection pools, file handles and socket handles.

Given this above theory behind sizing threadpools; My experience suggests that the best way to come up to an optimal pool size is by profiling the application at various work loads and come up with the pool sizes for light, medium and heavy workload sizes.

Also, never ever hard-code the max threadpool size - it should always be a configurable parameter so that it can be adjusted in the field as per the workloads encountered.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.