I'm working on a standard server application. The processing of each request is broken down into several stages (the second stage requires the first to be finished, and so on). Now, one of these stages takes a rather long time but can itself be broken down into several dozen or so tasks, which do not depend on each other, and hence are parallelizable. I would like to add a thread pool to SlowStageService
, and am wondering how best to size it. It would be best for the thread pool to have at least one vacancy most of the time, which implies that it must process tasks as fast or faster than the server is receiving requests; this yields a reasonable lower bound on the size of the thread pool. However, I would like to be generous with the sizing, since many of the numbers involved in the calculation are likely to change.
So my question is: What is the downside of making my pool too large (say by a factor of 3 or 4), and having many idle threads? I understand it wastes some resources, but it doesn't actually reserve any CPU and make it thereby unavailable to other requests, right? How much slack in practice one can usually have? Say I've calculated 6 threads is the minimum I need; is it safe to go to say 12 on a 24-core box under medium load?
As I'm writing this, it sounds like maybe what I need is a ThreadPoolExecutor
with a fairly narrow (3 or 4x) range?