-5

In our project, I create lots of java threads to do some continuous work.

In most time these threads are sleeping so total cpu cost is low.

with 1168 threads , system overload should low than 0.20

But then I noticed that if I create more than 1000 thread with jvm, then I will get

fork: retry: Resource temporarily unavailable

when i wanna connect vm (I mean VM , not jvm) by ssh , this seem's a serious problem about system resource ..

And I'm very worried about what would happen then in my program..

9
  • 3
    You are just running out of available threads/processes. Why on earth do you need over 1000 threads?
    – thkala
    Jan 26, 2013 at 16:39
  • Seems to me that you are exceeding linux kernel parameters. Most of them are very tunable. See: stackoverflow.com/questions/9361816/…
    – Gray
    Jan 26, 2013 at 16:41
  • @thkala yes I have really lots of seperated work should be done in seperated thread. And as i said , lots of time , these threads is waiting for system IO
    – jackalope
    Jan 26, 2013 at 16:44
  • @jackalope: What do your logs say? Which resource are you running out of? Process/thread slots? Memory?
    – thkala
    Jan 26, 2013 at 16:49
  • 1
    @jackalope: Launching a bazillion threads is rarely the answer. About double the number of processors should be enough - you should use asynchronous I/O for everything else...
    – thkala
    Jan 26, 2013 at 16:50

2 Answers 2

4

So many threads do not look like a good design. I propose to receive all events that currently wake up your threads in one loop that could immediately convert them into Futures or Runnables and post to some ExecutorService. This service could manage a thread pool for you. Standard approach with ServerSocket and Socket is implemented very much that way.

7
  • yes i have lots of ExecutorService actually......
    – jackalope
    Jan 26, 2013 at 16:49
  • You only one that should handle all tasks. You can select implementation that will allocate a clever number of additional threads as required. Jan 26, 2013 at 16:50
  • yes that may be a good idea!
    – jackalope
    Jan 26, 2013 at 16:52
  • @AudriusMeškauskas: won't the thread pool stall if the tasks are I/O bound as the OP mentioned? I am not aware of any standard Java thread pool that performs scheduling and load balancing on its own...
    – thkala
    Jan 26, 2013 at 17:02
  • ThreadPoolExecutor Jan 26, 2013 at 18:22
1

As mentioned here, the total number of threads and processes on a Linux system is limited, both directly and indirectly via other resource limits. You are apparently crossing that limit and the fork() call needed to create a new SSH session fails.

Why are you creating so many threads, anyway?

You rarely gain any performance advantage when the number of threads significantly exceeds the number of available processor cores, and each thread does have a minimum set of memory requirements that is not negligible.

If you are dealing with some form of the C10K problem, you should consider combining multiple threads with the use of asynchronous I/O.

3
  • noticed this value in my vm equals 124413 , any suggestion??
    – jackalope
    Jan 26, 2013 at 16:44
  • @jackalope: it's not just that number. The limits imposed by the memory subsystem are probably more relevant. In any case, it seems to me that you need to fix your application - increasing the limits only delays the inevitable. Is there anything in your logs that can help you track down the culprit?
    – thkala
    Jan 26, 2013 at 16:48
  • sorry but no logs found in /var/log/messages
    – jackalope
    Jan 26, 2013 at 16:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.