35

Usually when I use mpirun, I can "overload" it, using more processors than there acctually are on my computer. For example, on my four-core mac, I can run mpirun -np 29 python -c "print 'hey'" no problem. I'm on another machine now, which is throwing the following error:

$ mpirun -np 25 python -c "print 'hey'"
--------------------------------------------------------------------------
There are not enough slots available in the system to satisfy the 25 slots 
that were requested by the application:
  python

Either request fewer slots for your application, or make more slots available
for use.
--------------------------------------------------------------------------

Why isn't "overclocking" mpirun working here? Is there a way I can overcome this error message and successfully run with more processors than are available?

6
  • You probably mean oversubscribe, rather than overload or overclock it. BTW, which MPI implementation do you use?
    – Harald
    Feb 29, 2016 at 16:50
  • @Harald I have gotten this question on a couple of mpi-related questions, and the answer is that I just don't know. Is it possible for a non-administrator to find this information?
    – kilojoules
    Feb 29, 2016 at 16:57
  • 1
    mpirun --version yields (Open MPI) 1.7.3
    – kilojoules
    Feb 29, 2016 at 17:04
  • That's pretty weird. I'm using OpenMPI 1.10 on a quad-core system and your mpirun command works like a charm.
    – Harald
    Feb 29, 2016 at 17:23
  • Could you try with this information regarding oversubscribing on openmpi using a hostfile ? open-mpi.org/faq/?category=running#oversubscribing
    – Harald
    Feb 29, 2016 at 17:25

2 Answers 2

58

Apparently oversubscribing can be attained using the "--oversubscribe" option with mpirun - did the trick for me with running torque/maui

3
  • 2
    This seems a lot nicer than the accepted answer. Note that the --oversubscribe flag is a feature of OpenMPI 3.x. This flag does not exist in OpenMPI 2.x, but here oversubscription is allowed by default.
    – jmd_dk
    Oct 20, 2017 at 21:06
  • 1
    Well... Openmpi 3.0 did not exist when OP raised the question. It's good to see that openmpi made oversubscription easier.
    – Harald
    Feb 18, 2018 at 12:00
  • both work but personally, I would go for this solution
    – Ashlou
    Jul 5, 2018 at 7:58
31

According to https://www.open-mpi.org/faq/?category=running#oversubscribing you could oversubscribe your node using a hostfile. Before proceeding, be careful that this way you can severely degrade the performance of the node. Also, if the system you use to run the application is using a queue system, this may not be valid.

First create a hostfile (named hostfile) containing

localhost slots=25

The simply run your application like

mpirun --hostfile hostfile -np 25 python -c "print 'hey'"
2
  • 3
    The link you referenced, specifically says one shouldn't state more slots than available in the machine. Slots should state the actual quantity of processors available (or less) and, to oversubscribe, the --oversubscribe flag should be used
    – griloHBG
    Aug 16, 2020 at 21:57
  • I meet with "Open RTE was unable to open the hostile xxx". But the file was there. Any help? Jan 2, 2022 at 9:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.