3

I have been trying to resolve problems to be able to run openmpi on multiple nodes.

Initially I had a problem with $PATH and $LD_LIBRARY_PATH variables not being updated from .bashrc file by openmpi session, so I manually added --prefix /path/to/openmpi to resolve this issue.

Turns out that even the anaconda path variables are not being loaded as well. So ultimately I need ~/.bashrc file to be sourced from my home directory. How can I do that? Can anyone help me out please?

UPDATE 01:

I wrote a simple shell script to check the version of python

python --version

and tried to run it with openmpi on local as well as remote machine as follows:

mpirun --prefix /home/usama/.openmpi --hostfile hosts -np 4 bash script

And it returns

Python 2.7.12
Python 3.6.8 :: Anaconda, Inc.
Python 3.6.8 :: Anaconda, Inc.
Python 2.7.12

Confirming my suspicion that whatever openmpi is doing to run remote processes doesn't invoke / set proper environment variables from the ~/.bashrc file. Any help from someone who has worked with multi-node openmpi?

UPDATE 02:

A simple ssh environment grep tell me that my environment variables are not updated which might be the cause of the problem. (I have even tried to set it up in ~/.ssh/environment file)

$ ssh remote-node env | grep -i path

It seems to be loading only the /etc/environment file with only basic paths setup. How to I rectify this?

11
  • How exactly are you attempting to run something? Are you sure Bash is being invoked?
    – tripleee
    May 5, 2019 at 9:16
  • Well the openmpi FAQ says that an ssh connection is made to remote servers to start new process. I am assuming this involves bash? The error I am getting is about python libraries not being installed on remote machine when they are installed using conda and environment variables are set in .bashrc
    – usamazf
    May 5, 2019 at 9:19
  • Again, please edit your question to indicate exactly how you run things. Noninteractive SSH will read $HOME/.ssh/environment on the destination but we really need to see the full picture before making specific recommendations.
    – tripleee
    May 5, 2019 at 9:29
  • 1
    I would call "don't" the only appropriate answer. Users generally maintain .bashrc assuming that it impacts only their interactive shells, and thus that they can define functions that override builtin commands at will, set unusual DEBUG traps or command_not_found hooks, and otherwise do things that can make scripts behave unpredictably. May 5, 2019 at 15:04
  • 1
    I didn't say there was no way, I said the way you're asking for is ill-advised. Better to use a separate dotfile rather than sharing one also used for interactive configuration, or -- better -- generating commands that explicitly set the values they need. May 6, 2019 at 10:41

1 Answer 1

1

maybe you should run like this.I guess. two ways help you!

first:

mpirun --prefix /home/usama/.openmpi --hostfile hosts -np 4 . ~/.bashrc && bash script

second:

## 1. add this line to the script
. ~/.bashrc

## 2. run command as you do
mpirun --prefix /home/usama/.openmpi --hostfile hosts -np 4 bash script

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.