8

I need to run in the same environment a script workflow.py that use python 2.7 and another script stepA.py that use python 3.7.

Let suppose that stepA.py is called from workflow.py, one possibility is to setup two conda env, one with python2.7 and another with python3.7, run workflow.py in the py2env and write something like

subprocess.run('bash -c "source activate py3env; stepA.py"', shell=True)

in workflow.py to launch stepA.py.

I do not like this solution, instead I would like to modify the first row workflow.py and stepA.py by indicating the python version to be used, e.g.

#!/usr/bin/env python2.7

instead of simply

#!/usr/bin/env python

this second solution seems to me more atomic and dry.

I tried something like

$ conda --version
conda 4.7.12
$ conda create -n py3env python=3.8
[...]
$ conda env list
# conda environments:
base                  *  /sto1/ref/miniconda2
py3env                   /sto1/ref/miniconda2/envs/py3env
$ cat ./test.py
#!/usr/bin/env conda run -n py3env python
import os,sys
print sys.executable
print os.__file__
$ ./test.py
/usr/bin/env: conda run -n pheatmap_1 python: No such file or directory

So the conda run solution seem not working.

What is the proper way to set the environment for inside the script (in the #! row or with other strategies)?

5
  • Duplicate question: How do I activate a conda env in a subshell?. In your case, you are missing the -n flag to name the env in the conda run.
    – merv
    Commented Dec 6, 2019 at 15:25
  • Hi @merv, I added the -n but still do not work, see in the edited question the error it gives to me.
    – mox
    Commented Dec 13, 2019 at 14:55
  • It looks like it's not finding Conda? Is the shell you are running the script from set up to use Conda? I.e., you've run conda init and conda is defined? Please add how you launch your script. E.g., ./workflow.py?
    – merv
    Commented Dec 13, 2019 at 15:03
  • @merv I edited the question and added a complete test.
    – mox
    Commented Dec 17, 2019 at 11:01
  • I'll note that I ran into "conda run" not working when I installed the latest conda on a new system roughly when this question was posted. Reading the docs, it looked like they disabled that ability. Dialing back the conda version let me run "conda run" again, which is critical for my workflow because I have aspects of different codes that must run together, but interfere with each other so have to be run in different conda environments. Commented Aug 12, 2020 at 18:30

1 Answer 1

2

You should be giving the full path to your python which should be given as executable

for eg:

#!/usr/bin/env /user/intelpython/latest/envs/py3env/bin/python
code

In this /user/intelpython/latest/envs/py3env/bin/python(My full path to a particular python) can be either python3 or 2 To get the absoulte path activate an environment and use which python command

4
  • Thank you for the int, but hardcoding the path will make this solution not portable. I wonder if there is a better solution.
    – mox
    Commented Dec 6, 2019 at 14:37
  • Okk... Makes sense
    – ArunJose
    Commented Dec 6, 2019 at 14:38
  • Is python2.7 the right name of your python 2 executable and is it added to the path
    – ArunJose
    Commented Dec 6, 2019 at 14:46
  • You can check with command if "which python2.7" return it's path
    – ArunJose
    Commented Dec 6, 2019 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.